top button
Flag Notify
    Connect to us
      Site Registration

Site Registration

MySQL: datafeed in one MySql table and output to another table

+3 votes
432 views

I have a datafeed which is constantly sent to a MySql table. The table grows constantly as the data feeds in. I would like to write a python script which process the data in this table and output the processed data to another table in another MySql database in real-time.

Which are the python libraries which are suitable for this purpose? Are there any useful sample code or project on the web that I can use as reference?

posted Jan 17, 2014 by Tarun Singhal

Share this question
Facebook Share Button Twitter Share Button LinkedIn Share Button

1 Answer

+1 vote

Is there a reason you do not want to move these rows with a mysql command? drop table if exists temp;insert into temp select * from source where ...;insert into target select * from temp;delete from source where id in (select id from temp);

answer Jan 17, 2014 by Dewang Chaudhary
Similar Questions
+3 votes
How to connect MYSQL using python

I am using Python 2.7, MySQl 5.5, mysqldb(driver) and trying to connect to the mysql db using Python.

What I have tried 

import MySQLdb
db = MySQLdb.connect(host="localhost",
                     user="root", # username
                      passwd="xxxxx", #  password
                      db="TEST") # name of the data base

Even tried 127.0.0.1 as local host 

#ERROR
Traceback (most recent call last):
  File "C:/Python27/connect.py", line 6, in <module>
    db="prakash") # name of the data base
  File "C:\Python27\lib\site-packages\MySQLdb\__init__.py", line 81, in Connect
    return Connection(*args, **kwargs)
  File "C:\Python27\lib\site-packages\MySQLdb\connections.py", line 187, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
  OperationalError: (2003, "Can't connect to MySQL server on 'localhost' (10061)")

I am new to python, please help how to proceed?

+1 vote
python script that connects to mySQL DB ask for a password

Is there a way to make python script that connects to mySQL DB ask for a password on the:

conn = mdb.connect(host, user)
0 votes
What is the proper wayto execute dynamic MySQL queries in Python

What is the proper way in which I can execute dynamic MySQL queries in Python? I want to do dynamic queries for both CREATE and INSERT statement.
Here is my attempted code:

sql="create table %s (%%s, %%s, %%s ... )" % (tablename,''.join(fields)+' '.join(types))
cur.execute(sql)

where 'field' is the fieldnames stored in a list and 'types' are the fieldtypes stored in a list.

+1 vote
Install MySQL-python failed?

$ python -m pip install MySQL-python

Collecting MySQL-python
 Downloading MySQL-python-1.2.5.zip (108kB)
 100% | –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ –ˆ| 112kB 260kB/s
 930 [main] python2.7 12948 child_info_fork::abort: address space needed by 'datetime.dll' (0x870000) is already occupied
 Error [Errno 11] Resource temporarily unavailable while executing command python setup.py egg_info
Exception:
Traceback (most recent call last):
 File "/usr/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
 status = self.run(options, args)
 File "/usr/lib/python2.7/site-packages/pip/commands/install.py", line 324, in run
 requirement_set.prepare_files(finder)
 File "/usr/lib/python2.7/site-packages/pip/req/req_set.py", line 380, in prepare_files
 ignore_dependencies=self.ignore_dependencies))
 File "/usr/lib/python2.7/site-packages/pip/req/req_set.py", line 634, in _prepare_file
 abstract_dist.prep_for_dist()
 File "/usr/lib/python2.7/site-packages/pip/req/req_set.py", line 129, in prep_for_dist
 self.req_to_install.run_egg_info()
 File "/usr/lib/python2.7/site-packages/pip/req/req_install.py", line 439, in run_egg_info
 command_desc='python setup.py egg_info')
 File "/usr/lib/python2.7/site-packages/pip/utils/__init__.py", line 667, in call_subprocess
 cwd=cwd, env=env)
 File "/usr/lib/python2.7/subprocess.py", line 390, in __init__
 errread, errwrite)
 File "/usr/lib/python2.7/subprocess.py", line 917, in _execute_child
 self.pid = os.fork()
OSError: [Errno 11] Resource temporarily unavailable

Anyone hit similar issue?

...