The problem

You have set up your database program as a daemon, so it runs persistently and can be called by users throughout the network. Based on what we have covered in earlier chapters, it retrieves all the records into the application process itself before processing each one as a list. You have debugged it thoroughly and so you know it works. Still, within a week of deployment, the service is jamming up and colleagues are drawing comparisons between your program's processing time and the speed of molasses in January. What happened?

To understand the problem, it is worth revisiting the process by which a query is processed in MySQLdb. After making the connection, one passes an execute() call according to the cursor object—cursor.execute() for an instance of MySQLdb.Cursor that is named cursor. When that execute() call is passed, MySQLdb does not return the results, but returns the number of rows affected by the query (the results are retrieved, but not immediately returned). Consider...