Reading large amounts of data
Some applications require large amounts of data that have to be read by the database. Usually, two major cases can be distinguished:
- Reading large amounts of data and shipping them to the client
- Reading large amounts of data so that the server can process them
In this section, both the cases will be covered because they have slightly different requirements.
Making use of cursors
Cursors are a fundamental concept provided by all reasonable relational databases. However, cursors are often ignored and not used the way they should be, which leads to problems all along the way.
Why would anybody want to use a cursor, and what is it all about anyway? Consider the following example:
SELECT * FROM very_large_table;
For PostgreSQL, this is no problem at all. It will calculate the result and send it to the client server. However, the client may instantly crash because it will try to store all of the data in the memory. If the table is really large, the client may just not...
