Modifying data in a database
It is not unusual for data integrations to be simply about getting data from one system, modifying it, then passing it onto another system to consume the classic Extract, Transform and Load (ETL) scenario. However, sometimes we will also need to modify the data in the database that we are sourcing data from or, indeed, the database we are sending data to.
To illustrate this, imagine we have a database table containing customer orders. We need to extract the data out of this table and send it to another system so that the orders can be fulfilled. However, most of the time we will need to be selective about the data that we send over. We cannot select all rows from the table, because presumably some will have already been sent. We could filter our result set by date/time so that the job only runs once per hour and each time it sends data created in the last hour. This would probably work perfectly well for most scenarios, but even this kind of filter is not quite...
