11.1 Description
In the previous chapters, particularly those starting with Chapter 9, Project 3.1: Data Cleaning Base Application, the question of ”persistence” was dealt with casually. The previous chapters all wrote the cleaned samples into a file in ND JSON format. This saved delving into the alternatives and the various choices available. It’s time to review the previous projects and consider the choice of file format for persistence.
What’s important is the overall flow of data from acquisition to analysis. The conceptual flow of data is shown in Figure 11.1.
This differs from the diagram shown in Chapter 2, Overview of the Projects, where the stages were not quite as well defined. Some experience with acquiring and cleaning data helps to clarify the considerations around saving and working with data.
The diagram shows a few of the many choices for persisting interim data. A more complete list of...
