Dealing with difficult data

Up until now, we have discussed reading data into SAS in typical SAS and non-SAS formats. Because SAS is able to handle such large datasets, it is common to receive data from legacy systems that were built many decades ago but continue to be used to store big data. These systems can produce data extracts that are difficult to read into SAS environments.

This section will cover the following:

• How legacy data is typically documented, and how the information in this documentation can be helpful for informing the development of infile code.

• How file viewer and big data editor applications can be helpful in developing infile code for legacy data extracts.

• How to adjust infile options, format, and informat to troubleshoot reading in difficult data.

• How to use PROC PRINT and PROC CONTENTS as part of troubleshooting reading in difficult data.

• How to use file positions to read in difficult data from a fixed width file.

• How to verify...