Avoiding reparsing the entire file on every change
The lexical and syntax analysis necessary to parse input and detect and report syntax errors presented in this book from Chapter 2, Programming Language Design, to Chapter 8, Checking Types on Arrays, Method Calls, and Structure Accesses, are substantial algorithms. Although the Flex and Yacc tools we’ve used are high-performance, if given a large input file, scanning and parsing become slow enough that users will not want to reparse the whole file each time a user modifies it in an IDE text editor. In testing, we found that reparsing the entire file became a problem on files larger than 1,000 lines.
Sophisticated incremental parsing algorithms that minimize the amount that must be reparsed after changes are the subject of Ph.D. dissertations and research articles. For the Unicon IDE, a simple approach is taken. Whenever the cursor moves away from a line that has been changed, a parsing unit is selected, starting with the...
