I/O in the 21st century – knowing the streams

Many I/O-related activities handle streams of data. A stream is a sequence of data elements made available over time. Wikipedia says:

"A stream can be thought of as a conveyor belt that allows items to be processed one at a time rather than in large batches."

At the lowest level, all streams are bytes, but using a high-level interface could obviously help the programmer handle their data. This is the reason why a stream object usually has methods such as read, seek, write, and so on, just to make handling a byte stream a bit simpler.

In this recipe, you'll see some stream utilization examples.

Getting ready

In the good old Pascal days, there was a set of functions to handle the I/O (Assign, Reset, Rewrite, Close, and many more). Now, we have a bunch of classes. All Delphi streams inherit from TStream and can be used as the internal stream of one of the adapter classes (by adapter, I mean an implementation of the Adapter, or Wrapper, design patterns from the Gang of Four (GoF) famous book about design patterns).

There are 10 fundamental types of streams:

Class	Use
System.Classes.TBinaryWriter	Writer for binary data
System.Classes.TStreamWriter	Writer for characters to a stream
System.Classes.TStringWriter	Writer for a string
System.Classes.TTextWriter	Writer of a sequence of characters; it is an abstract class
System.Classes.TWriter	Writes component data to an associated stream
System.Classes.TReader	Reads component data from an associated stream
System.Classes.TStreamReader	Reader for stream of characters
System.Classes.TStringReader	Reader for strings
System.Classes.TTextReader	Reader for sequence of characters; it is an abstract class
System.Classes.TBinaryReader	Reader for binary data

You can check out the complete list and their intended uses on the Embarcadero website at http://docwiki.embarcadero.com/RADStudio/en/Streams,_Reader_and_Writers.

As Joel Spolsky says, You can no longer pretend that plain text is ASCII. So, while we write streams, we have to pay attention to the encoding our text uses and the encoding our counterpart is waiting for.

One of the most frequent necessities is to efficiently read and write a text file using the correct encoding:

"The Single Most Important Fact About Encodings... It does not make sense to have a string without knowing what encoding it uses. You can no longer stick your head in the sand and pretend that 'plain' text is ASCII."

– Joel Spolsky

The point Joel is making is that the content of a string doesn't know about the type of character encoding it uses.

When you think about file handling, ask yourself—Could this file become 10 MB? And 100 MB? And 1 GB? How will my program behave in that case? Handling a file one line at time and not loading all the file contents in memory is usually good insurance for these cases. A stream of data is a good way to do this. In this recipe, you'll see the practical utilization of streams, stream writers, and stream readers.

How it works...

The project is not complex. All the interesting stuff happens in btnWriteFile and btnReadFile. To write the file, TStreamWriter is used. TStreamWriter (similar to its counterpart TStreamReader) is a wrapper for a TStream descendant and adds some useful high-level methods to write to the stream. There are a lot of overloaded methods (Write/WriteLine) to allow easy writing to the underlying stream. However, you can access the underlying stream using the BaseStream property of the wrapper. Just after having written the file, the memo reloads the file using the same encoding used to write it and displays it. This is only a fast check for this recipe; you don't need TMemo at all in your real project. The btnReadFile simply opens the file using a stream and passes the stream to TStreamReader which, using the right encoding, will read the file one line at a time (note that text is stored in the .pas file, which is in this case encoded as UTF-8, while by default Delphi .pas files use ASCII encoding).

Now, let's run some checks. Run the program and with the encoding set to ASCII, click on btnWriteFile. The memo will show garbage text, as shown in the following screenshot. This is because we are using the wrong encoding for the data we are writing in the file:

Figure 1.16: Garbage text written to the file using the wrong encoding

Now, select UTF8 from the radio group and retry it. By clicking on btnWriteFile, you will see the correct text in the memo. Try to change the current encoding using ASCII and click on btnReadFile. You will still get garbage text. Why? Because the file has been read with the wrong encoding. You have to know the encoding beforehand to safely read the file's contents. To read the text that we wrote, we have to use the same encoding. Play with other encodings to see the different behavior.

There's more...

Streams are very powerful and their uniform interface helps us write portable and generic code. With the help of streams and polymorphism, we can write code that uses TStream to do some work, without knowing what kind of stream it is!

Also, a well-known possibility is that if you ever need to write a program that needs to access the good old TD_INPUT, STD_OUTPUT, or STD_ERROR, you can use THandleStream to wrap these system handles with a nice TStream interface using the following code:

program StdInputOutputError; 
//the following directive instructs the compiler to create a  
//console application and not a GUI one, which is the default. 
{$APPTYPE CONSOLE}  
uses 
  System.Classes, // required for Stream classes 
  Winapi.Windows; // required to have access to the STD_* handles 
var 
  StdInput: TStreamReader; 
  StdOutput, StrError: TStreamWriter; 
begin 
  StdInput := TStreamReader.Create( 
THandleStream.Create(STD_INPUT_HANDLE)); 
  StdInput.OwnStream; 
  StdOutput := TStreamWriter.Create( 
THandleStream.Create(STD_OUTPUT_HANDLE)); 
  StdOutput.OwnStream; 
  StdError := TStreamWriter.Create( 
THandleStream.Create(STD_ERROR_HANDLE)); 
  StdError.OwnStream; 
  { HERE WE CAN USE OURS STREAMS } 
  // Let's copy a line of text from STD_IN to STD_OUT 
  StdOutput.writeln(StdInput.ReadLine); 
  { END - HERE WE CAN USE OURS STREAMS } 
  StdError.Free; 
  StdOutput.Free; 
  StdInput.Free; 
end; 

Moreover, when you work with file-related streams, the TFile class (contained in System.IOUtils.pas) is very useful, and has some helper methods to write shorter and more readable code.