I/O in the 21st century – knowing the streams

Many I/O-related activities handle "streams" of data. A stream is a sequence of data elements made available over time. As Wikipedia says, "A stream can be thought of as a conveyor belt that allows items to be processed one at a time rather than in large batches."

At the lowest level, all streams are bytes, but using a high-level interface could obviously help the programmer handle their data. This is the reason why a stream object usually had methods such as read, seek, write, and so on, just to make handling a byte stream a bit simpler.

In this recipe, you'll see some stream utilization examples.

Getting ready

In the good old Pascal days, there were a set of functions to handle the I/O (AssignFile, Reset, Rewrite, CloseFile, and many more). Now, we've a bunch of classes. All Delphi streams inherit from TStream and can be used as the internal stream of one of the adapter classes (by adapter, I mean an implementation of the Adapter, or Wrapper, design pattern from the Gang of Four famous book about design patterns).

There are 10 fundamental types of streams.

Class	Use
System.Classes.TBinaryWriter	Writer for binary data
System.Classes.TStreamWriter	Writer for characters to stream
System.Classes.TStringWriter	Writer for a string
System.Classes.TTextWriter	Writer of sequence of characters; it is an abstract class
System.Classes.TWriter	Writes component data to an associated stream
System.Classes.TReader	Reads component data from an associated stream
System.Classes.TStreamReader	Reader for stream of characters
System.Classes.TStringReader	Reader for strings
System.Classes.TTextReader	Reader for sequence of characters; it is an abstract class
System.Classes.TBinaryReader	Reader for binary data

You can check out the complete list and their intended use directly on the Embarcadero website at http://docwiki.embarcadero.com/RADStudio/en/Streams,_Reader_and_Writers.

As Joel Spolsky says, "You can no longer pretend that "plain" text is ASCII." So, while we write streams, we've to pay attention to which encoding our text has and which encoding our counterpart is waiting for.

One of the most frequent necessities is to efficiently read and write a text file using the correct encoding.

	"The Single Most Important Fact About Encodings… It does not make sense to have a string without knowing what encoding it uses. You can no longer stick your head in the sand and pretend that "plain" text is ASCII."
	--Joel Spolsky (http://www.joelonsoftware.com/articles/Unicode.html)

The point Joel is making is that the content of a string doesn't know about the type of character encoding it uses.

When you think about file handling, ask yourself, "Could this file become 10 MB? And 100 MB? And 1 GB? How will my program behave in that case?" Handling a file one line at time and not loading all the file contents in memory is usually a good insurance for these cases. A stream of data is a good way to do this. In this recipe, you'll see the practical utilization of streams, stream writers, and stream readers.

How it works…

The project is not complex. All the interesting stuff happens in btnWriteFile and btnReadFile.

To write the file, TStreamWriter is used. TStreamWriter (as its counterpart TStreamReader) is a wrapper for a TStream descendant and adds some useful high-level methods to write to the stream. There are a lot of overloaded methods (Write/WriteLine) to allow an easy writing to the underlying stream. However, you can access the underling stream using the BaseStream property of the wrapper. Just after having written the file, the memo reloads the file using the same encoding used to write it, and shows it. This is only a fast check for this recipe, you don't need TMemo at all in your real project. The btnReadFile simply opens the file using a stream and passes the stream to TStreamReader that, using the right encoding, will read the file one line at a time.

Now, let's run some checks. Run the program and with the encoding set to ASCII, click on btnWriteFile. The memo will show garbage text, as shown in the following screenshot. This is because we are using the wrong encoding for the data we are writing in the file:

Figure 8.1: Garbage text written to the file using the wrong encoding. No one line text is equal to the original one. It is necessary to know the encoding for the text before writing and reading it

Now, select UTF8 from the RadioGroup and retry. By clicking on btnWriteFile, you will see the correct text in the memo. Try to change the current encoding using ASCII and click on btnReadFile. You will still get garbage text. Why? Because the file has been read with the wrong encoding. You have to know the encoding before to safely read file's contents. To read the text that we wrote, we have to use the same encoding. Play with other encodings to see the different behavior.

There's more…

Streams are very powerful and their uniform interface helps us write portable and generic code. With the help of streams and polymorphism, we can write code that uses TStream to do some work without knowing which kind of stream it is!

Also, a less known possibility, if you ever will write a program that needs to access the good old STD_INPUT, STD_OUTPUT, or STD_ERROR, is that you can use THandleStream to wrap these system handles to a nice TStream interface with the following code:

program StdInputOutputError;
//the following directive instructs the compiler to create a 
//console application and not a GUI one, which is the default.
{$APPTYPE CONSOLE} 
uses
  System.Classes, // required for Stream classes
  Winapi.Windows; // required to have access to the STD_* handles
var
  StdInput: TStreamReader;
  StdOutput, StrError: TStreamWriter;
begin
  StdInput := TStreamReader.Create(
THandleStream.Create(STD_INPUT_HANDLE));
  StdInput.OwnStream;
  StdOutput := TStreamWriter.Create(
THandleStream.Create(STD_OUTPUT_HANDLE));
  StdOutput.OwnStream;
  StdError := TStreamWriter.Create(
THandleStream.Create(STD_ERROR_HANDLE));
  StdError.OwnStream;
  { HERE WE CAN USE OURS STREAMS }
  // Let's copy a line of text from STD_IN to STD_OUT
  StdOutput.writeln(StdInput.ReadLine);
  { END - HERE WE CAN USE OURS STREAMS }
  StdError.Free;
  StdOutput.Free;
  StdInput.Free;
end;

Moreover, when you work with file-related streams, the TFile class (contained in System.IOUtils.pas) is very useful, and has some helper methods to write shorter and more readable code.