Managing files with fs module

Node.js provides several core modules, including the fs module. fs stands for File System, and this module provides the APIs to interact with the file system.

In this recipe, we'll learn how to read, write, and edit files using the synchronous functions available in the fs module.

Getting ready

1. Create another directory for this recipe:

   $ mkdir working-with-files
   $ cd working-with-files

2. And now let's create a file to read. Run the following in your shell to create a file containing some simple text:

   $ echo Hello World! > hello.txt

3. We'll also need a file for our program—create a file named readWriteSync.js:

   $ touch readWriteSync.js

   Important note

   touch is a command-line utility included in Unix-based operating systems that is used to update the access and modification date of a file or directory to the current time. However, when touch is run with no additional arguments on a non-existent file, it will create an empty file with that name. touch is a typical way of creating an empty file.

How to do it

In this recipe, we'll synchronously read the file named hello.txt, manipulate the contents of the file, and then update the file using synchronous functions provided by the fs module:

1. We'll start by requiring the built-in modules fs and path. Add the following lines to readWriteSync.js:

   const fs = require("fs");
   const path = require("path");

2. Now let's create a variable to store the file path of the hello.txt file that we created earlier:

   const filepath = path.join(process.cwd(), "hello.txt");

3. We can now synchronously read the file contents using the readFileSync() function provided by the fs module. We'll also print the file contents to STDOUT using console.log():

   const contents = fs.readFileSync(filepath, "utf8");
   console.log("File Contents:", contents);

4. Now, we can edit the content of the file—we will convert the lowercase text into uppercase:

   const upperContents = contents.toUpperCase();

5. To update the file, we can use the writeFileSync() function. We'll also add a log statement afterward indicating that the file has been updated:

   fs.writeFileSync(filepath, upperContents);
   console.log("File updated.");

6. Run your program with the following:

   $ node readWriteSync.js
   File Contents: Hello World!
   File updated.

You now have a program that, when run, will read the contents of hello.txt, convert the text content into uppercase, and update the file.

How it works

The first two lines require the necessary core modules for the program.

const fs = require("fs"); will import the core Node.js File System module. The API documentation for the Node.js File System module is available at https://nodejs.org/api/fs.html. The fs module provides APIs to interact with the file system using Node.js. Similarly, the core path module provides APIs for working with file and directory paths. The path module API documentation is available at https://nodejs.org/api/path.html.

Next, we defined a variable to store the file path of hello.txt using the path.join() function and process.cwd(). The path.join() function joins the path sections provided as parameters with the separator for the specific platform (for example, / on Unix and \ on Windows environments).

process.cwd() is a function on the global process object that returns the current directory of the Node.js process. In this program, it is expecting the hello.txt file to be in the same directory as the program.

Next, we read the file using the fs.readFileSync() function. We pass this function the file path to read and the encoding, "utf8". The encoding parameter is optional—when the parameter is omitted, the function will default to returning a Buffer object.

To perform manipulation of the file contents, we used the toUpperCase() function available on string objects.

Finally, we updated the file using the fs.writeFileSync() function. We passed the fs.writeFileSync() function two parameters. The first was the path to the file we wished to update, and the second parameter was the updated file contents.

Important note

Both the readFileSync() and writeFileSync() APIs are synchronous, which means that they will block/delay concurrent operations until the file read or write is completed. To avoid blocking, you'll want to use the asynchronous versions of these functions covered in the There's more section.

There's more

Throughout this recipe, we were operating on our files synchronously. However, Node.js was developed with a focus on enabling the non-blocking I/O model, therefore, in many (if not most) cases, you'll want your operations to be asynchronous.

Today, there are three notable ways to handle asynchronous code in Node.js—callbacks, Promises, and async/await syntax. The earliest versions of Node.js only supported the callback pattern. Promises were added to the JavaScript specification with ECMAScript 2015, known as ES6, and subsequently, support for Promises was added to Node.js. Following the addition of Promise support, async/await syntax support was also added to Node.js.

All currently supported versions of Node.js now support callbacks, Promises, and async/await syntax. Let's explore how we can work with files asynchronously using these techniques.

Working with files asynchronously

Asynchronous programming can enable some tasks or processing to continue while other operations are happening.

The program from the Managing files with fs module recipe was written using the synchronous functions available on the fs module:

const fs = require("fs");
const path = require("path");
const filepath = path.join(process.cwd(), "hello.txt");
const contents = fs.readFileSync(filepath, "utf8");
console.log("File Contents:", contents);
const upperContents = contents.toUpperCase();
fs.writeFileSync(filepath, upperContents);
console.log("File updated.");

This means that the program was blocked waiting for the readFileSync() and writeFileSync() operations to complete. This program can be rewritten to make use of the asynchronous APIs.

The asynchronous version of readFileSync() is readFile(). The general convention is that synchronous APIs will have the term "sync" appended to their name. The asynchronous function requires a callback function to be passed to it. The callback function contains the code that we want to be executed when the asynchronous function completes.

1. The readFileSync() function in this recipe could be changed to use the asynchronous function with the following:

   const fs = require("fs");
   const path = require("path");
   const filepath = path.join(process.cwd(), "hello.txt");
   fs.readFile(filepath, "utf8", (err, contents) => {
     if (err) {
       return console.log(err);
     }
     console.log("File Contents:", contents);
     const upperContents = contents.toUpperCase();
     fs.writeFileSync(filepath, upperContents);
     console.log("File updated.");
   });

   Observe that all of the processing that is reliant on the file read needs to take place inside the callback function.

2. The writeFileSync() function can also be replaced with the asynchronous function, writeFile():

   const fs = require("fs");
   const path = require("path");
   const filepath = path.join(process.cwd(), "hello.txt");
   fs.readFile(filepath, "utf8", (err, contents) => {
     if (err) {
       return console.log(err);
     }
     console.log("File Contents:", contents);
     const upperContents = contents.toUpperCase();
     fs.writeFile(filepath, upperContents, function (err) {
       if (err) throw err;
       console.log("File updated.");
     });
   });

3. Note that we now have an asynchronous function that calls another asynchronous function. It's not recommended to have too many nested callbacks as it can negatively impact the readability of the code. Consider the following:

   first(args, () => {
       second(args, () => {
           third(args, () => {});
       });
   });

4. There are approaches that can be taken to avoid callback hell. One approach would be to split the callbacks into named functions. For example, our file could be rewritten so that the writeFile() call is contained within its own named function, updateFile():

   const fs = require("fs");
   const path = require("path");
   const filepath = path.join(process.cwd(), "hello.txt");
   fs.readFile(filepath, "utf8", (err, contents) => {
     if (err) {
       return console.log(err);
     }
     console.log("File Contents:", contents);
     const upperContents = contents.toUpperCase();
     updateFile(filepath, upperContents);
   });
   function updateFile(filepath, contents) {
     fs.writeFile(filepath, contents, (err) => {
       if (err) throw err;
       console.log("File updated.");
     });
   }

   Another approach would be to use Promises, which we'll cover in the Using the fs Promise API section of this chapter. But as the earliest versions of Node.js did not support Promises, the use of callbacks is still prevalent in many npm modules and existing applications.

5. To demonstrate that this code is asynchronous, we can use the setInterval() function to print a string to the screen while the program is running. The setInterval() function enables you to schedule a function to happen at a specified delay in milliseconds. Add the following line to the end of your program:

   setInterval(() => process.stdout.write("**** \n"), 1).unref();

   Observe that the string continues to be printed every millisecond, even in between when the file is being read and rewritten. This shows that the file reading and writing have been implemented in a non-blocking manner because operations are still completing while the file is being handled.

6. To demonstrate this further, you could add a delay between the reading and writing of the file. To do this, wrap the updateFile() function in a setTimeout() function. The setTimeout() function allows you to pass it a function and a delay in milliseconds:

   setTimeout(() => updateFile(filepath, upperContents), 10);

7. Now the output from our program should have more asterisks printed between the file read and write, as this is where we added the 10ms delay:

   $ node file-async.js
   **** 
   **** 
   File Contents: HELLO WORLD!
   **** 
   **** 
   **** 
   **** 
   **** 
   **** 
   **** 
   **** 
   **** 
   File updated

We can now see that we have converted the program from the Managing files with fs module recipe to handle the file operations asynchronously using the callback syntax.

Using the fs Promises API

The fs Promises API was released in Node.js v10.0.0. The API provides File System functions that return Promise objects rather than callbacks. Not all of the original fs module APIs have equivalent Promise-based APIs, as only a subset of the original APIs were converted to use Promise APIs. Refer to the Node.js API documentation for the full list of fs functions provided via the fs Promises API: https://nodejs.org/dist/latest/docs/api/fs.html#fs_fs_promises_api.

A Promise is an object that is used to represent the completion of an asynchronous function. The naming is based on the general definition of the term Promise—an agreement to do something or that something will happen. A Promise object is always in one of the three following states:

• Pending
• Fulfilled
• Rejected

A Promise will initially be in the pending state and will remain in the pending state until it becomes either fulfilled—when the task has completed successfully—or rejected—when the task has failed:

1. To use the API, you'll first need to import it:

   const fs = require("fs").promises;

2. It is then possible to read the file using the readFile() function:

   fs.readFile(filepath, "utf8").then((contents) => {
       console.log("File Contents:", contents);
   });

3. You can also combine the fs Promises API with the use of the async/await syntax:

   const fs = require("fs").promises;
   const path = require("path");
   const filepath = path.join(process.cwd(), "hello.txt");
   async function run() {
     try {
       const contents = await fs.readFile(filepath, "utf8");
       console.log("File Contents:", contents);
     } catch (error) {
       console.error(error);
     }
   }
   run();

Now we've learned how we can interact with files using the fs Promises API.

Important note

It was necessary to wrap the async/await example in a function as await must only be called from within an async function. There is an active proposal at ECMA TC39, the standardization body for ECMAScript (JavaScript), to support Top-Level Await, which would enable you to use the await syntax outside of an async function.

See also

• The Inspecting file metadata recipe in this chapter
• The Watching for file updates recipe in this chapter