Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Redis Essentials
Redis Essentials

Redis Essentials: Harness the power of Redis to integrate and manage your projects efficiently

Arrow left icon
Profile Icon Maxwell Dayvson da Silva
Arrow right icon
$19.99 per month
Full star icon Full star icon Full star icon Full star icon Half star icon 4.6 (18 Ratings)
Paperback Sep 2015 230 pages 1st Edition
eBook
$24.99 $35.99
Paperback
$43.99
Subscription
Free Trial
Renews at $19.99p/m
Arrow left icon
Profile Icon Maxwell Dayvson da Silva
Arrow right icon
$19.99 per month
Full star icon Full star icon Full star icon Full star icon Half star icon 4.6 (18 Ratings)
Paperback Sep 2015 230 pages 1st Edition
eBook
$24.99 $35.99
Paperback
$43.99
Subscription
Free Trial
Renews at $19.99p/m
eBook
$24.99 $35.99
Paperback
$43.99
Subscription
Free Trial
Renews at $19.99p/m

What do you get with a Packt Subscription?

Free for first 7 days. $19.99 p/m after that. Cancel any time!
Product feature icon Unlimited ad-free access to the largest independent learning library in tech. Access this title and thousands more!
Product feature icon 50+ new titles added per month, including many first-to-market concepts and exclusive early access to books as they are being written.
Product feature icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Product feature icon Thousands of reference materials covering every tech concept you need to stay up to date.
Subscribe now
View plans & pricing
Table of content icon View table of contents Preview book icon Preview Book

Redis Essentials

Chapter 1. Getting Started (The Baby Steps)

Redis is a NoSQL (Not only SQL) advanced key-value data store. It is also referred to as a data structure server because of its powerful data types, such as Strings, Hashes, Lists, Sets, Sorted Sets, Bitmaps, and HyperLogLogs. By default, Redis saves all data in the memory, therefore read and write operations are very fast. It can also cause data to persist in the disk. Data persistence in Redis can be achieved by creating a binary snapshot of the stored data or a human-readable file with a sequence of all executed commands over time. These are respectively known as snapshotting and journaling.

Additionally, Redis includes configurable key expiration, transactions, and publish/subscribe features. It also provides Lua scripting to extend Redis to create new commands. Combined, these features transform Redis into the Swiss Army knife of data type storage.

Redis stands for REmote DIctionary Server. It was written in C by Salvatore Sanfilippo in 2006 and currently has many contributors. There are Redis clients available for over 30 programming languages. The open source project can be found at https://github.com/antirez/redis. The official Redis documentation is also a really good resource of knowledge and can be found at http://redis.io.

Redis is a well-established open source project and has been used in production for years by big companies, including Twitter, GitHub, Tumblr, Pinterest, Instagram, Hulu, Flickr, and The New York Times.

This chapter is going to show you how to install Redis, introduce the command-line interface, introduce a Node.js client for Redis, and then present three data types in detail: Strings, Lists, and Hashes.

Redis data types are a very extensive subject. There is enough information to write a book that just describes how they work. We will present the most relevant and useful commands for each data type along with real-life use cases in the first two chapters. Chapter 2, Advanced Data Types (Earning a Black Belt), is going to cover other data types: Sets, Sorted Sets, Bitmaps, and HyperLogLogs. After this chapter and the next have explained all data types, Chapter 3, Time Series (A Collection of Observations), will present a time series implementation that uses multiple data types.

Note

Please note that all data types will be shown with the first letter capitalized (for example, Strings, Lists, Bitmaps, Sets, Sorted Sets, and HyperLogLogs) so that we can distinguish between a Redis data type and other existing terms.

Installation

At the time of writing this book, the stable version of Redis was 3.0. All examples presented in this book will work with this version, but it is very likely that newer versions of Redis are going to work as well. Redis is very strict in terms of backward compatibility, so it provides API stability between minor versions. We recommend that you install the latest version of Redis to get the recent bug fixes and performance improvements. Most of the content in this book will remain useful even if you work with a more recent version.

Officially, Redis can be compiled and used on Linux, OS X, OpenBSD, NetBSD, and FreeBSD.

Redis is not officially supported on Windows. However, the Microsoft Open Tech group develops and maintains a Windows port targeting Win64 architecture, which can be found at https://github.com/MSOpenTech/redis. We are not going to cover Windows installation or guarantee that the examples presented in this book will work on Windows.

Installing from source

The first thing we need to do is open a terminal and run the following commands to download and install Redis. The following commands can be executed in any *nix operating system (Ubuntu, CentOS, Debian, OS X, and so on). Some build tools are required to build Redis from source (for example, gcc, make, and so on). On Ubuntu and Debian, these tools can be installed by the package build-essentials.

On OS X, you will need Xcode and Command Line Tools Package installed. After the required build tools are installed, open a terminal window and execute the following commands:

$ curl -O http://download.redis.io/releases/redis-3.0.2.tar.gz
$ tar xzvf redis-3.0.2.tar.gz
$ cd redis-3.0.2
$ sudo make install

Every time you see a dollar sign ($) at the beginning of a code block, it means we are executing the command in a terminal window.

Tip

Another way to install Redis is by using package managers, such as yum, apt, or brew. Make sure your package manager has Redis 3.0 or later available.

Hello Redis (command-line interface examples)

Redis comes with several executables. In this section, we are going to focus on redis-server and redis-cli.

redis-server is the actual Redis data store. It can be started in standalone mode or in cluster mode. For now, we are only going to use the single-instance mode and later (in Chapter 9, Redis Cluster and Redis Sentinel (Collective Intelligence)) we will cover cluster mode.

redis-cli is a command-line interface that can perform any Redis command (it is a Redis client). It makes learning to execute commands in Redis more intuitive.

This chapter is also going to introduce a Node.js client, and later (in Chapter 5, Clients for Your Favorite Language (Become a Redis Polyglot)) we will see how to use Redis with PHP, Python, and Ruby clients.

By default, Redis binds to port 6379, runs in standalone mode, and can be started with this line:

$ redis-server

Since no configuration was specified in this example, Redis will use default configurations.

It will output its PID (process ID) and the port that the clients should connect to, which is 6379 by default.

Hello Redis (command-line interface examples)

Note

Important note:

The following conventions will be used in this book for redis-cli:

  • Commands are written in bold, uppercase letters (SET).
  • Keys are written in italicized, lowercase letters (GET mykey).
  • Values are written without any text formatting (SET mykey "my value").

The next snippet shows how to connect to the Redis server using redis-cli. Once connected, we use the SET command to create a key with a string value and then the GET command to read the key value:

$ redis-cli
127.0.0.1:6379> SET philosopher "socrates"
OK
127.0.0.1:6379> GET philosopher
"socrates"
127.0.0.1:6379>

The HELP command is useful for learning about command syntax. It displays the command parameters with a summary and examples. See the following example:

$ redis-cli
127.0.0.1:6379> HELP SET

  SET key value [EX seconds] [PX milliseconds] [NX|XX]
  summary: Set the string value of a key
  since: 1.0.0
  group: string

The KEYS command is also useful, as it returns all stored keys that match a pattern (it is a glob-style pattern, like the Unix shell glob pattern). In the following code, all stored key names that start with the letter "p" are returned:

$ redis-cli
127.0.0.1:6379> KEYS p*
1) "philosopher"

The redis-cli is a great tool for debugging and testing commands, but making real examples and applications using redis-cli is impractical. This book is going to use the JavaScript language and Node.js to support examples and explanations. We chose JavaScript because of its current popularity. The Node.js website (https://nodejs.org) provides binaries for Mac OS X, Windows, and Linux, which makes installation of Node.js really simple. Keep in mind that this is not a JavaScript book; we are going to use basic features of the language in our examples. If you do not know how to code in JavaScript, do not worry. A quick syntax reference is presented, and it should be enough to understand all the examples in this book.

Note

You can reproduce all the samples presented here in your favorite language. Redis will produce the same results regardless of the programming language.

Installing Node.js

Download and install Node.js from its website using the available binary packages. At the time of writing this book, the latest version of Node.js was 0.12.4. All examples are guaranteed to work with this version.

Node.js comes with a package manager called Node Package Manager (NPM), which is responsible for managing and installing all Node.js dependencies and libraries. Think of it as pip for Python or cpan for Perl.

We recommend that you create a folder called redis-essentials to save all the files and libraries necessary for running the examples. We also recommend that you create one folder for each chapter of this book for organization purposes.

All Node.js examples in this book require the library redis, which can be installed with NPM:

$ cd redis-essentials
$ npm install redis

NPM will create a folder called node_modules. This is where the redis client is installed.

JavaScript syntax quick reference guide

If you know the basics of JavaScript, you can skip this section. Here is a quick overview of JavaScript:

  • Use the keyword var to define a variable:
    var myAge = 31;
  • Use // for inline comments and /* */ for multiline comments:
    // this is an inline comment
    /* this
    is a
    multi-line
    comment
    */
  • Conditional statements:
    if (myAge > 29) {
      console.log("I am not in my twenties anymore!");
    } else {
      console.log("I am still in my twenties!");
    }
  • Defining a function:
    function nameOfMyFunction(argument1, argument2) {
      console.log(argument1, argument2);
    }
  • Executing a function:
    nameOfMyFunction("First Value", "Second Value");
  • A function can also behave as a class and have methods, properties, and instances. Properties are accessed through the keyword this:
    function Car(maxSpeed) {
      this.maxSpeed = maxSpeed;
      this.currentSpeed = 0;
    }
  • The standard way to create a prototyped method for a function in JavaScript is by using the property prototype:
    Car.prototype.brake = function() {
      if (this.currentSpeed > 0) {
        this.currentSpeed -= 5;
      }
    };
    
    Car.prototype.accelerate = function() {
      if (this.currentSpeed < this.maxSpeed) {
        this.currentSpeed += 5;
      }
    };
  • To create an instance of a class in JavaScript, use the keyword new:
    var car = new Car(100);
    car.accelerate();
    car.accelerate();
    car.brake();
  • Arrays and objects:
    var myArray = [];
    var myObject = {};
  • Callbacks in JavaScript:
    var friends = ["Karalyn", "Patrik", "Bernardo"];
    friends.forEach(function (name, index) {
      console.log(index + 1, name); // 1 Karalyn, 2 Patrik, 3 Bernardo
    });

A callback in this example is an anonymous function that is passed to another function as a parameter, so it is called (or executed) inside the other function. As you can see in the preceding example, the forEach array method expects a callback function. It executes the provided callback once for each element in the array. It is very common to find asynchronous functions/methods that expect callbacks in JavaScript.

If you want to know more about JavaScript syntax and features, we recommend the Mozilla Developer Network website at https://developer.mozilla.org/en-US/docs/Web/JavaScript.

Hello World with Node.js and Redis

This section shows the basics of creating a JavaScript program using Redis. It is important to understand this foundation since the upcoming examples use the same principles.

Note

In this book, all filenames, function names, and variable names are italicized. Some sentences follow this convention:

  • Create a file called my-filename.js.
  • Execute the function myFunctionName.
  • Create a variable called myVariableName.

Create a file called hello.js with the following code:

var redis = require("redis"); // 1
var client = redis.createClient(); // 2
client.set("my_key", "Hello World using Node.js and Redis"); // 3
client.get("my_key", redis.print); // 4
client.quit(); // 5

Note

Please note that all the code snippets in this book will have inline comments with numbers. After the code is presented, it will be explained by referencing those numbers.

  1. Require the redis library in Node.js. This is equivalent to import in Go, Python, or Java.
  2. Create the Redis client object.
  3. Execute the Redis command SET to save a String in a key called my_key.
  4. Execute the Redis command GET to get the value stored in my_key, and then output it.
  5. Close the connection with the Redis server.

    Note

    Lines 1, 2, and 5 of this example will be used in the majority of the examples that use Node.js.

Run hello.js with the node command (node is the Node.js interpreter):

$ node hello.js
Reply: Hello World using Node.js and Redis

Redis data types

After you have understood how Redis data types work, you will be able to design better applications and make better use of the available resources. It will also help you decide whether Redis is the right solution for your problem. The main reason for Redis to have many data types is very simple: one size does not fit all, and different problems require different solutions.

Although you do not need to use all the data types, it is important to understand how they work so that you can choose the right ones. By the end of this book, you will have a full understanding of these data types and know how to improve the performance of your applications using Redis.

Strings

Strings are the most versatile data types in Redis because they have many commands and multiple purposes. A String can behave as an integer, float, text string, or bitmap based on its value and the commands used. It can store any kind of data: text (XML, JSON, HTML, or raw text), integers, floats, or binary data (videos, images, or audio files). A String value cannot exceed 512 MB of text or binary data.

The following are some use cases for Strings:

  • Cache mechanisms: It is possible to cache text or binary data in Redis, which could be anything from HTML pages and API responses to images and videos. A simple cache system can be implemented with the commands SET, GET, MSET, and MGET.
  • Cache with automatic expiration: Strings combined with automatic key expiration can make a robust cache system using the commands SETEX, EXPIRE, and EXPIREAT. This is very useful when database queries take a long time to run and can be cached for a given period of time. Consequently, this avoids running those queries too frequently and can give a performance boost to applications.
  • Counting: A counter can easily be implemented with Strings and the commands INCR and INCRBY. Good examples of counters are page views, video views, and likes. Strings also provide other counting commands, such as DECR, DECRBY, and INCRFLOATBY.

String examples with redis-cli

The MSET command sets the values of multiple keys at once. The arguments are key-value pairs separated by spaces.

The MGET command retrieves the values of multiple key names at once, and the key names are separated by spaces.

The following is a combined example for the preceding commands:

$ redis-cli
127.0.0.1:6379> MSET first "First Key value" second "Second Key value"
OK
127.0.0.1:6379> MGET first second
1) "First Key value"
2) "Second Key value"

The EXPIRE command adds an expiration time (in seconds) to a given key. After that time, the key is automatically deleted. It returns 1 if the expiration is set successfully and 0 if the key does not exist or cannot be set.

The TTL (Time To Live) command returns one of the following:

  • A positive integer: This is the amount of seconds a given key has left to live
  • -2: If the key is expired or does not exist
  • -1: If the key exists but has no expiration time set
$ redis-cli
127.0.0.1:6379> SET current_chapter "Chapter 1"
OK
127.0.0.1:6379> EXPIRE current_chapter 10
(integer) 1
127.0.0.1:6379> GET current_chapter
"Chapter 1"
127.0.0.1:6379> TTL current_chapter
(integer) 3
127.0.0.1:6379> TTL current_chapter
(integer) -2
127.0.0.1:6379> GET current_chapter
(nil)
127.0.0.1:6379>

The commands INCR and INCRBY have very similar functionality. INCR increments a key by 1 and returns the incremented value, whereas INCRBY increments a key by the given integer and returns the incremented value. DECR and DECRBY are the opposites of INCR and INCRBY. The only difference is that DECR and DECRBY decrements a key.

The command INCRBYFLOAT increments a key by a given float number and returns the new value. INCRBY, DECRBY, and INCRBYFLOAT accept either a positive or a negative number:

$ redis-cli 
127.0.0.1:6379> SET counter 100
OK
127.0.0.1:6379> INCR counter
(integer) 101
127.0.0.1:6379> INCRBY counter 5 
(integer) 106
127.0.0.1:6379> DECR counter
(integer) 105
127.0.0.1:6379> DECRBY counter 100
(integer) 5
127.0.0.1:6379> GET counter
"5"
127.0.0.1:6379> INCRBYFLOAT counter 2.4
"7.4"

The preceding commands shown are atomic, which means that they increment/decrement and return the new value as a single operation. It is not possible for two different clients to execute the same command at the same time and get the same result—no race conditions happen with those commands.

For example, if the counter key is 1 and two different clients (A and B) increment their counters at the same time with INCR, client A will receive the value 2 and client B will receive 3.

Note

Redis is single threaded, which means that it always executes one command at a time. Sometimes, commands are mentioned as atomic, which means that a race condition will never happen when multiple clients try to perform operations on the same key at the same time.

Building a voting system with Strings using Node.js

This section builds a set of Node.js functions used to upvote and downvote articles. The idea is that there is a set of articles, and users can define their popularity by voting up or down.

Now let's save a small collection of articles in Redis using redis-cli. We will only add three article headlines to make the example easier to understand. In a real-world situation, you would use a Redis client for your programming language (rather than redis-cli), and the articles would be retrieved from a database:

$ redis-cli
127.0.0.1:6379> SET article:12345:headline "Google Wants to Turn Your Clothes Into a Computer"
OK
127.0.0.1:6379> SET article:10001:headline "For Millennials, the End of the TV Viewing Party"
OK
127.0.0.1:6379> SET article:60056:headline "Alicia Vikander, Who Portrayed Denmark's Queen, Is Screen Royalty"
OK

To complete this example, we will need two keys in Redis for each article. We have already defined our first key to store the headline of each article. Observe this key name structure: article:<id>:headline. The second key name will have a similar structure: article:<id>:votes. This nomenclature is important in order to create abstractions. The IDs may be passed around, and even if the key format changes, the application logic will remain the same. Also, it is easy to extend the application if other metadata (URL, summary, and so on) needs to be stored.

Our code will have three functions: the first increments the number of votes in an article by 1, the second decrements the number of votes in an article by 1, and the third displays the article headline and the number of votes. All three functions (upVote, downVote, and showResults) require the article ID as the argument. Perform the following set of steps:

Create a file called articles-popularity.js in the chapter 1 folder where all of the code from this section should be saved:

var redis = require("redis"); //1
var client = redis.createClient(); // 2

function upVote(id) {  // 3
  var key = "article:" + id + ":votes"; // 4
  client.incr(key);  // 5
}
  1. Require the redis library in Node.js. This is equivalent to import in other languages.
  2. Create a Redis client instance.
  3. Create an upVote function that has the article ID as the argument.
  4. Define your key name using the article:<id>:votes structure.
  5. Use the INCR command to increment the number of votes by 1.

The function downVote is basically the same as upVote. The only difference is that it uses the command DECR instead of INCR:

function downVote(id) { // 1
  var key = "article:" + id + ":votes"; // 2
  client.decr(key); // 3
}
  1. Create a function downVote that has the article ID as the argument.
  2. Define your key name using the structure article:<id>:votes (just as we did in the upVote function).
  3. Use the DECR command to decrement the number of votes by 1.

The function showResults shows the article headline and the number of votes that an article has:

function showResults(id) { 
  var headlineKey = "article:" + id + ":headline";
  var voteKey = "article:" + id + ":votes"; 
  client.mget([headlineKey, voteKey], function(err, replies) { // 1
    console.log('The article "' + replies[0] + '" has', replies[1], 'votes'); // 2
  });
}
  1. Use the MGET command to pass an array of keys and a callback function. For every key that does not hold a String value or does not exist, the value null is returned.

    In the anonymous function, the argument replies has two values: index 0, which has the headline, and index 1, which has the number of votes.

  2. Display a message with the article headline and number of votes.

    Note

    Note:

    The Node.js client that we are using is strictly asynchronous. All Redis commands have an optional callback function for handling errors and replies from the Redis server.

    In the previous MGET example, the only way to handle the key values is by passing a callback to client.mget().

    Please make sure you fully understand the idea of callbacks mentioned before. This is necessary in order to understand other examples using Node.js.

It is time to call our functions upVote, downVote, and showResults. Add the following to articles-popularity.js too:

upVote(12345); // article:12345 has 1 vote
upVote(12345); // article:12345 has 2 votes
upVote(12345); // article:12345 has 3 votes
upVote(10001); // article:10001 has 1 vote
upVote(10001); // article:10001 has 2 votes
downVote(10001); // article:10001 has 1 vote
upVote(60056); // article:60056 has 1 vote

showResults(12345);
showResults(10001);
showResults(60056); 

client.quit();

Then execute it using the following command line:

$ node articles-popularity.js
The article "Google Wants to Turn Your Clothes Into a Computer" has 3 votes
The article "For Millennials, the End of the TV Viewing Party" has 1 votes
The article "Alicia Vikander, Who Portrayed Denmark's Queen, Is Screen Royalty" has 1 votes

Tip

Downloading the example code

You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

Lists

Lists are a very flexible data type in Redis because they can act like a simple collection, stack, or queue. Many event systems use Redis's Lists as their queue because Lists' operations ensure that concurrent systems will not overlap popping items from a queue—List commands are atomic. There are blocking commands in Redis's Lists, which means that when a client executes a blocking command in an empty List, the client will wait for a new item to be added to the List. Redis's Lists are linked lists, therefore insertions and deletions from the beginning or the end of a List run in O(1), constant time.

The task of accessing an element in a List runs in O(N), linear time, but accessing the first or last element always runs in constant time.

A List can be encoded and memory optimized if it has less elements than the list-max-ziplist-entries configuration and if each element is smaller than the configuration list-max-ziplist-value (in bytes). Chapter 4, Commands (Where the Wild Things Are) provides more details on these configurations.

The maximum number of elements a List can hold is 232-1, which means there can be more than 4 billion elements per List.

Some real-world use cases of Lists are as follows:

  • Event queue: Lists are used in many tools, including Resque, Celery, and Logstash
  • Storing most recent user posts: Twitter does this by storing the latest tweets of a user in a List

In this section, we will show you some List commands using the redis-cli, and then present a generic task queue system in Node.js.

List examples with redis-cli

Since Lists in Redis are linked lists, there are commands used to insert data into the head and tail of a List. The command LPUSH inserts data at the beginning of a List (left push), and the command RPUSH inserts data at the end of a List (right push):

$ redis-cli
127.0.0.1:6379> LPUSH books "Clean Code"
(integer) 1
127.0.0.1:6379> RPUSH books "Code Complete"
(integer) 2
127.0.0.1:6379> LPUSH books "Peopleware"
(integer) 3

The command LLEN returns the length of a List. The command LINDEX returns the element in a given index (indices are zero-based). Elements in a List are always accessed from left to right, which means that index 0 is the first element, index 1 is the second element, and so on. It is possible to use negative indices to access the tail of the List, in which -1 is the last element, -2 is penultimate element, and so on. LINDEX does not modify a List:

$ redis-cli
127.0.0.1:6379> LLEN books
(integer) 3
127.0.0.1:6379> LINDEX books 1
"Clean Code"

The command LRANGE returns an array with all elements from a given index range, including the elements in both the start and end indices. As we mentioned previously, indices are zero-based and can be positive or negative. See the following example:

$ redis-cli
127.0.0.1:6379> LRANGE books 0 1
1) "Peopleware"
2) "Clean Code"
127.0.0.1:6379> LRANGE books 0 -1
1) "Peopleware"
2) "Clean Code"
3) "Code Complete"

The command LPOP removes and returns the first element of a List. The command RPOP removes and returns the last element of a List. Unlike LINDEX, both LPOP and RPOP modify the List:

$ redis-cli
127.0.0.1:6379> LPOP books
"Peopleware"
127.0.0.1:6379> RPOP books
"Code Complete"
127.0.0.1:6379> LRANGE books 0 -1
1) "Clean Code"

Implementing a generic Queue System

The following implementation is going to use JavaScript prototypes, and it is going to be similar to a class-based solution seen in many programming languages.

Create a file called queue.js in the chapter 1 folder with the following code:

function Queue(queueName, redisClient) { // 1
  this.queueName = queueName;  // 2
  this.redisClient = redisClient; // 3
  this.queueKey = 'queues:' + queueName; // 4
  // zero means no timeout 
  this.timeout = 0; // 5
}
  1. Create a function called Queue, which receives a queue name and the Redis client object as parameters.
  2. Save queueName as a property.
  3. Save redisClient as a property.
  4. Set the property queueKey to the proper Redis key name, based on the function parameter.
  5. Set the property timeout to zero, which means that when List commands are executed, they will have no timeout.

We need to implement three methods to perform queue operations: size, push, and pop.

The first method we are going to create is size:

Queue.prototype.size = function(callback) { // 1
  this.redisClient.llen(this.queueKey, callback); // 2
};
  1. Create the Queue method size, which expects a callback as an argument.
  2. Execute LLEN on the queue key name and pass the callback as an argument. This is necessary because the Redis client is asynchronous.

The implementation of the push method is as follows:

Queue.prototype.push = function(data) { // 1
  this.redisClient.lpush(this.queueKey, data); // 2
};
  1. Create the Queue method push that expects one argument. This argument can be anything that can be represented as a string.
  2. Execute LPUSH by passing the queue key name and the data argument.

As this is a generic queue system and Redis lists only store bytes, we assume that all of the data that is sent to the queue can be transformed into a JavaScript string. If you want to make it more generic, you can use JSON serialization and store the serialized string. The previous example used LPUSH because we were implementing a queue, and by definition, items are inserted at the front of the queue and removed from the end of the queue. A helpful way to remember this is FIFO (First In, First Out)—we went from left to right.

The implementation of the pop method is as follows:

Queue.prototype.pop = function(callback) { // 1
  this.redisClient.brpop(this.queueKey, this.timeout, callback); // 2
};
  1. Create the Queue method pop, which expects a callback as an argument.
  2. Execute BRPOP, passing the queue key name, the queue timeout property, and the callback as arguments.

As we mentioned earlier, elements are inserted at the front of the queue and removed from the end of the queue, which is why BRPOP was used (if RPUSH was used, then BLPOP would be necessary).

The command BRPOP removes the last element of a Redis List. If the List is empty, it waits until there is something to remove. BRPOP is a blocking version of RPOP. However, RPOP is not ideal. If the List is empty, we would need to implement some kind of polling by ourselves to make sure that items are handled as soon as they are added to the queue. It is better to take advantage of BRPOP and not worry about empty lists.

A concrete producer/consumer implementation is shown next. Different log messages are pushed into the "logs" queue by the producer and then popped by the consumer in another terminal window.

The complete Queue code, saved as queue.js, is as follows:

function Queue(queueName, redisClient) {
  this.queueName = queueName;
  this.redisClient = redisClient;
  this.queueKey = 'queues:' + queueName;
  // zero means no timeout 
  this.timeout = 0;
}

Queue.prototype.size = function(callback) {
  this.redisClient.llen(this.queueKey, callback);
};
Queue.prototype.push = function(data) {
  this.redisClient.lpush(this.queueKey, data);
};

Queue.prototype.pop = function(callback) {
  this.redisClient.brpop(this.queueKey, this.timeout, callback);
};

exports.Queue = Queue; // 1
  1. This is required to expose Queue to different modules. This explicit export is specific to Node.js, and it is necessary in order to run require("./queue").

Create a file called producer-worker.js in the chapter 1 folder, which is going to add log events to a queue named "logs", and save the following:

var redis = require("redis");
var client = redis.createClient();
var queue = require("./queue"); // 1
var logsQueue = new queue.Queue("logs", client); // 2
var MAX = 5;
for (var i = 0 ; i < MAX ; i++) { // 3
    logsQueue.push("Hello world #" + i); // 4
}
console.log("Created " + MAX + " logs"); // 5
client.quit();
  1. Require the module queue, which we've already created and saved as queue.js.
  2. Create an instance of the function Queue defined in the queue.js file.
  3. Create a loop that runs five times.
  4. Push some logs into the logs queue.
  5. Print the number of logs created.

Execute the producer file to push logs into the queue:

$ node producer-worker.js
Created 5 logs

Save the following code in a file called consumer-worker.js:

var redis = require("redis");
var client = redis.createClient();
var queue = require("./queue"); // 1
var logsQueue = new queue.Queue("logs", client); // 2

function logMessages() { // 3
  logsQueue.pop(function(err, replies) { // 4
    var queueName = replies[0];
    var message = replies[1];
    console.log("[consumer] Got log: " + message); // 5

    logsQueue.size(function(err, size) { // 6
      console.log(size + " logs left"); 
    });

    logMessages(); // 7
  });
}

logMessages(); // 8
  1. Require the queue module (this is the queue.js file).
  2. Create a Queue instance named logs and pass the Redis client to it.
  3. Create the function logMessages.
  4. Retrieve an element from the queue instance using the pop method. If the List is empty, this function waits until a new element is added. The timeout is zero and it uses a blocking command, BRPOP, internally.
  5. Display a message retrieved from the queue.
  6. Display the queue size after popping a message from the queue.
  7. Call the function (recursively) to repeat the process over and over again. This function runs forever.
  8. Call logMessages to initialize the queue consumption.

This queue system is completed. Now run the file consumer-worker.js and watch the elements being popped in the same order in which they were added by producer-worker.js:

$ node consumer-worker.js
[consumer] Got log: Hello world #0
4 logs left
[consumer] Got log: Hello world #1
3 logs left
[consumer] Got log: Hello world #2
2 logs left
[consumer] Got log: Hello world #3
1 logs left
[consumer] Got log: Hello world #4
0 logs left

This file will run indefinitely. More messages can be added to the queue by executing producer-worker.js again in a different terminal, and the consumer will continue reading from the queue as soon as new items are added.

The example shown in this section is not reliable enough to deploy to production. If anything goes wrong with the callbacks that pop from the queue, items may be popped but not properly handled. There is no such thing as a retry or any way to track failures.

A good way of solving the reliability problem is to use an additional queue. Each element that is popped from the queue goes to this additional queue. You must remove the item from this extra queue only if everything has worked correctly. You can monitor this extra queue for stuck elements in order to retry them or create failure alerts. The command RPOPLPUSH is very suitable for this situation, because it does a RPOP in a queue, then does a LPUSH in a different queue, and finally returns the element, all in a single step—it is an atomic command.

Hashes

Hashes are a great data structure for storing objects because you can map fields to values. They are optimized to use memory efficiently and look for data very fast. In a Hash, both the field name and the value are Strings. Therefore, a Hash is a mapping of a String to a String.

Previously, in the String example, we used two separate keys to represent an article headline and its votes (article:<id>:headline and article:<id>:votes). It is more semantic to use a Hash in that case because the two fields belong to the same object (that is, the article).

Another big advantage of Hashes is that they are memory-optimized. The optimization is based on the hash-max-ziplist-entries and hash-max-ziplist-value configurations. Chapter 4, Commands (Where the Wild Things Are), provides more details on these configurations.

Internally, a Hash can be a ziplist or a hash table. A ziplist is a dually linked list designed to be memory efficient. In a ziplist, integers are stored as real integers rather than a sequence of characters. Although a ziplist has memory optimizations, lookups are not performed in constant time. On the other hand, a hash table has constant-time lookup but is not memory-optimized.

Note

Instagram had to back-reference 300 million media IDs to user IDs, and they decided to benchmark a Redis prototype using Strings and Hashes. The String solution used one key per media ID and around 21 GB of memory. The Hash solution used around 5 GB with some configuration tweaks. The details can be found at http://instagram-engineering.tumblr.com/post/12202313862/storing-hundreds-of-millions-of-simple-key-value.

This section is going to show the most used Hash commands using redis-cli, and then present an application that stores movie metadata in Node.js (similar to the http://www.imdb.com website).

Using Hashes with redis-cli

The command HSET sets a value to a field of a given key. The syntax is HSET key field value.

The command HMSET sets multiple field values to a key, separated by spaces. Both HSET and HMSET create a field if it does not exist, or overwrite its value if it already exists.

The command HINCRBY increments a field by a given integer. Both HINCRBY and HINCRBYFLOAT are similar to INCRBY and INCRBYFLOAT (not presented in the following code):

$ redis-cli
127.0.0.1:6379> HSET movie "title" "The Godfather"
(integer) 1
127.0.0.1:6379> HMSET movie "year" 1972 "rating" 9.2 "watchers" 10000000
OK
127.0.0.1:6379> HINCRBY movie "watchers" 3
(integer) 10000003

The command HGET retrieves a field from a Hash. The command HMGET retrieves multiple fields at once:

127.0.0.1:6379> HGET movie "title"
"The Godfather"
127.0.0.1:6379> HMGET movie "title" "watchers"
1) "The Godfather"
2) "10000003"

The command HDEL deletes a field from a Hash:

127.0.0.1:6379> HDEL movie "watchers"
(integer) 1

The command HGETALL returns an array of all field/value pairs in a Hash:

127.0.0.1:6379> HGETALL movie
1) "title"
2) "The Godfather"
3) "year"
4) "1972"
5) "rating"
6) "9.2"
127.0.0.1:6379>

It is possible to retrieve only the field names or field values of a Hash with the commands HKEYS and HVALS respectively.

In the next section, we are going to use Hashes to implement a voting system similar to the one presented with Strings.

A voting system with Hashes and Node.js

This section creates a set of functions to save a link and then upvote and downvote it. This is a very simplified version of something that a website like http://www.reddit.com does.

Create a file called hash-voting-system.js in the chapter 1 folder, where all of the code from this section should be saved:

var redis = require("redis"); // 1
var client = redis.createClient(); // 2

function saveLink(id, author, title, link) { // 3
  client.hmset("link:" + id, "author", author, "title", title, "link", link, "score", 0); // 4
}
  1. Require the module redis.
  2. Create a Redis client instance.
  3. Create a function saveLink that has id, author, title, and link as arguments.
  4. Use HMSET to create a Hash with all fields.

The upVote and downVote functions use the same command (HINCRBY). The only difference is that downVote passes a negative number:

function upVote(id) { // 1
  client.hincrby("link:" + id, "score", 1); // 2
}
function downVote(id) { // 3
  client.hincrby("link:" + id, "score", -1); // 4
}
  1. Create an upVote function, which has the link ID as the argument.
  2. Use the command HINCRBY to increment the field score value.
  3. Create a downVote function, which has its link ID as the argument.
  4. Use the HINCRBY command to decrement the field score value. There is no HDECRBY command in Hash. The only way to decrement a Hash field is by using HINCRBY and a negative number.

The function showDetails shows all the fields in a Hash, based on the link ID:

function showDetails(id) { // 1
  client.hgetall("link:" + id, function(err, replies) { // 2
    console.log("Title:", replies['title']); // 3
    console.log("Author:", replies['author']); // 3
    console.log("Link:", replies['link']); // 3
    console.log("Score:", replies['score']); // 3
    console.log("--------------------------");
  });
}
  1. Create a function showDetails that has link ID as the argument.
  2. Use the HGETALL command to retrieve all the fields of a Hash.
  3. Display all the fields: title, author, link, and score.

Use the previously defined functions to save two links, upvote and downvote them, and then display their details:

saveLink(123, "dayvson", "Maxwell Dayvson's Github page", "https://github.com/dayvson");
upVote(123);
upVote(123);
saveLink(456, "hltbra", "Hugo Tavares's Github page", "https://github.com/hltbra");
upVote(456);
upVote(456);
downVote(456);

showDetails(123);
showDetails(456);

client.quit();

Then execute hash-voting-system.js:

$ node hash-voting-system.js
Title: Maxwell Dayvson's Github page
Author: dayvson
Link: https://github.com/dayvson
Score: 2
--------------------------
Title: Hugo Tavares's Github page
Author: hltbra
Link: https://github.com/hltbra
Score: 1
--------------------------

Tip

The command HGETALL may be a problem if a Hash has many fields and uses a lot of memory. It may slow down Redis because it needs to transfer all of that data through the network. A good alternative in such a scenario is the command HSCAN.

HSCAN does not return all the fields at once. It returns a cursor and the Hash fields with their values in chunks. HSCAN needs to be executed until the returned cursor is 0 in order to retrieve all the fields in a Hash:

$ redis-cli
127.0.0.1:6379> HMSET example "field1" "value1" "field2" "value2" "field3" "value3"
OK
127.0.0.1:6379> HSCAN example 0
1) "0"
2) 1) "field2"
   2) "value2"
   3) "field1"
   4) "value1"
   5) "field3"
   6) "value3"

Summary

This chapter began with information about Redis's history and some of its design decisions. We explained how to install Redis and demonstrated that the redis-cli tool can be a very powerful tool for debugging and learning Redis.

Some examples in this book that require a programming language are implemented in Node.js. Therefore, a quick reference to JavaScript's syntax and Node.js installation were shown.

Redis data types is an extensive subject, and it has been split into two chapters. This chapter explained how Strings, Lists, and Hashes work. The next chapter will cover Sets, Sorted Sets, Bitmaps, and HyperLogLogs and give practical examples.

Left arrow icon Right arrow icon

Description

Redis is the most popular in-memory key-value data store. It's very lightweight and its data types give it an edge over the other competitors. If you need an in-memory database or a high-performance cache system that is simple to use and highly scalable, Redis is what you need. Redis Essentials is a fast-paced guide that teaches the fundamentals on data types, explains how to manage data through commands, and shares experiences from big players in the industry. We start off by explaining the basics of Redis followed by the various data types such as Strings, hashes, lists, and more. Next, Common pitfalls for various scenarios are described, followed by solutions to ensure you do not fall into common traps. After this, major differences between client implementations in PHP, Python, and Ruby are presented. Next, you will learn how to extend Redis with Lua, get to know security techniques such as basic authorization, firewall rules, and SSL encryption, and discover how to use Twemproxy, Redis Sentinel, and Redis Cluster to scale infrastructures horizontally. At the end of this book, you will be able to utilize all the essential features of Redis to optimize your project's performance.

Who is this book for?

If you are a competent developer with experience of working with data structure servers and want to boost your project's performance by learning about features of Redis, then this book is for you.

What you will learn

  • Build analytics applications using Bitmaps and Hyperloglogs
  • Enhance scalability with Twemproxy, Redis Sentinel, and Redis Cluster
  • Build a Time Series implementation in Node.js and Redis
  • Create your own Redis commands by extending Redis with Lua
  • Get to know security techniques to protect your data (SSL encryption, firewall rules, basic authorization)
  • Persist data to disk and learn the trade-offs of AOF and RDB
  • Understand how to use Node.js, PHP, Python, and Ruby clients for Redis
  • Avoid common pitfalls when designing your next solution

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Sep 08, 2015
Length: 230 pages
Edition : 1st
Language : English
ISBN-13 : 9781784392451
Languages :
Tools :

What do you get with a Packt Subscription?

Free for first 7 days. $19.99 p/m after that. Cancel any time!
Product feature icon Unlimited ad-free access to the largest independent learning library in tech. Access this title and thousands more!
Product feature icon 50+ new titles added per month, including many first-to-market concepts and exclusive early access to books as they are being written.
Product feature icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Product feature icon Thousands of reference materials covering every tech concept you need to stay up to date.
Subscribe now
View plans & pricing

Product Details

Publication date : Sep 08, 2015
Length: 230 pages
Edition : 1st
Language : English
ISBN-13 : 9781784392451
Languages :
Tools :

Packt Subscriptions

See our plans and pricing
Modal Close icon
$19.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
$199.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts
$279.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total $ 153.97
Redis Essentials
$43.99
Mastering Redis
$60.99
Learning Redis
$48.99
Total $ 153.97 Stars icon

Table of Contents

10 Chapters
1. Getting Started (The Baby Steps) Chevron down icon Chevron up icon
2. Advanced Data Types (Earning a Black Belt) Chevron down icon Chevron up icon
3. Time Series (A Collection of Observations) Chevron down icon Chevron up icon
4. Commands (Where the Wild Things Are) Chevron down icon Chevron up icon
5. Clients for Your Favorite Language (Become a Redis Polyglot) Chevron down icon Chevron up icon
6. Common Pitfalls (Avoiding Traps) Chevron down icon Chevron up icon
7. Security Techniques (Guard Your Data) Chevron down icon Chevron up icon
8. Scaling Redis (Beyond a Single Instance) Chevron down icon Chevron up icon
9. Redis Cluster and Redis Sentinel (Collective Intelligence) Chevron down icon Chevron up icon
Index Chevron down icon Chevron up icon

Customer reviews

Top Reviews
Rating distribution
Full star icon Full star icon Full star icon Full star icon Half star icon 4.6
(18 Ratings)
5 star 72.2%
4 star 22.2%
3 star 0%
2 star 0%
1 star 5.6%
Filter icon Filter
Top Reviews

Filter reviews by




Francisco Oct 29, 2015
Full star icon Full star icon Full star icon Full star icon Full star icon 5
This is a really good Redis book. I do have a good experience working with Redis, and was still able to profit from this book. The book goes from the very basic (installing Redis and running the first command with redis-cli) to more advanced features and techniques (using Redis as a time-series database or the inner workings of Redis Cluster and Sentinel), always with a good coverage of real-world examples.I also liked the way the authors presented the problems related to replication and sharding and then presented the official tools for that, giving a good notion on how the actual history happened, as Redis wasn't designed in the first place with replication and sharding in mind. The authors do show to have large experience in the topic, and the book is also very well written. Definitely a good book to have in the collection.
Amazon Verified review Amazon
Jim Fathman Oct 25, 2015
Full star icon Full star icon Full star icon Full star icon Full star icon 5
This is the Redis book I have been waiting for. It introduces fundamental operations using the Redis command line interface, and then shows useful examples of the same operations using Node.js. The combination of Node.js and Redis is a popular mix, so the approach in this book should appeal to a large audience of server side developers.The writing quality and editing is uncommonly good in this book. I am a fan of PACKT books which are frequently first to market for new technologies, but they are sometimes rushed and written by amateurs. Not so here. The authors and reviewers are legit, have solid technical chops, and it shows.I buy a lot of technical books from Amazon, read them, and sell them back to clear shelf space. I keep a few of the better books. This book is a keeper. I hope the authors will return with more excellent writing about other parts of the Node.js ecosystem.
Amazon Verified review Amazon
Amazon Customer Jun 07, 2017
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Lots of basic and not-so-obvious info.
Amazon Verified review Amazon
Steve L. Jan 05, 2018
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Fast delivery, would use again.
Amazon Verified review Amazon
wei lu Mar 19, 2016
Full star icon Full star icon Full star icon Full star icon Full star icon 5
I like the content, although the book cover design needs a bit work; I've been using redis for a while, it was a great read, learn bunch tricks that I didn't know, and really like couple of interesting real world examples! Thumbs up!
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

What is included in a Packt subscription? Chevron down icon Chevron up icon

A subscription provides you with full access to view all Packt and licnesed content online, this includes exclusive access to Early Access titles. Depending on the tier chosen you can also earn credits and discounts to use for owning content

How can I cancel my subscription? Chevron down icon Chevron up icon

To cancel your subscription with us simply go to the account page - found in the top right of the page or at https://subscription.packtpub.com/my-account/subscription - From here you will see the ‘cancel subscription’ button in the grey box with your subscription information in.

What are credits? Chevron down icon Chevron up icon

Credits can be earned from reading 40 section of any title within the payment cycle - a month starting from the day of subscription payment. You also earn a Credit every month if you subscribe to our annual or 18 month plans. Credits can be used to buy books DRM free, the same way that you would pay for a book. Your credits can be found in the subscription homepage - subscription.packtpub.com - clicking on ‘the my’ library dropdown and selecting ‘credits’.

What happens if an Early Access Course is cancelled? Chevron down icon Chevron up icon

Projects are rarely cancelled, but sometimes it's unavoidable. If an Early Access course is cancelled or excessively delayed, you can exchange your purchase for another course. For further details, please contact us here.

Where can I send feedback about an Early Access title? Chevron down icon Chevron up icon

If you have any feedback about the product you're reading, or Early Access in general, then please fill out a contact form here and we'll make sure the feedback gets to the right team. 

Can I download the code files for Early Access titles? Chevron down icon Chevron up icon

We try to ensure that all books in Early Access have code available to use, download, and fork on GitHub. This helps us be more agile in the development of the book, and helps keep the often changing code base of new versions and new technologies as up to date as possible. Unfortunately, however, there will be rare cases when it is not possible for us to have downloadable code samples available until publication.

When we publish the book, the code files will also be available to download from the Packt website.

How accurate is the publication date? Chevron down icon Chevron up icon

The publication date is as accurate as we can be at any point in the project. Unfortunately, delays can happen. Often those delays are out of our control, such as changes to the technology code base or delays in the tech release. We do our best to give you an accurate estimate of the publication date at any given time, and as more chapters are delivered, the more accurate the delivery date will become.

How will I know when new chapters are ready? Chevron down icon Chevron up icon

We'll let you know every time there has been an update to a course that you've bought in Early Access. You'll get an email to let you know there has been a new chapter, or a change to a previous chapter. The new chapters are automatically added to your account, so you can also check back there any time you're ready and download or read them online.

I am a Packt subscriber, do I get Early Access? Chevron down icon Chevron up icon

Yes, all Early Access content is fully available through your subscription. You will need to have a paid for or active trial subscription in order to access all titles.

How is Early Access delivered? Chevron down icon Chevron up icon

Early Access is currently only available as a PDF or through our online reader. As we make changes or add new chapters, the files in your Packt account will be updated so you can download them again or view them online immediately.

How do I buy Early Access content? Chevron down icon Chevron up icon

Early Access is a way of us getting our content to you quicker, but the method of buying the Early Access course is still the same. Just find the course you want to buy, go through the check-out steps, and you’ll get a confirmation email from us with information and a link to the relevant Early Access courses.

What is Early Access? Chevron down icon Chevron up icon

Keeping up to date with the latest technology is difficult; new versions, new frameworks, new techniques. This feature gives you a head-start to our content, as it's being created. With Early Access you'll receive each chapter as it's written, and get regular updates throughout the product's development, as well as the final course as soon as it's ready.We created Early Access as a means of giving you the information you need, as soon as it's available. As we go through the process of developing a course, 99% of it can be ready but we can't publish until that last 1% falls in to place. Early Access helps to unlock the potential of our content early, to help you start your learning when you need it most. You not only get access to every chapter as it's delivered, edited, and updated, but you'll also get the finalized, DRM-free product to download in any format you want when it's published. As a member of Packt, you'll also be eligible for our exclusive offers, including a free course every day, and discounts on new and popular titles.