Mongoose for Application Development

Chapter 1. Introducing Mongoose to the Technology Stack

We are going to take a look at the technology stack we'll be using throughout the book. After a brief discussion of Node, npm, MongoDB, and Express we will introduce Mongoose as an ODM (Object-Document Modeler), cover good and bad use cases, and introduce the two cornerstones of Mongoose.

By the end of this chapter you will have an understanding of the technology stack and where Mongoose fits in. We will also have set up the sample project that we will build throughout the book.

The technology stack – Node.js, npm, MongoDB, and Express

The benefits of Node are making it ever more popular, and a set of compatible technologies are fast becoming the basis of a new standard development stack.

The language and the server – JavaScript and Node

For quite some time JavaScript was mainly thought of as a lightweight browser-based scripting language. Microsoft did support a JavaScript version of Classic ASP, but it was largely side-lined in favor of the VBScript version.

Fast-forward 10 years and there were some pretty impressive JavaScript-based apps on the Web, for example Gmail. The general view of JavaScript as a programming language was starting to change back then.

In 2010 a new server-side system called Node.js was starting to make waves in the Internet developer community. Node.js was a new way for using JavaScript on the server side again. And it was quick and efficient. It could make scaling a web application much more cost-effective by reducing the amount of hardware required per-site visitor or request.

By 2012 Node was one of the buzzwords of the start-up scene, and you can see why. Firstly, most Web developers have some JavaScript experience so it doesn't require you to learn a new language from scratch. This also means you can share some code between front-end and back-end, so you don't have to code the same thing twice in two different languages. An excellent example of this is form validation; you want real-time validation in the browser for a better user experience, but you also need to validate on the server side to protect your system. So you code the same thing twice. Using Node you can use the same validation script on the server side and the browser, so you only have to code it once, in JavaScript.

Second, there is the reduced cost of scaling, especially when dealing with large numbers of concurrent users. On the same hardware Node's efficiencies allow it to handle many more requests than classic stacks on Apache or IIS. That being said, adding scalability to Node is more complicated than other stacks. Unlike the others you can't just put it on a more powerful machine and set it running. By default Node will currently only run one process, using only one core of a machine. There are methods to address this issue, using a load balancer in front of several processes running alongside each other for example, and there are plans to enable future versions of Node to be able to manage this natively, directly addressing this issue.

The benefits of scalability do have a cost. The single process is a more complicated approach to server-side programming and requires a change in mindset.

Single-threaded versus multithreaded

Traditional stacks are generally multithreaded. This means that every new visitor or session is given a new thread, and these are never shared. One session's activity generally doesn't impact another, until the server resources are exhausted. For example, if Session 1 is doing a complex database write operation it may take a couple of seconds, but Session 2 continues running oblivious to this.

Node is single-threaded. This means that every visitor or session is added to that one thread. So it is possible for a two-second database write operation to hold up every other user for two seconds. Multiply this by just 10 users and you've got a big problem on your hands.

Addressing this requires a different way of coding.

Blocking versus non-blocking code

In the traditional stack, the approach to coding would be one step after the other, as in the following steps:

First, take the data.
Then write this data to the database.
Send a confirmation message.
Wait for the next request.

This is blocking code, as you can only do one thing at a time. This is fine in the multithreaded stack, as you're only ever responding to one person's requests.

In the single-threaded stack, you may have to respond to several people's requests at the same time, so you can't afford to be stuck doing time-consuming operations or waiting for someone else to do something. To do this, the approach to coding becomes more like the following:

You give this data to someone.
They write this data to the database.
When they are done, they send a confirmation message; if this isn't something they can do, then they add it to your request list.
You're going to take the next request.

This is non-blocking code. You can only do one at a time. So you're getting someone else to do something for you, and telling them what to do when they have finished. You can then deal with the next request coming in without any delay.

JavaScript callbacks

The way to code this in JavaScript is to use callbacks. Most JavaScript coders start using them before they even know it, particularly anybody who uses libraries such as jQuery.

Take the basic jQuery document.ready method as shown in the following:

$(document).ready(function() {
  console.log("document ready");
});

This is an event driven callback. The $(document).ready() part is a method function of jQuery, and we are sending it a function function() that it can run at the appropriate time. So we are saying "Hi ready, here is what I want you to do once the document is ready, I'll leave it up to you to decide when that is". The callback function we are using in this example is the following code snippet:

function() {
  console.log("document ready");
}

Running the callback

The jQuery .ready() function is pretty complicated, so we're not going to look at that here. However, the construct is very useful to understand. Look at the following code snippet:

ready = function (callback) {
  // do something
  // do something else
  // ....
  // and so on
  callback();
};

So ready itself is a function, and this function accepts one parameter callback. The callback parameter is generally an anonymous function, like the one we looked at earlier. A very important point to note is that callback now exists in the scope of the ready function. This means that your callback function has access to any variables or objects created in the ready function.

A Node.js example

Now consider the following standard Node "hello world" example:

var http = require('http');
http.createServer(function (req, res) {
  res.writeHead(200, {
    'Content-Type': 'text/plain'
  });
  res.write('Hello world');
res.end();
})
listen(8888, '127.0.0.1');

Look familiar? This is sending a callback to the http.createServer method function. See how the parameters—req and res—are being sent to the callback function even though they haven't been defined or created anywhere. This works because the http.createServer function will create these objects before calling this callback function that will use them.

The database – MongoDB

MongoDB has become the main database of choice for working with Node. Note that there are Node drivers for many other databases, including MySQL, Microsoft SQL Server, Reddis, PostgreSQL, CouchDB, and more.

MongoDB is popular as it is fast and flexible with excellent community support. It is a document-oriented database, fitting in somewhere between Key-Value stores and traditional relational databases. Despite being a document store, MongoDB also enables rich querying and secondary indexing of documents, setting it apart from other databases and making it a very powerful option.

MongoDB stores documents as BSON, which is effectively binary-encoded JSON. When you run a query you get a JSON object returned (or a string in JSON format, depending on the driver). Look at the following code snippet for example:

{ "_id" : ObjectId("4ffbc45c35097b5a1583ad71"), "firstname" : "Simon", "lastname" : "Holmes }

So, a document is a set of keys (for example, firstname) and values (for example, Simon). The _id entry is a unique identifier that the underlying MongoDB driver will—by default—create for each new document.

If you are more experienced with relational databases, it may help you to think of a document as a bit like a row in a table. In this analogy, the key can be thought of as a column. An important difference is that each document doesn't have to contain the exact same set of keys, and there is no direct need to have keys with empty values taking up space.

A collection of documents is called a collection. The closest analogy is a table. So in your database you could well have multiple collections, such as a users collection, posts collection, and stats collection.

MongoDB is also extremely scalable, with many built-in capabilities for distributing across multiple servers, without compromising speed or data integrity.

With everything combined, it makes MongoDB a great playmate for Node.

The framework – Express

Express is a web application framework for Node.

When you create a Node project, you have to do a lot more groundwork than you might be used to. Until you create it, there isn't even a web server. Then you have to deal with serving static files in a non-blocking way, figure out the routing mechanism, view engine, session management, and so on.

Or you can create an Express project and let it do all of this for you, in a tried-and-tested way. At the end of this chapter, we'll see how easy it is to set up an Express project.

Note that Express is not required to build a Node application, but it is a great starting point for building web applications.

What Mongoose is all about

Mongoose is an object modeling tool for MongoDB and Node.js. What this means in practical terms is that you can define your data model in just one place, in your code.

Yes, that's right. You don't have to create a schema in the database, link that to an ORM or map it into your project objects and classes. You can just define your data structure in JSON inside your project.

The first time I created a project like this I was amazed at how much time and frustration it saves. Even now I still get that warm glow when I start a new project or prototype using Mongoose. It's like taking a shortcut to work down deserted country lanes while everybody else is gridlocked on the highway.

A schema definition can be as simple as the following code snippet:

var userSchema = new mongoose.Schema({
  firstname: String,
  lastname: String,
  createdOn: Date
});

A document in MongoDB created from this schema would be like the following code snippet:

{ "__v" : 0, "_id" : ObjectId("51412597e8e6d3e35c000001"),"createdOn" : ISODate("2013-03-14T01:19:19.866Z"),"firstname" : "Simon", " lastname " : "Holmes" }

If you want to refactor, then you can just do it from within your code, saving a huge amount of development time.

What is Mongoose good for?

Mongoose is primarily useful when you want to interact with structured data in MongoDB. It allows you to define a schema for your data, so that you can interact with your MongoDB data in a structured and repeatable way.

Mongoose helps with many common MongoDB tasks, and removes some of levels of complexity from the nested callbacks you find yourself getting lost in with the native MongoDB driver.

Mongoose also returns the data to you as a JSON object that you can use directly, rather than the JSON string returned by MongoDB.

Mongoose also has a whole suite of helper functions and methods that we'll explore throughout the subsequent chapters of this book.

What Mongoose is not ideally suited for

Mongoose is probably not the answer for you if you are primarily working with the following:

Schema-less data
Random documents
Pure Key-Value pairs

The cornerstones of Mongoose

There are two aspects of Mongoose that we need to introduce you to before going much further:

Schemas
Models

This is a very high-level view; so don't worry if you're not 100 percent confident with this just yet, as we'll be covering it in a lot more detail later in this book.

Mongoose schemas

As we saw earlier, a schema is fundamentally describing the data construct of a document. This schema defines the name of each item of data, and the type of data, whether it is a string, number, date, Boolean, and so on.

var userSchema = new mongoose.Schema({
  name: String,
  email: String,
  createdOn: Date,
  verified: Boolean
});

In most scenarios you would have one schema for each collection within the database.

Schemas are a powerful aspect of Mongoose, which can also be extended with helper functions and additional methods. But we'll describe more about that in a later chapter.

Mongoose models

A model is a compiled version of the schema. One instance of the model will map to one document in the database.

Creating a User instance based on the schema userSchema is a one line task:

var User = mongoose.model('User', userSchema);

It is the model that handles the reading, creating, updating, and deleting of documents.

Installing the full stack

Now that we know a little bit about the technology, it's time to get everything installed. What follows is not an in-depth installation tutorial, but covers the necessary steps to get everything installed on Ubuntu. The installation process is very similar on other Linux distributions and Mac OS X. For Windows users, Node and MongoDB have installation packages you can run, and after that everything else is pretty much the same.

Installing the prerequisites

Node needs a couple of things installed to run properly, Python and a C compiler. So let's make sure those are installed and up-to-date with a terminal command. I use the Ubuntu Advanced Packaging Tool (APT) to install most software, as it makes it this easy:

$ sudo apt-get install python-software-properties python g++ make

Installing Node.js

Again we'll use APT to install Node.js. As Node is actively developed and frequently updated, APT doesn't always have the latest build. The latest version on APT is currently several versions behind, so we will add the repository of one of the Node.js developers. This is also the approach recommended on the Node website nodejs.org.

$ sudo add-apt-repository ppa:chris-lea/node.js
$ sudo apt-get update
$ sudo apt-get install nodejs

If everything has gone smoothly then you should be able to check the installed version displayed in terminal using the following command:

$ node --version

You should be greeted with a message back in the terminal along the lines of v0.10.0.

Installing npm

npm is a package manager for Node, although npm does not actually stand for Node Package Manager. As of Node Version 0.6.3, npm is actually automatically installed when you install Node.

You can quickly check that this has been done by checking the version number on the terminal.

$ npm --version
1.1.23

If it is not there for some reason, simply install it with APT with the following command:

$ sudo apt-get install npm

Installing MongoDB

Again we're using APT here.

$ sudo apt-get install mongodb

In order to test MongoDB, we need to run it. You can do this by just typing mongodb into terminal, but I prefer to run it as a persistent service so that you don't have to restart it.

$ sudo service mongodb start

This should give you a confirmation message in terminal that the mongodb process is running.

Now to finally test that it has installed correctly we'll drop into the built-in MongoDB shell. In terminal, enter the following:

$ mongo

This should do three things:

Enter you into the MongoDB shell
Show you the version of the shell being used
Connect to the test database that comes with MongoDB

You should see something like the following screenshot:

Installing Express.js

Now we're starting to get into the Node-based applications. So those of you who are working on different operating systems can start paying attention again now. We are going to install Express using npm.

$ sudo npm install -g express

Note

See the -g flag there in the command line? This installs express globally, meaning that we can use it wherever we want.

Once again you can test the installation by checking the version number.

$ express --version

Installing Mongoose

Finally we get round to installing Mongoose!

There are two ways of installing Mongoose, both using npm. I recommend using the latter one.

Direct installation into project

If you're creating a quick test app, or will never have to move or publish the project elsewhere, you can use npm to install Mongoose directly into your project. Navigate to the root folder of your site or project in terminal and run the following command:

$ sudo npm install mongoose

Using project dependencies – package.json

Any Node project can have a package.json file, and any packages you install via npm will have one. This file is normally in the root of your project, and can contain various metadata, such as project name, description, version number, and authors. It also helps manage the project's dependencies.

If you ever need to move, copy, or distribute your project you will find this invaluable. The dependency management means that you are not forced to remember which versions of which modules you added to each particular project. It becomes even more necessary if you distribute it widely, as it removes any doubt from the end-user's mind as to what the project needed.

Before we get into creating one, the following code snippet is an example of a package.json file following code snippet that you might expect to see when you create a new project using Express.

{
  "name": "application-name",
  "version": "0.0.1",
  "private": true,
  "scripts": {
    "start": "node app"
  },
 "dependencies": {
  "express": "3.0.0rc5",
  "jade": "*"
 }
}

Note

The list of dependencies are: a specific version of express and the latest version of jade.

To install a specific version of Mongoose—which is the recommended approach, especially if you are going to distribute your project—you would update the dependencies section to look like the following code snippet:

"dependencies": {
  "express": "3.0.0rc5",
  "jade": "*",
  "mongoose": "3.6"
}

To install or update all of the dependencies listed in a package.json file, using terminal, you simply navigate to the folder containing the file and run the following command:

$ npm install

Creating a project

Now that we have the entire building blocks ready, let's set the foundations. Throughout this book, we are going to build a simple project management web app, called MongoosePM.

Let's start off by creating an Express project.

In terminal, navigate to a relevant folder, for example, /MyWebsites in Linux, or C:\Users\Your Username\Documents\My Web Sites\ in Windows.
Enter the following command:
```
$ express --sessions mongoosepm
```

This will create a new folder called mongoosepm. Inside this folder it will create a new Express project. By default an Express project doesn't support user sessions; an Express project is essentially stateless. By adding the --sessions parameter in the command line we add session support to our project, so that we can follow a user from page to page without having to constantly re-authenticate.

After Express has finished doing its thing you'll be able to see a whole load of new files and folders within your mongoosepm folder. The intricacies of Express are beyond the scope of this book, which is one of the reasons we have mainly stuck with the default settings.

The next step is normally to install the dependencies, but first let's add Mongoose and Connect to the dependency list. So open up the package.json file in the root of the project and add a couple of lines to the dependencies hash.

For example, if your dependencies section looks like the following code snippet:

"dependencies": {
  "express": "3.0.0rc5",
  "jade": "*"
}

Then you change it with this (see the following code):

"dependencies": {
  "express": "3.0.0rc5",
  "jade": "*",
  "connect": "2.7.x",
  "mongoose": "3.6.x"
}

It is recommended that you don't "wildcard" the version for most modules, as there is no guarantee that code based on older versions will work on newer versions. Putting a wildcard in for the patch version like we have for Connect and Mongoose, is recommended as they should only be patch updates and be backwards compatible.

To install these dependencies, go to the mongoosepm folder in terminal and run the following command:

$ npm install

Once this has finished working, we should have a fully functional Express website to play with. Let's test it to make sure!

Run the following command from terminal:

$ node app

You should see the following confirmation message directly in terminal:

Express server listening on port 3000

So open up a web browser, and point it to http://localhost:3000. All being well, you should be presented with the default Express project landing page, as shown in the following screenshot:

Tip

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

Filter reviews by

All

Amazon verified reviews

Alex B Jul 14, 2014

Amazing book. There are a lot of tips and tricks in there and if you use mongoose a lot you will actually find a lot of cool coding secrets and explanations to weird things that go on under the hood. Its a must read if you like Mongoose.

Amazon Verified review

CharlieBear Apr 30, 2014

This is an excellent book for anyone moving from C#/.NET or any other non-web based development platform. It is a great way to get a quick start if you want to jump onto the Javascript bandwagon and get ready for web apps that run and behave like Windows apps.The book covers everything you need to build a browser based application (or mobile apps too) with all code being Javascript. The examples are easy to follow and to the point, including the traditional CRUD application. Routes, Jade templates, JSON, AJAX, data validation and plugins are all also covered. It's just a good book to get started writing client and server side applications. Also covers a lot of the different methods for fetching and working with data. Mongoose has a "code first" database creation feature that is very cool.There is one problem due to changes made to Node.js launch for Express/Mongoose since the date of publication. I'd recommend skipping the getting started instructions in the book (chapter 1) and use this web site instead, especially if you are running with MS Windows:[...]I found the above URL after following the book's instructions, step-by-step (using Windows 7) and couldn't get Express to fire up. Other than that, I'd recommend using this book for anyone who wants to get started writing highly responsive web based/mobile applications.

Robby May 07, 2014

This book helped me a lot. It's small, to the point. The code works. It got me up to speed. Good job

Konstantin Sep 26, 2013

I bought this book from the publishers site to help me get up to speed with using Mongoose, Node.js, MongoDB, and Express. I must say i am very impressed with it. The book is very well written, and the examples work as expected. (unlike other programming books you buy). The sample project in the book comes in very handy and teaches you how to build a CRUD like application using these technology stack. Note that the major focus on this book is mongoose! Though you get to understand how the other stack work, they are not in-depth. Mongoose however is cover pretty deeply.

Eric Rabbath May 02, 2014

This is an excellent short book on using Mongoose to connect your MongoDB back-end database to your Node.js application. The explanations are very clear and to the point, and the examples given are always an excellent illustration of the point being discussed.The ongoing example is also well chosen, and well used to integrate the various concepts into a complete working application. It uses Express to build the application, and while the explanations in the book are very clear and complete, a basic understanding of the key concepts of the Express framework (routes, request, response objects) is useful to really understand what is going on.As mentioned by others, the installation instructions are a little out of date, but this can very easily be solved with a simple web search.

Mongoose for Application Development: Mongoose streamlines application development on the Node.js stack and this book is the ideal guide to both the concepts and practical application. From connecting to a database to re-usable plugins, it's all here.

What do you get with eBook?

Contact Details

Billing Address

JavaScript callbacks

Running the callback

A Node.js example

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with eBook?

Contact Details

Billing Address

Product Details

Packt Subscriptions

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

Filter reviews by

People who bought this also bought

About the author

FAQs