Top 10 features you need to know about
We created and published our very first node module in the previous section. In this section we will learn tips and tricks to use features provided by npm to our advantage. The npm command-line utility offers a range of functionalities to help smoothen the development and deployment of node modules. Let's find out how can we exploit them. Here are the 10 things you should know.
Before we proceed, remember the glossary term prefix.
Note
Prefix: {prefix} is the full path where node.js installation is kept on the system. On most of the systems, this is /usr/local
. On Windows systems it is usually C:/Users/Username
. So wherever you see the word prefix inside curly braces, think of this definition.
As we know that npm keeps multiple versions of a node module, we can associate a tag, which is a short name, to a particular version. For example, many popular node modules usually keep their latest stable version under a stable tag. npm automatically tags the most recent version of a node module as latest. Command for tagging is:
Remember we are tagging a specific version of a package with the tag name so specifying both package-name
and version
are mandatory. The specified version should have been published before tagging it on npm registry. Providing a tag name is optional, if not, present the npm defaults to latest.
We have already seen npm install
in action in the previous section. Let's revisit this command and see what we missed.
From inside a module directory which contains a valid package.json
, installs all the packages listed as dependencies
and devDependencies
inside package.json
, into the ./node_modules
folder. If no package.json
exists at current path, npm will exit with an error. The version of the package that will be installed, depends on as specified in package.json
. The dependency package's names are specified as keys of dependencies
, devDependencies
, and
optionalDependencies
property and their values indicate version. The version follows semver semantics. * means latest available version. Some more are:
> or >=, that is, greater than or greater than or equal to the specified version
< or <=, less than or less than or equal to the specified version
A version range, for example, >=0.5 <=1.0
An exact version string, for example, 1.2.3-beta.
Take a look at the sample package.json
:
In this project, exact Version 0.9.8 of the oauth
package will be installed. Latest available express package will be installed. Latest mongoskin
available after 0.2.0 (inclusive) will be installed. Lodash Version less than 1.0.0-rc.3 will be installed. Any passport version available between 0.1.0 (not including 0.1.0 itself) and 0.1.15 (inclusive) will be installed with recent version taking precedence over earlier versions.
To understand semver (semantic versioning) semantics, that is exactly how package versions are sorted or which version pattern is considered recent than another, checkout http://semver.org/ and https://npmjs.org/doc/semver.html.
Note: npm version semantics differ slightly from semver:
Pass to the argument either a folder name or a tarball name on filesystem containing a valid node module or a URL that serves such a tarball. This and subsequent commands accepts space separated list of dependencies to install in any combination:
Looks up the latest version of <package-name>
in the npm registry and installs it into the local node_modules
directory.
Optionally provide --save, --save-dev, --save-optional
switches to automatically update corresponding section in package.json
. These switches are valid for subsequent install commands mentioned. devDependencies
are dependencies which are required during development of the module but not while using it in production, for example, a testing framework, or assertion library which are required during development but are never used while actually running the code in production.
An optionalDependency
is one without which a module should be able to run successfully. For example, the redis
module comes with hiredis
specified as an optional dependency. The hiredis
module is a native module written in C++ that parses response from the redis server. If somehow hiredis
could not be installed, redis uses JavaScript parser instead. So the redis
module is able to function properly even without its optional dependency.
The preceding command installs the package with a given version number, tag, or version range. You can also specify version ranges, for example, $ npm install mocha@">0.2.0 <=0.3.0"
. As discussed previously, they will be resolved according to semver semantics.
For users who use git for version control, which is very likely since most node.js developers use git, you can directly install a package by cloning its git repository. By default, latest commit to master branch is fetched. You can specify a particular commit hashtag or git tag by appending #<commit-hashtag | git-tag>
in the URL. One reason you would want to use a git URL directly is when the code is not published on npm, for example, when it is not an open source module and is not available on npm registry. Although you can have your own private npm registry, that will come at a cost of some technical overhead. This is the simplest way of obtaining private code at the moment. The downside of using a git URL directly is that you lose semver versioning, and installation is usually slower than installing from npm repository directly. While using a git URL, it is advisable to use a tag rather than simply a git URL, as it gives some control over the code that comes in. If you simply save a git URL in your package.json
, whatever committed latest to the branch specified (master by default) will be installed and that could be something broken. Tags are usually stable releases, which can be relied upon.
The next command is issued from inside a project. It updates all the dependencies inside the project to the latest. Providing a list of package names or unique identifiers, it only updates those packages. Provide a --global
or -g
flag to update globally installed packages:
The next command uninstalls all the packages, list of package names, or package identifiers, provided:
Binaries and global installs
A lot of software programs provide a command-line interface to use them. For example, npm itself is an excellent command-line program. Similarly, express and tower.js provide command-line helpers that create scaffoldings for web applications. The files containing the programs that run on a command from the command line are loosely referred to as binaries or executables.
To use binaries provided by external modules, express, for example, install them globally. Any package can be installed from npm registry globally by passing --global
or -g
command-line flag to the install command. For example:
The previous command installs express
globally. The executable will be available on the command line.
The previous command will create a new folder myapp
and create express scaffold for a minimal express web application.
To include an executable in a node module, it is required to include a file that can parse the command-line arguments and provide a command-line entry point to the application. The binary will probably call the programmatic API of the node module.
It is required to register the binary entry point in package.json
. Take a look at express's reduced package.json
.
The package.json
tells us that express keeps its binary in express/bin
folder with the name express and if we care to look inside express's directory structure we will find that to be the case. Let's try to create a binary ourselves. Let's extend our simplemath
library and provide a command-line API for just adding two numbers.
First make package.json
aware of a binary:
The highlighted code in the preceding code snippet, as we will see, will expose the ./bin/simplemath-add
file (binary) to the command line from the system's binary folder.
Now add a file ./bin/simplemath-add
to the project with the following contents.
The first line in the previous program is called a shebang. In *nix environments, it is used to inform the command line to treat the rest of the file as binary of the given program. We tell it to treat our file as a node.js binary. Let us run the program and see. First provide execute permission to the file if you are on a *nix machine:
And it works perfectly well. But wait, why do we need to give the full path of our binary? We want to be able to access the command-line API globally, from any folder, without needing to type the full path. We need to install our package globally. After we publish this code to npm registry and then install simplemath
from registry with a --global
or simply -g
flag, we will be able to do so. You can try it if you want.
You do not necessarily have to publish and install/update your package globally from npm registry, there is another way to do this in development environment. That is through linking, which is our next topic.
The next command shows the folder in which globally installed module's binaries are kept:
The next command will show path to binaries kept inside a local install:
The next command updates all globally installed packages. If a list of package names or identifiers are mentioned, updates only them:
The next command uninstalls a globally installed package:
Often you want to use the latest, bleeding edge version of your node module, or someone else's module, which hasn't been published yet. One option is to keep the latest project folder into node_modules
folder of another project. But that can quickly become taxing with deeply nested dependencies, or impossible to maintain if the node module is a circular dependency. Also imagine a case where we want to use the bleeding edge version of our node module in three different projects. Every time we make changes to our module, we will have to copy paste it in all three. npm has a better solution:
Linking allows you to bootstrap a project as a dependency in another, so that the changes in one immediately can be used or tested in the other. To understand this let us again take the example of our simplemath node module. Imagine we are building another node module called advancedmath
, that internally uses simplemath as a dependency. Perhaps we are building both modules simultaneously and we always want to use the latest version of simplemath
in advancedmath
.
So advancedmath
will perhaps, have following folder structure:
package.json
of advancedmath
should look like the following code snippet:
As we have already discussed, we won't simply keep simplemath
folder inside advancedmath/node_modules
folder. There are better ways to keep ourselves sane. We will link the simplemath
into advancedmath
. You will require admin privileges on the computer, to do this, refer to the following commands:
The previous command links simplemath
into a global installation and creates the simplemath binary that we have created in previous section, available globally. So we can do the following without publishing and installing our module globally from registry.
Now let us bootstrap it to advancedmath
, so that changes in simplemath
are immediately reflected in advancedmath
dependency of it.
The previous command has bootstrapped simplemath
into advancedmath
. It should be noted that any changes in binary would also affect the executable we are accessing from the command line.
If you are familiar with UNIX symlinks, it will probably interest you that a symlink to the globally installed simplemath
folder is created inside advancedmath/node_modules
directory. Hence any change is immediately reflected. Now we can independently develop the two modules without the advancedmath
developer worrying about keeping abreast with simplemath
, except for API changes of course. Notice that we didn't have to give the full path of simplemath
while linking it inside advancedmath
folder, although we may do so if we want, but the result will be same, simplemath
will be installed globally before being linked to advancedmath
.
Any folder installed globally, either by linking or by installing with -g
flag can be linked to other projects as dependencies.
We can remove the bootstrapping anytime by executing npm rm simplemath
or npm uninstall simplemath
from inside advancedmath
folder or by simply deleting the simplemath
symlink from advancedmath/node_modules
.
To remove global installation of the package, use the same command that is used to remove global installation of external packages.
As seen in the previous section, linking also symlinks the binaries included in the module to the system's global binary folder. On the Linux environments those may be included in /usr/bin, usr/sbin, /usr/local/bin
, usr/local/sbin
, and so on.
Bear in mind that linking does not work on Windows without cygwin.
If you have used git and know what .gitignore
does, .npmignore
does pretty much the same thing. It is used to provide lists of files and folders to exclude while publishing the package. For example, our module might be generating a lot of log information in a log file while being tested. We don't want to include those in deployable builds since they are irrelevant to other users and unnecessarily increase package size.
We also might have some configuration files containing sensitive information like usernames and passwords. We don't want to have to remember every time we deploy to delete or move those files out of the folder before publishing the module. .npmignore
is a newline delimited list of file names or patterns. The following patterns are recognized:
filename
: File or folder with this name anywhere in the module will not be included in the bundle. For example, we don't want to include test files in the production bundle so we can add tests.js
. We also don't want to include the node_modules folder because we expect npm to install all the dependencies recursively for our module users, so we might also include node_modules.
*.ext
: File ending in .ext. For example *.log will ignore all files ending in .log and hence all of the log files.
folder/<filename>
: Ignore file only in particular folder.
folder/*.ext
: File ending in .ext in a particular folder.
folder/*
: All files inside a particular folder.
folder/**/filename
: any number of folders between folder and file.
directory/
: Only matches a directory named directory and not files named directory.
Remember, lines starting with hash (#
) are treated as comments. To override that behavior, escape #
with a backslash (\
). For a comprehensive list of patterns visit http://git-scm.com/docs/gitignore. The page is for .gitignore
but same rules applies to .npmignore
as well.
A common .npmignore
file has the following content:
The above two patterns are ignored by default during npm publishing and don't need to be mentioned in the .npmignore
file explicitly.
A common npmignore for all packages for all the users can be defined at {prefix}/etc/npmignore
. This default can be overridden by specifying a file path in globalignoreconfig configuration. To know more about npm configuration please see the Config section.
npm allows you to run arbitrary scripts on certain events, for example, before a module gets downloaded, or before the module gets published. These scripts could be anything, for example, to clean up files or to clean up some system state. Following scripts are available from npm scripts official documentation:
prepublish
: Run before the package is published. (Also run on local npm install without any arguments.)
publish
, postpublish
: Run after the package is published.
preinstall
: Run before the package is installed.
install
, postinstall
: Run after the package is installed.
preuninstall
, uninstall
: Run BEFORE the package is uninstalled.
postuninstall
: Run after the package is uninstalled.
preupdate
: Run before the package is updated with the update command.
update
, postupdate
: Run after the package is updated with the update command.
pretest
, test
, posttest
: Run by the npm test
command.
prestop
, stop
, poststop
: Run by the npm stop
command.
prestart
, start
, poststart
: Run by the npm start
command. If there is a server.js
file present in the root of the package this command will default to node server.js
.
prerestart
, restart
, postrestart
: Run by the npm restart
command.
Note
npm restart will run the stop and start scripts if no restart script is provided.
Test
, start
, stop
, and restart
scripts are run when the commands are called directly from the command line. Remember we had delegated test scripts to npm in previous sections by providing node tests.js
to test
property of the scripts
object inside our project's package.json
. Similarly, we can provide scripts to be run through the command line for other events mentioned previously as well.
An interesting bit to know is that these npm scripts are aware of binaries inside ./node_modules/.bin
even if they aren't installed globally.
npm configuration are lists of key-value pairs comprising convenient defaults, information of user and computer, SSL certificates, and so on. For example, npm config stores the URL of standard registry; or npmjs username of a user who is currently logged in and its session information; or number of retries npm should make if a package installation fails for any reason. Let us check out what the registry URL is in npm config:
If the user wishes, he/she can change the default and provide a URL of a custom registry maintained by someone else or by themselves. The command for printing a config value on console is:
Or simply:
Command for changing a config setting is:
Or simply:
npm reads these values from a number of sources. The following are the sources from where npm picks up configs in the order of their precedence.
Config information provided in the command line is of highest precedence. For example, to install a package from an alternate registry:
The previous command doesn't persist the config
value. To provide any config information in a command, include a flag with the config key followed by the value you want to set for that command: --<key> <value>
.
Next, npm looks for a config value in the environment variable. The name of an environment variable corresponding to a config should be the config key preceded by npm_config_
. So for example, to provide the configuration for loglevel in an environment variable, the variable should be named npm_config_loglevel
.
Usually the file placed in $HOME/.npmrc
, where $HOME
is the logged-in user's home on the computer (/home/username
on Unix, C:/Users/Username
on Windows), or the userconfig configuration value set in either an environment variable or in the command line.
This file is an INI file, a formatted list of key-value pairs. We can use environment variables in this file by replacing with ${ENVIRONMENT_VARIABLE}
. So for example, to provide configuration for maximum number of retries, npm should make for installing a package, we would write following in the config
file:
Global config file is similar to user config file. But global config file is for all the users on the computer. This file should be placed at {prefix}/etc/npmrc
.
Defaults refer to the defaults from npm or node.js source. This is not meant to be changed by npm users but by distributors who maintain their own node.js or npm fork.
So these are the places from where npm reads the configuration. Following the command is to get a list of configs defined on a computer:
Provide a --long
or -l
switch to get the full list. To edit the user config file, execute the following command:
This will open the user config file ($HOME/.npmrc
) on the system's default text editor. On Linux this defaults to vi, for Windows, notepad is the default. To read man pages for config:
npm has this way of fetching the latest package available, according to the semver version-range we give it. Whether we are publishing our package to registry or using it internally within our organization, sometimes it is not a good idea to publish a module which will probably be used by many in production, with loosely packed dependencies. You might never know what hell might break loose with updates to any of your dependencies or your dependencies' dependencies and so on.
Sometimes it is useful to deploy a node.js application with dependencies locked to exact versions against which it was tested so that we can have predictable results in the end. Although we can manage to provide exact versions of each of our dependencies, we cannot expect the same from every author. Our dependencies and their dependencies might have dependencies loosely defined.
Shrinkwrap command, given it was run inside a valid npm package root, creates a file npm-shrinkwrap.json
in the project root. It is a .json
file similar to package.json
except that its dependencies
property recursively defines the exact version of each and every dependency that our project uses. The prerequisites of executing the shrinkwrap command are:
All the dependencies indicated in package.json
should be installed, no packages should be missing
No extraneous packages, that is other than those mentioned in package.json
dependencies should be installed
npm prune removes all the extraneous packages, that is, those packages not mentioned in package.json
.
Running npm install
inside a project or a package with npm-shrinkwrap.json
file will pick up dependency versions from the npm-shrinkwrap.json
file instead of package.json
.
To shrinkwrap a project, run the following command inside the project root.
You will find the mentioned file. Take a look at reduced npm-shrinkwrap.json
obtained on running shrinkwrap on mocha.
Take a look at jade and its dependency mkdirp
, the version of both the packages have recursively been locked down.
Tip
Run npm ls
in a project root and see the how npm beautifully prints the dependency tree on the console. It even indicates if there are unmet or extraneous dependencies in the project.
We used publish command in quick start, lets revisit and see what all things we can do with our packages on npm registry. Inside a project root npm publish
publishes the node module. We can also do the following:
From outside the project root, we can either specify the path to folder or the path to tarball or a URL to the tarball to publish. A new version should be mentioned in package.json
every time we publish a module. Trying to publish a module with the same version twice will result in failure. But adding a --force
flag will overwrite the version.
To unpublish a version use the following command:
The previous command removes the provided version of the package. To remove entire package and free the namespace on npm, drop the version part and provide --force
flag.
Tip
npm pack
from inside a package root creates a tarball of the package without node_modules
folder. Npm uses this command for creating a tarball to send to registry for publishing. It comes handy in inspecting the final contents that will go on npm publish. Check out other variants of pack command from man pages. npm help pack
And finally you can add/remove module owners to the packages that you own and that are published on registry.
To add an owner, follow the given command:
In the previous command <username>
is the username of the user registered on http://npmjs.org and <package-name>
is the name of the published package owned by you. This command adds an owner, which has full rights to publish, upgrade, unpublish, or completely remove the package from npm. The added owner can even remove other owners, including the one who added him in the first place, so be careful with this command and add owners only that you completely trust. This might change, and npm suggests providing more fine-grained access control in future.
Similarly, to remove an owner use the following command:
And to list owners of a package use the following command:
To run the previous command you don't need to be the owner of the mentioned package.
Lastly, the best way to help yourself is to use help provided through npm man pages. Here are few commands to get quick help:
Just running npm command provides a brief, non-exhaustive list of npm commands. Providing an -l
switch will provide a more exhaustive list.
The previous command opens man page for frequently asked npm questions.
The previous command provides quick help on the command.
The previous command opens man page for the term and/or associated command, and gives a detailed description. For example, npm help publish
or npm help star
.
The previous command opens a detailed document of npm, its purpose and the way it works.
You can also text search through npm man pages using the help-search
command.
The help-search
command will search for text and display the search results and their locations in man pages on command line.