Learning Cypher: Write powerful and efficient queries for Neo4j with Cypher, its official query language

Neo4j Server is the best choice for interoperability, safety, and monitoring. In fact, the REST interface allows all modern platforms and programming languages to interoperate with it. Also, being a standalone application, it is safer than the embedded configuration (a potential crash in the client wouldn't affect the server), and it is easier to monitor. If we choose to use this mode, our application will act as a client of the Neo4j server.

To start Neo4j Server on Windows, download the package from the official website (http://www.neo4j.org/download/windows), install it, and launch it from the command line using the following command:

C:\Neo4jHome\bin\Neo4j.bat

You can also use the frontend, which is bundled with the Neo4j package, as shown in the following screenshot:

To start the server on Linux, you can either install the package using the Debian package management system, or you can download the appropriate package from the official website (http://www.neo4j.org/download) and unpack it with the following command:

# tar -cf  <package>

After this, you can go to the new directory and run the following command:

# ./bin/neo4j console

Anyway, when we deploy the application, we will install the server as a Windows service or as a daemon on Linux. This can be done easily using the Neo4j installer tool.

On the Windows command launch interface, use the following command:

# bin\Neo4jInstaller.bat install

When installing it from the Linux console, use the following command:

# neo4j-installer install

To connect to Neo4j Server, you have to use the REST API so that you can use any REST library of any programming language to access the database. Though any programming language that can send HTTP requests can be used, you can also use online libraries written in many languages and platforms that wrap REST calls, for example, Python, .NET, PHP, Ruby, Node.js, and others.

An embedded Neo4j database is the best choice for performance. It runs in the same process of the client application that hosts it and stores data in the given path. Thus, an embedded database must be created programmatically. We choose an embedded database for the following reasons:

For testing purposes, all Java code examples provided with this book are made using an embedded database.

<build>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-compiler-plugin</artifactId>
      <version>3.1</version>
      <configuration>
        <source>1.7</source>
        <target>1.7</target>
      </configuration>
    </plugin>
  </plugins>
</build>

GraphDatabaseFactory graphDbFactory = new GraphDatabaseFactory();

GraphDatabaseService graphDb = graphDbFactory
      .newEmbeddedDatabase("data/dbName");

import org.neo4j.graphdb.factory.GraphDatabaseSettings;
// ...
GraphDatabaseService db = graphDbFactory
                .newEmbeddedDatabaseBuilder(DB_PATH)
                .setConfig(GraphDatabaseSettings
                   .nodestore_mapped_memory_size, "20M")
                .newGraphDatabase();

neostore.nodestore.db.mapped_memory=20M

GraphDatabaseService db = graphDbFactory
                    .newEmbeddedDatabaseBuilder(DB_PATH)
                    .loadPropertiesFromFile("neo4j.properties")
                    .newGraphDatabase();

To execute Cypher queries on a Neo4j database, you need an instance of ExecutionEngine; this class is responsible for parsing and running Cypher queries, returning results in a ExecutionResult instance:

import org.neo4j.cypher.javacompat.ExecutionEngine;
import org.neo4j.cypher.javacompat.ExecutionResult;
// ...
ExecutionEngine engine = 
  new ExecutionEngine(graphDb);
ExecutionResult result = 
  engine.execute("MATCH (e:Employee) RETURN e");

Note that we use the org.neo4j.cypher.javacompat package and not the org.neo4j.cypher package even though they are almost the same. The reason is that Cypher is written in Scala, and Cypher authors provide us with the former package for better Java compatibility.

Now with the results, we can do one of the following options:

Dumping to a string is useful for testing purposes:

String dumped = result.dumpToString();

If we print the dumped string to the standard output stream, we will get the following result:

Here, we have a single column (e) that contains the nodes. Each node is dumped with all its properties. The numbers between the square brackets are the node IDs, which are the long and unique values assigned by Neo4j on the creation of the node.

When the result is a single column, or we need only one column of our result, we can get an iterator over one column with the following code:

import org.neo4j.graphdb.ResourceIterator;
// ...
ResourceIterator<Node> nodes = result.columnAs("e");

Then, we can iterate that column in the usual way, as shown in the following code:

while(nodes.hasNext()) {
   Node node = nodes.next();
   // do something with node
}

However, Neo4j provides a syntax-sugar utility to shorten the code that is to be iterated:

import org.neo4j.helpers.collection.IteratorUtil;
// ...
for (Node node : IteratorUtil.asIterable(nodes)) {
   // do something with node
}

If we need to iterate over a multiple-column result, we will write this code in the following way:

ResourceIterator<Map<String, Object>> rows = result.iterator();
for(Map<String,Object> row : IteratorUtil.asIterable(rows)) {
   Node n = (Node) row.get("e");
   try(Transaction t = n.getGraphDatabase().beginTx()) {
       // do something with node
   }
}

The iterator function returns an iterator of maps, where keys are the names of the columns. Note that when we have to work with nodes, even if they are returned by a Cypher query, we have to work in transaction. In fact, Neo4j requires that every time we work with the database, either reading or writing to the database, we must be in a transaction. The only exception is when we launch a Cypher query. If we launch the query within an existing transaction, Cypher will work as any other operation. No change will be persisted on the database until we commit the transaction, but if we run the query outside any transaction, Cypher will open a transaction for us and will commit changes at the end of the query.

If you have ever used the Neo4j Java API, you might wonder why we should write the following code:

ExecutionEngine engine = 
  new ExecutionEngine(graphDb, StringLogger.SYSTEM);
ExecutionResult result = 
  engine.execute("MATCH (e:Employee) RETURN e");
ResourceIterator<Node> nodes = result.columnAs("e");

You can get the same result with the Java API with a single line of code:

import org.neo4j.tooling.GlobalGraphOperations;
// ...
ResourceIterable<Node> empl = GlobalGraphOperations.at(graphDb)
                    .getAllNodesWithLabel(HrLabels.Employee);

However, pattern matching is much more powerful. By making slight changes to the query, we can get very important and different results; for example, we can find nodes that have relationships with other nodes. The query is as follows:

MATCH (n:Employee) --> (cc:CostCenter)
RETURN cc,n

The preceding query returns all employees that have a relation with any cost center:

Again, as you can see, both n and cc are within round brackets. Here, the RETURN clause specifies both n and cc, which are the two columns returned. The result would be the same if we specified an asterisk instead of n and cc in the RETURN clause:

MATCH (n:Employee) --> (cc:CostCenter)
RETURN *

In fact, similar to SQL, the asterisk implies all the variables referenced in the patterns, but unlike SQL, not all properties of the entities are involved, just those of the referenced ones. In the previous query, relationships were not returned because we didn't put a variable in square brackets.

By making another slight change to the query, we can get all the employees that have a relation with a specific cost center, for example CC1. We have to filter the code property as shown in the following code:

MATCH (n:Employee) --> (:CostCenter { code: 'CC1' })
RETURN n

If we compare this query with the previous one, we can note three differences, which are listed as follows:

The --> symbol specifies the direction of the relation; in this case, outgoing from n. In the case of MATCH expressions, we can also use the <-- symbol for inverse direction. The following expression is exactly equivalent to the previous expression:

MATCH (:CostCenter { code: 'CC1' } ) <-- (n:Employee)
RETURN n

The preceding expression will give the same result:

+-----------------------------------------+
| n                                       |
+-----------------------------------------+
| Node[2]{name:"Nathan",surname:"Davies"} |
| Node[3]{name:"Rose",surname:"Taylor"}   |
+-----------------------------------------+

If we don't have a preferred direction, we will use the -- symbol:

MATCH (n:Employee) -- (:CostCenter { code: 'CC1' } )
RETURN n

In our example, the latter query will return the same result as the previous one because in our model, relationships go from employees to cost centers.

If we wish to know the existing relationships between the employees and cost centers, we will have to introduce another variable:

MATCH (n:Employee) -[r]- (:CostCenter { code: 'CC1' })
RETURN n.surname,n.name,r

The variable r matches any relationship that exists between the employees and cost center CC1 and is returned in a new column:

+-----------------------------------------------------------+
| n.surname | n.name   | r                                  |
+-----------------------------------------------------------+
| "Davies"  | "Nathan" | :BELONGS_TO[0]{from:1297292400000} |
| "Taylor"  | "Rose"   | :MANAGER_OF[2]{from:1268002800000} |
+-----------------------------------------------------------+

So, here we have the last rule: relationship expressions must be specified in square brackets.

To filter the employees who belong to a specific cost centre, we have to specify the relationship type:

MATCH (n) -[:BELONGS_TO]-> (:CostCenter { code: 'CC1' } )
RETURN n

This query matches any node n, which has a relation of the BELONGS_TO type with any node cc that has the value CC1 as a property code:

+-----------------------------------------+
| n                                       |
+-----------------------------------------+
| Node[2]{name:"Nathan",surname:"Davies"} |
+-----------------------------------------+

We can specify multiple relationships using the | operator. The following query will search for all employees who belong to or are managers of the cost center CC1:

MATCH (n) -[r:BELONGS_TO|MANAGER_OF]-> (:CostCenter{code: 'CC1'})
RETURN n.name,n.surname,r

This time we returned only the name and surname, while the relationship is returned in the second column:

+-----------------------------------------------------------+
| n.name   | n.surname | r                                  |
+-----------------------------------------------------------+
| "Nathan" | "Davies"  | :BELONGS_TO[0]{from:1297292400000} |
| "Rose"   | "Taylor"  | :MANAGER_OF[2]{from:1268002800000} |
+-----------------------------------------------------------+

By making a slight change to the query in the preceding code, we can return the manager as well as the employees of the cost center as the result. This can be implemented as shown in the following query:

MATCH (n) -[:BELONGS_TO]-> (cc:CostCenter) <-[:MANAGER_OF]- (m)
RETURN n.surname,m.surname,cc.code

In this query, we can see the expressivity of Cypher—a very intuitive syntax to translate the "node n belonging to the cost center having a manager m" pattern. The result is the following code:

+---------------------------------+
| n.surname | m.surname | cc.code |
+---------------------------------+
| "Davies"  | "Taylor"  | "CC1"   |
+---------------------------------+

Of course, we can chain an increasing number of relationship expressions to describe very complex patterns:

MATCH (n) -[:BELONGS_TO]->
      (cc:CostCenter) <-[:MANAGER_OF]- (m) <-[:REPORTS_TO]- (k)
RETURN n.surname,m.surname,cc.code, k.surname

Another query that is very useful in real-world applications is finding nodes reachable from one node with a certain number of steps and a certain depth. The ability to execute this kind of query, and search the neighborhood, is one of the strong points of graph databases:

MATCH (:Employee {surname: 'Smith'}) -[*2]- (neighborhood)
RETURN neighborhood

This query returns the nodes that you can reach, starting from the Davies node, by visiting exactly two relationships of the graph. The result contains duplicated nodes because we have several paths to reach each of them:

+---------------------------------------------------------------+
| neighborhood                                                  |
+---------------------------------------------------------------+
| Node[4]{name:"Heather",surname:"Underwood",middleName:"Mary"} |
| Node[6]{}                                                     |
| Node[1]{code:"CC2"}                                           |
| Node[6]{}                                                     |
+---------------------------------------------------------------+
4 rows

MATCH (:Employee {surname: 'Smith'}) -[ *2]- (neighborhood)
RETURN DISTINCT neighborhood

This time, we haven't specified any relationship type in the square brackets, so it matches any type. The expression *2 means exactly two steps. With a little change, we can also ask for the relationships we visited:

MATCH (:Employee {surname: 'Davies'}) -[r*2]- (neighborhood)
RETURN neighborhood, r

Of course, by changing the number in the expression, we can get the query to navigate any number of relationships. However, we could also want all the nodes that are reachable from a number of relationships in a range of step numbers, for example, from two to three:

MATCH (:Employee {surname: 'Smith'}) -[r*2..3]- (neighborhood)
RETURN neighborhood,r

This is very useful in real-world applications such as social networks because it can be used to build lists, for example, a list of people you may know.

If we also want the starting node in the result, we can modify the range to start from 0:

MATCH (:Employee{surname: 'Smith'}) -[r*0..2]- (neighborhood)
RETURN neighborhood,r

In our applications, we often need to get some information related to something that could be missing. For example, if we want to get a list of all employees who have a specific number of employees reporting to them, then we must deal with those employees too who have no employees reporting to them. In fact, we can write:

MATCH (e:Employee) <-[:REPORTS_TO]- (m:Employee)
RETURN e.surname,m.surname

From this, the following result is obtained:

+-------------------------+
| e.surname   | m.surname |
+-------------------------+
| "Taylor"    | "Davies"  |
| "Underwood" | "Smith"   |
+-------------------------+

However, this is not what we are looking for. In fact, we want all the employees, with all the employees that report to them as an option. This type of relation is similar to the OUTER JOIN clause of SQL and can be done in Cypher using OPTIONAL MATCH. This keyword allows us to use any pattern expression that can be used in the MATCH clause, but it describes only a pattern that could match. If the pattern does not match, the OPTIONAL MATCH clause sets any variable to null variable:

MATCH (e:Employee)
OPTIONAL MATCH (e) <-[:REPORTS_TO]- (m:Employee)
RETURN e.surname,m.surname, c.code

In this query, we slightly changed the previous one; we just inserted OPTIONAL MATCH (e). The effect is that the first part (e:Employee) must match, but the pattern following OPTIONAL MATCH may or may not match. So, this query returns any employee e, and if e has a relationship of the REPORTS_TO type with any other employee, this query is returned in m; otherwise, m will be a null value. The result is as follows:

 +-------------------------+
| e.surname   | m.surname |
+-------------------------+
| "Davies"    | <null>    |
| "Taylor"    | "Davies"  |
| "Underwood" | "Smith"   |
| "Smith"     | <null>    |
+-------------------------+

Now, let's say that we also want to know whether the employee is the manager of any center cost, and if so, which one. Also, we want to know the cost center of any employee. For this, we can write the following code:

MATCH (e:Employee)
OPTIONAL MATCH (c:CostCenter) <–[:MANAGER_OF]- (e) <-[:REPORTS_TO]- (m:Employee)
RETURN e.surname,m.surname

The preceding code returns the following result:

+----------------------------------+
| e.surname   | m.surname | c.code |
+----------------------------------+
| "Davies"    | <null>    | <null> |
| "Taylor"    | "Davies"  | "CC1"  |
| "Underwood" | <null>    | <null> |
| "Smith"     | <null>    | <null> |
+----------------------------------+

What happened? Does it look like Smith does not report to Underwood anymore? This weird result is due to the fact that the whole pattern in OPTIONAL MATCH must match. We can't have partially matched patterns. Since we can add as many OPTIONAL MATCH expressions as we want to, we have to write the following code to get the result we are looking for:

MATCH (e:Employee)
OPTIONAL MATCH (e) <-[:REPORTS_TO]- (m:Employee)
OPTIONAL MATCH (e) -[:MANAGER_OF]-> (c:CostCenter)
RETURN e.surname, m.surname, c.code

In fact, the result is the following code:

+----------------------------------+
| e.surname   | m.surname | c.code |
+----------------------------------+
| "Davies"    | <null>    | <null> |
| "Taylor"    | "Davies"  | "CC1"  |
| "Underwood" | "Smith"   | <null> |
| "Smith"     | <null>    | <null> |
+----------------------------------+

This query works because we have two OPTIONAL MATCH clauses that can independently generate a successful match.

As we have seen earlier, graph databases are useful to find paths between two nodes:

MATCH path = (a{surname:'Davies'}) -[*]- (b{surname:'Taylor'})
RETURN path

This query uses a construct which we have not used so far—the path assignment, path =. The assignment of variables can be done only with paths. Note that the query in the preceding code returns all the possible paths from two nodes. Here, the result is two paths in our database:

[Node[2]{name:"Nathan",surname:"Davies"},:BELONGS_TO[0]{from:1297292400000},Node[0]{code:"CC1"},:MANAGER_OF[2]{from:1268002800000},Node[3]{name:"Rose",surname:"Taylor"}] |
 [Node[2]{name:"Nathan",surname:"Davies"},:REPORTS_TO[1]{},Node[3]{name:"Rose",surname:"Taylor"}]

However, what if we need the shortest path between them? The shortest path is the path with the least number of nodes visited. Clearly, we could iterate over all the paths and take the shortest, but Cypher provides a function that does the work for us:

MATCH (a{surname:'Davies'}), (b{surname:'Taylor'})
RETURN allShortestPaths((a)-[*]-(b)) as path

Let's see what is new in this query:

When we execute a query like the previous code, Cypher must find the nodes and relationships that match the pattern. However, to do so, it must start to search from a set of nodes or relationships. We can let Cypher find the starting points of a query on its own, but we can also specify them because we want to search a pattern that starts from a specific node, or a specific relation resulting in an important improvement in the performances of the query.

We can assign starting points to variables in the query using the START keyword. The previous query, for example, could be rewritten in the following way:

START a=node(2), b=node(3)
RETURN allShortestPaths((a)-[*]-(b)) AS path

If we execute this query, and compare the time elapsed in executing this query and the previous one, we can easily prove that the latter is dramatically faster. The drawback is that we need to know the ID of the node.

The Cypher Java API wants us to pass all the parameters in the map. The following code is a class example that has a public method to find all employees by their surname:

import java.util.Map;
import java.util.HashMap;
import org.neo4j.cypher.javacompat.ExecutionEngine;
import org.neo4j.cypher.javacompat.ExecutionResult;
import org.neo4j.graphdb.Node;
import org.neo4j.helpers.collection.IteratorUtil;

public class EmployeeRepository {

   public Iterator<Node> bySurname(String surname) {
      Map<String, Object> params = new HashMap<>();
      params.put("inputSurname", surname);
      ExecutionResult result = engine
         .execute("MATCH (n:Employee {surname: {inputSurname}})" +       
                  "RETURN n",
                  params);
        Iterator<Node> nodes = result.columnAs("n");
        return nodes;
    }
}

The bySurname method takes the surname of the employees as a parameter to search, and it creates a new HashMap and puts the parameter in the map. Finally, the map is passed to the execute method of ExecutionEngine, and the result is treated in the usual way.