Introducing Apache ZooKeeper

Apache ZooKeeper is a software project of the Apache Software Foundation; it provides an open source solution to the various coordination problems in large distributed systems. ZooKeeper was originally developed at Yahoo!

Tip

A paper on ZooKeeper, ZooKeeper: Wait-free Coordination for Internet-scale Systems by Patrick Hunt and Mahadev Konar from Yahoo! Grid and Flavio P. Junqueira and Benjamin Reed from Yahoo! Research, was published in USENIX ATC 2010. You can access the full paper at http://bit.ly/XWSYiz.

ZooKeeper, as a centralized coordination service, is distributed and highly reliable, running on a cluster of servers called a ZooKeeper ensemble. Distributed consensus, group management, presence protocols, and leader election are implemented by the service so that the applications do not need to reinvent the wheel by implementing them on their own. On top of these, the primitives exposed by ZooKeeper can be used by applications to build much more powerful abstractions to solve a wide variety of problems. We will dive deeper into these concepts in Chapter 4, Performing Common Distributed System Tasks.

Apache ZooKeeper is implemented in Java. It ships with C, Java, Perl, and Python client bindings. Community-contributed client libraries are available for a plethora of languages such as Go, Scala, Erlang, and so on.

Tip

A full listing of the client bindings for ZooKeeper can be found at https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZKClientBindings.

Apache ZooKeeper is widely used by a large number of organizations, such as Yahoo! Inc., Twitter, Netflix, and Facebook, in their distributed application platforms as a coordination service. We will discuss more about how ZooKeeper is used in the real world in Chapter 7, ZooKeeper in Action.

Tip

A detailed listing of organizations and projects using ZooKeeper as a coordination service is available at https://cwiki.apache.org/confluence/display/ZOOKEEPER/PoweredBy.