ZooKeeper allows distributed processes to coordinate with each other through a shared hierarchical name space of data registers (we call these registers znodes) modeled after the familiar directory tree structure of file systems. But unlike normal file systems ZooKeeper provides its clients with high throughput, low latency, highly available, strictly ordered access to the znodes.
Like the distributed processes it coordinates, ZooKeeper itself is intended to be replicated over a sets of hosts called an ensemble. These hosts maintain an in-memory image of the data tree along with transaction logs and snapshots in a persistent store. Because the data is kept in-memory ZooKeeper is able to get very high throughput and low latency numbers.
Order is very important to ZooKeeper. To achieve this each update gets stamped with a number that reflects the order of all ZooKeeper transactions. Subsequent operations can use the order to implement higher-level abstractions, such as synchronization primitives.
Given a cluster of Zookeeper servers, one node acts as the leader and handles all write requests. The rest of the ZooKeeper servers, calledfollowers, receive message proposals from the leader and agree upon message delivery. The messaging layer takes care of replacing leaders on failures and syncing followers with leaders.
The target market for ZooKeeper are multi-host, multi-process C and Java based systems that operate in a data center.
Use case for Zookeeper
- HBase uses Zookeeper to coordinate activities its “head node” was responsible for. The move to using Zookeeper means the central control is no longer a single point of failure.
- Zookeeper can be used to coordinate resource locking.
- It is used by Yahoo! as the coordination and failure recovery service for Yahoo! Message Broker and by the Fetching Service of the Yahoo! crawler.
Why not simply use a database?
- The ZooKeeper client libraries ensures that everything is dispatched in order.
- A client will see a watch event for a znode it is watching before seeing the new data that corresponds to that znode.
- The order of watch events from ZooKeeper corresponds to the order of the updates as seen by the ZooKeeper service.