Skip to main content

ZooKeeper

Apache® Druid uses Apache® ZooKeeper (ZK) for management of current cluster state.

Minimum ZooKeeper versions​

Apache Druid supports all stable versions of ZooKeeper. For information about ZooKeeper's stable version, see ZooKeeper releases.

ZooKeeper operations​

The operations that happen over ZK are:

  1. Coordinator leader election
  2. Overlord leader election
  3. Service (node) announcement and discovery — services announce their presence so other services can find them

Segment loading, segment discovery, and Overlord ↔ Middle Manager task management no longer use ZooKeeper — they are served over HTTP.

Coordinator leader election​

Druid uses the Curator LeaderLatch recipe to perform leader election at path

${druid.zk.paths.coordinatorPath}/_COORDINATOR

Overlord leader election​

Druid uses the same LeaderLatch recipe for Overlord leader election at path

${druid.zk.paths.overlordPath}/_OVERLORD

Service announcement and discovery​

Each Druid service announces a DruidNode record (host, port, role, services) under the internal-discovery path so that other services can enumerate cluster members by role:

${druid.zk.paths.base}/internal-discovery/${nodeRole}/${druid.host}

Brokers and Coordinators use this path to find Historicals, Peons, and Indexers. They then poll each discovered service's HTTP /druid-internal/v1/segments endpoint to get its current set of served segments.