Zookeeper

We use zookeeper to track locks between the autonomous components. To make zookeeper easier to use, we use the netflix curator api http://curator.incubator.apache.org/

Zookeeper maintains a tree structure internally. In each leaf in the tree should correspond to some real world thing. A leaf can be locked, and the system is based on the idea that you will lock the leaf before accessing the thing it represents.

We have two "kinds" of leafs. 

The specific lock paths are as follows

SBOI lock = "/SBOI/" + runnable.getComponentName();

example: /SBOI/SampleComponent

batch lock = "/"+runnable.getComponentName() + "/B" + batch.getBatchID() + "-RT" + batch.getRoundTripNumber();

example: /SampleComponent/B400022028241-RT1

Why?

An autonomous components will be able to "poll" the Newspaper Batch Event Framework for events that have happened. The component will know which events should have occurred for a batch before I can start processing the batch. Also which events should have occurred and failed, and which events should not have occurred. The component also needs to know which batches it is currently working on itself. This is what the zookeeper tracks the locks for.

Autonomous component locking procedure 

Each of the autonomous components will exist as cron jobs. Periodically, often every minute or thereabouts, the cron job will activate. It will then follow the procedure below. Each autonomous component will have a field called "simultaneousProcesses", which indicate how many children it can spawn.

1. Lock "SBOI+component name"
2. query SBOI for batches
3. For each returned batch
   1. Attempt to lock the "batch+component name"
   2. Create child until number of children == "simultaneousProcesses"
4. unlock "SBOI+component name"
5. Start each child
6. Wait for all children to complete their work
7. unlock "batch+component name"

Each child will do the work described and store the result back into DOMS.

The autonomous component will, as described above, NOT finish before all spawned children have finished. It is the responsibility of the cron system to not start the autonomous component before the previous invocation have finished