Page Comparison

...

Throughout the process, the state of a Batch can be tracked by the Surveillance Interface

HTML CommentWhat matters is not that there is a surveillance interface but that there is a monitor components that records a persistent state for each batch. - CSR

Each step in the process is handled by an autonomous component. Are these what are commonly called Autonomous Agents? - CSR

...

• digitize a batch of newspaper microfilm
• Upload the batch to our servers (by rsync)
• Notify us that the receival process can begin for this batch (The "Batch Object Creation" event)

There are some minor questions of detail here about who creates the intial object (e.g. a record in state "NEW" in a database) is it:

• Us, when we send the batch out to Ninestars
• Ninestars, after they have successfully uploaded the files to us
• Us, after we have received a message (in some form ...) from Ninestars that the files have been uploaded

Any of these could be made to work. - CSR

The first real robot is the "Autonomous Bitrepository Ingester". It polls for "Batch Object Creation" events, so it will receive batches right after Ninestars have uploaded them. For a batch, it will iterate over the jpeg2000 files and for each:

...

The next robot is the "Autonomous Doms Ingester". It polls for the "Bitrepository Ingest" event, so it will always run on batches after they have been ingested into the bit repository. It will create the metadata structure (batch->reel->newspaper->page) structure in DOMS with all the supplied metadata. When this task is done, we have no further need of the data in the Scratch storage, as it should all have been ingested into our preservation platform. Finally, it will add the "Metadata Ingest" event to the batch object. What is the story with content models? Does metadata-ingest include content-model validation? -CSR

Robots can occupy the same location on the assembly line. Here we have the first example of this. The "Autonomous JPylyzer" is a robot that, like the "Autonomous Doms Ingester" polls for "Bitrepository Ingest" events. The task of this robot is to run jpylyzer on the jpeg2000 files in the batch. The task will be done as a hadoop job. 

...