Harvest Channels
- Colin Samuel Rosenthal
Harvest Channels are used to send harvest requests from the harvest server to the deployed harvesters. A pool of harvesters is configured to listen for harvest jobs on a given harvest channel. Channels are either the snapshot harvest channel or a named selective harvest channel. There is always a single harvest channel for snapshot harvests (simply called SNAPSHOT), but there may be multiple selective harvest channels to manage specific crawls (e.g. NEWS, EMERGENCY,...).
The harvest channel administration GUI can be accessed by clicking the Harvest Channels left menu item:
Harvest Channel listing
The list of defined channels can be view by clicking the Channels link in the left menu.
Here is an example for a system with the snapshot channel and the default selective channel (FOCUSED).
New select harvest channels can be created by clicking the Create a new selective harvest channel link. This will expand into additional fields for creating the new Harvest Channel.
Harvest Channel Mappings
It is possible to map each harvest definition to a specific harvest channel. The means that all jobs based on this harvest definition will be handled by the pool of harvesters registered with the specific channel by the administrator.
The mapping for a harvest definition can be changed by clicking the Edit link and selecting the relevant channel in the dropdown which appears in the Harvest channel cell. (Newly-activated selective harvest jobs are always mapped to the first-defined selective channel.)