Versions Compared

Old Version 1

changes.mady.by.user Colin Samuel Rosenthal

Saved on

compared with

New Version 2

changes.mady.by.user Colin Samuel Rosenthal

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

A Heritrix3 harvest is defined by a Crawler-Bean file. This is a bean-definition file from the Spring framework. You can use Heritrix3's own documentation to create Crawler-Bean files which can then be uploaded to NetarchiveSuite via the GUI. NetarchiveSuite overwrites certain placeholder values in every Crawler-Bean definition before scheduling the harvest. The following placeholders are defined - some are required in every Crawler-Bean file, others are optional. When an optional placeholder is missing from the Crawler-Bean definition, then any attempt to redefine its value via the GUI will be ignored. There is no validation of Crawler-Bean files in this version of NetarchiveSuite, so a missing required placeholder will first manifest itself as a harvest job which fails to start. Some form for validation will be introduced in a later version of NetarchiveSuite.