Setup
Login to a bitarchive server, eg.
$ ssh kb-test-acs-001 $ export TESTX=TEST11A $ cd ${TESTX}Create dir for batchprograms:
$ mkdir batchprogs $ scp test@kb-prod-udv-001.kb.dk:/home/test/test-batch/* batchprogs/.
Run batch programs
ChecksumJob
Calculating the MD5 checksum on the archive files (only .arc files).
Run
java -cp lib/dk.netarkivet.archive.jar -Dsettings.common.applicationInstanceId=CHECKSUM -Ddk.netarkivet.settings.file=conf/conf/settings_IndexServerApplication.xml dk.netarkivet.archive.tools.RunBatch -Cbatchprogs/ChecksumJob.class -R.*.arc -Ooutput.checksum
This should produce a file output.checksum with be something like:
1-metadata-1.arc##9dca301f9ecd3663bbb3cdc91009562c 1-1-20130114144130-00001-kb-test-har-001.kb.dk.arc##1abe93f92569154da3f629255146dcfd 1-1-20130114144130-00002-kb-test-har-001.kb.dk.arc##6b2801ca68d7c96af8b05ba9e9ae47b4 1-1-20130114144130-00003-kb-test-har-001.kb.dk.arc##5c77d44ad9fc9290f2cd6639562ed11a 1-1-20130114144130-00000-kb-test-har-001.kb.dk.arc##c1e533f574a46576fc14393581428129
EvilBatch
Writes the sentence 'evil' for each bitarchive.
Run
java -cp lib/dk.netarkivet.archive.jar -Dsettings.common.applicationInstanceId=CHECKSUM -Ddk.netarkivet.settings.file=conf/settings_IndexServerApplication.xml dk.netarkivet.archive.tools.RunBatch -Cbatchprogs/EvilBatch.class -R.*.arc -Ooutput.evil
This should produce a file output.evil with:
Legal Legal Legal Legal
This is run on the KBN replica, which has four bitapps (according to the configuration file 'deploy_config_multi_bitapps.xml'). Therefore the sentence 'Legal' is written four times.
The SBN replica one has one bitapp, and it should therefore only have the sentence, 'Legal', written once.
Run
java -cp lib/dk.netarkivet.archive.jar -Dsettings.common.applicationInstanceId=EVILSB -Ddk.netarkivet.settings.file=conf/settings_IndexServerApplication.xml dk.netarkivet.archive.tools.RunBatch -Cbatchprogs/EvilBatch.class -R.*.arc -Ooutput.evil.sb -BSBN
This should produce a file output.evil with:
Legal
This is run on the SBN replica, which has only one bitapp (according to the configuration file 'deploy_config_multi_bitapps.xml'). Therefore the sentence 'Legal' is written once.
EvilBatch2
Tries to delete archived files, thus gets failed for all files.
Run
java -cp lib/dk.netarkivet.archive.jar -Dsettings.common.applicationInstanceId=EVIL2 -Ddk.netarkivet.settings.file=conf/settings_IndexServerApplication.xml dk.netarkivet.archive.tools.RunBatch -Cbatchprogs/EvilBatch2.class -R.*.arc -Eerror.evil2
Since the batch program is supposed to fail, nothing should be written to std-out, only to std-error. It is therefore the std-err which should be catched, which can be done with the -E argument.
This should write the following text in the console:
... INFO: JMSArcRepository listens for replies on channel '[Queue 'JOLF_COMMON_THIS_REPOS_CLIENT_130_226_228_5_ISA']' Running batch job 'batchprogs/EvilBatch2.class' on files matching 'arc.*' on replica 'KBN', output written to stdout errors written to file 'error.evil2' Mar 16, 2009 11:44:14 AM dk.netarkivet.archive.arcrepository.distribute.JMSArcRepositoryClient batch WARNING: The batch job 'ID:15-130.226.228.5(9e:6:7d:1b:b:ec)-39359-1237200254886: To JOLF_COMMON_THE_REPOS ReplyTo JOLF_COMMON_THIS_REPOS_CLIENT_130_226_228_5_ISA OK Job: dk.netarkivet.common.utils.batch.LoadableFileBatchJob processing EvilBatch2.class' resulted in the following error: Batch job failed on 3 files. Batch job failed on 2 files. Batch job failed on 3 files. Batch job failed on 3 files. Processed 11 files with 11 failures Writing errors to file: /home/test/JOLF/error.evil2 ...
This should produce a file error.evil2 with be something like:
Failed files: 1-1-20130114144130-00000-kb-test-har-001.kb.dk.arc 1-1-20130114144130-00003-kb-test-har-001.kb.dk.arc 1-1-20130114144130-00001-kb-test-har-001.kb.dk.arc 1-1-20130114144130-00002-kb-test-har-001.kb.dk.arc 1-metadata-1.arc