TEST11B

Owned by M
Last updated: Apr 18, 2018 by Sr
Stress test and crashtests of batch jobs and third-party batch jobs on 22 TB.

Start The Test

export TESTX=TEST11B
export PORT=807?
export MAILRECEIVERS=foo@bar.dk
export VERSION=5.4-RC1
export H3ZIP=/home/devel/nas_versions/bundler/NetarchiveSuite-heritrix3-bundler-$VERSION.zip
export CONF=deploy_config_TEST11B.xml
prepare_test.sh -d $CONF -v $VERSION -3 $H3ZIP
#### modify the prepared scripts in folder prepared_software/TEST11B to use <drive>:\bitarchive_X\ instead of <drive>:\bitarchive_X\TEST11B
### The following files need to be modified: dir_kb-test-bar-01.bitarkiv.kb.dk.bat, kb-test-bar-01.bitarkiv.kb.dk/security.policy, kb-test-bar-01.bitarkiv.kb.dk/settings_BitarchiveApplication_BitApp_[E|F|G|H|I|J].xml
cd prepared_software/TEST11B
sed -i -- 's/\\TEST11B\\filedir/\\filedir/g' dir_kb-test-bar-01.bitarkiv.kb.dk.bat
sed -i -- 's/\\TEST11B\\tempdir/\\tempdir/g' dir_kb-test-bar-01.bitarkiv.kb.dk.bat
sed -i -- 's/\\TEST11B\\atticdir/\\atticdir/g' dir_kb-test-bar-01.bitarkiv.kb.dk.bat
sed -i -- 's/bitarchive_1\\TEST11B/bitarchive_1/g' dir_kb-test-bar-01.bitarkiv.kb.dk.bat
sed -i -- 's/bitarchive_2\\TEST11B/bitarchive_2/g' dir_kb-test-bar-01.bitarkiv.kb.dk.bat
sed -i -- 's/bitarchive_3\\TEST11B/bitarchive_3/g' dir_kb-test-bar-01.bitarkiv.kb.dk.bat
sed -i -- 's/bitarchive_4\\TEST11B/bitarchive_4/g' dir_kb-test-bar-01.bitarkiv.kb.dk.bat
sed -i -- 's/bitarchive_5\\TEST11B/bitarchive_5/g' dir_kb-test-bar-01.bitarkiv.kb.dk.bat
sed -i -- 's/bitarchive_6\\TEST11B/bitarchive_6/g' dir_kb-test-bar-01.bitarkiv.kb.dk.bat

sed -i -- 's/\\TEST11B/\\/g' kb-test-bar-01.bitarkiv.kb.dk/settings_BitarchiveApplication_BitApp_?.xml
sed -i -- 's/TEST11B${\/}filedir${\/}/filedir${\/}/g' kb-test-bar-01.bitarkiv.kb.dk/security.policy
cd 
install_test.sh -d $CONF -v $VERSION -3 $H3ZIP
start_test.sh -d $CONF -v $VERSION -3 $H3ZP

Update The Filelist

In the GUI, update the Filestatus for KB. This will take several minutes to run. Most of the time is spent on updating the empty admin database, so you will get a lot of logs in GUIApplication saying stuff like

2018-04-17 17:16:07.803 [Thread-41] INFO  dk.netarkivet.archive.arcrepositoryadmin.ReplicaCacheHelpe
rs.addFileInformation 
- The file '44854-91-20090509044357-00076-sb-prod-har-005.arc' was not 
found in the database. Thus creating entry for the file.


The last logentry should end with "INFO: Completed findMissingFiles for replica 'BITARCHIVEReplica (KB) KBN'."

In the Bitpreservation GUI you should see the following:

Filestatus for: KBN
Number of files: 180,569
Missing files: 0

Update Checksum and FileStatus

In the Bitpreservation GUI, click on "Update checksum and filestatus for CS2". After a few minutes, the BitarchiveServer should start producing log messages like

15-08-2013 15:26:47 dk.netarkivet.common.utils.batch.BatchLocalFiles run
INFO: The batchjob 'class dk.netarkivet.common.utils.batch.ChecksumJob' 
has run for 135 seconds and has reached file '1-MB100.arc', which is 
number 10 out of 25890

Wait until the job has finished (about seven hours) with a message like

INFO: Finished batch job 
dk.netarkivet.archive.arcrepository.bitpreservation.ChecksumJob with 
result: 0 failures in processing 27620 files at 172.17.0.176_BitApp_E

on the BitApp and

Aug 15, 2013 10:11:34 PM dk.netarkivet.archive.arcrepositoryadmin.ReplicaCacheDatabase updateChecksumStatus
INFO: UpdateChecksumStatus operation completed!
Aug 15, 2013 10:11:34 PM dk.netarkivet.archive.arcrepository.bitpreservation.DatabaseBasedActiveBitPreservation findChangedFiles
INFO: Completed findChangedFiles for replica 'CHECKSUMReplica (CS2) CS2N'.

in the GUIApplication log.

Start Some Batch Jobs

On devel@kb-test-adm-001:

export TESTX=TEST11B
cd /home/devel/$TESTX/
mkdir batchprogs
### IN DEVELOPMENT ¤##################################
## The path below does not exist
##scp test@kb-prod-udv-001.kb.dk:/home/test/test
cp -pv test-batch/* batchprogs/.

Then start the following four jobs:

nohup java -cp lib/netarchivesuite-archive-core.jar -Ddk.netarkivet.settings.file=conf/settings_ArcRepositoryApplication.xml -Dsettings.common.applicationInstanceId=BATCH dk.netarkivet.archive.tools.RunBatch -Cbatchprogs/ChecksumJob.class -Ooutput.checksum &

 nohup java -cp lib/netarchivesuite-archive-core.jar -Ddk.netarkivet.settings.file=conf/settings_ArcRepositoryApplication.xml -Dsettings.common.applicationInstanceId=BATCH2 dk.netarkivet.archive.tools.RunBatch -Cbatchprogs/GoodPostProcessingJob.class -Ogood.out &

nohup java -cp lib/netarchivesuite-archive-core.jar -Ddk.netarkivet.settings.file=conf/settings_ArcRepositoryApplication.xml -Dsettings.common.applicationInstanceId=BATCH3 dk.netarkivet.archive.tools.RunBatch -Cbatchprogs/EvilPostProcessingJob.class -Oevil.out &

nohup java -cp lib/netarchivesuite-archive-core.jar -Ddk.netarkivet.settings.file=conf/settings_ArcRepositoryApplication.xml -Dsettings.common.applicationInstanceId=BATCH4 dk.netarkivet.archive.tools.RunBatch -Jbatchprogs/mime.jar -Nbatchprogs.MimeSize -Omimesize.out &

The "good" and "evil" jobs complete quickly. You can follow the progress of all the jobs simply with

 ps -fe | grep BATCH

Check The Output of "Good"

The output is put into the file 'good.out'. This file should contain the following text (sorted): 

0G5.arc
0G5.arc
...

Go to the status page and check the log for the BitarchiveMonitor . There should be the following messages: 

Jun 11, 2010 9:24:31 AM dk.netarkivet.archive.bitarchive.distribute.BitarchiveMonitorServer doBatchReply
INFO: BatchReplyMessage: 'BatchReplyMessage for batch job ID:10-130.226.228.6(d3:b5:49:8b:d6:94)-37536-1276241061823
FilesProcessed = 156775
FilesFailed = 0
ID:1906780-130.226.228.6(d4:77:b5:34:b8:ae)-52334-1276241071022: To TEST11B_COMMON_THIS_REPOS_CLIENT_130_226_228_6_GUIA_BATCH ReplyTo TEST11B_KB_THE_BAMON OK' sent from BA monitor to queue: '[Queue 'TEST11B_COMMON_THIS_REPOS_CLIENT_130_226_228_6_GUIA_BATCH']'

Jun 11, 2010 9:24:27 AM dk.netarkivet.common.utils.batch.GoodPostProcessingJob postProcess
INFO: Sorting the filenames

Jun 11, 2010 9:24:27 AM dk.netarkivet.common.utils.batch.GoodPostProcessingJob postProcess
INFO: Reading all the filenames.

Jun 11, 2010 9:24:27 AM dk.netarkivet.archive.bitarchive.distribute.BitarchiveMonitorServer doBatchReply
INFO: Post processing batchjob results for 'dk.netarkivet.common.utils.batch.LoadableFileBatchJob' with id 'ID:10-130.226.228.6(d3:b5:49:8b:d6:94)-37536-1276241061823'

Check The Output of "Evil"

The output is put into the file 'evil.out'. This file should contain the following text (unsorted): 

0G5.arc
1G5.arc
...

 Check The Output Of The Checksum Job

 This may take about eight hours to run. The output is put into the file 'output.checksum'. This file should contain the following text:

1-1-20090316092641-00003-kb-test-har-002.kb.dk.arc##c68b3e18f7b870b76d86de7970a822c2
2-2-20090316092643-00003-kb-test-har-001.kb.dk.arc##7d723dd4d374437c5e29e995521bf014
.......

Check The Output Of MimeSize

This may take up to 25 hours to run. The output is put into the file 'mimesize.out'. This file should contain the following text: 

..
text/html##567890
image/jpeg##1234567
...

 

Further tests