5.4 Release Test

Release Test Information

Previous 5.3.1 Release Test
The Netarkivet devel system is now documented at https://kb-dk.atlassian.net/wiki/pages/viewpage.action?pageId=12225230 (DK developers only)
Release test status: Started 31 Jan 2018
NetarchiveSuite 5.4 Release Notes: NetarchiveSuite 5.4.x Release Notes
Prepare Release Test Process

Run the tests (with proper definitions of TESTX, PORT and MAILRECEIVERS) as follows:

export VERSION=5.4-RC1
export H3ZIP=/home/devel/nas_versions/bundler/NetarchiveSuite-heritrix3-bundler-$VERSION.zip
export TESTX=???
export PORT=807?
export MAILRECEIVERS=foo@bar.dk
export CONF=default_deploy_config.xml
cleanup_all_test.sh -d $CONF -v $VERSION -3 $H3ZIP
prepare_test.sh -d $CONF -v $VERSION -3 $H3ZIP
install_test.sh -d $CONF -v $VERSION -3 $H3ZIP
start_test.sh -d $CONF -v $VERSION -3 $H3ZIP

Version	Commit	Reason for Rejection
5.4-RC1	https://github.com/netarchivesuite/netarchivesuite/commit/54d12da51c8cf3d1f544d6936f1a004f252eac02

Precondition

Check that the Full Test Suite (Master) and System Test are both green.

Sanity test

Run right after code freeze to verify the system integrity prior to the main release test.

Test	Status	Tester	Revision	Comments
TEST1	DONE	Colin Samuel Rosenthal	5.4-RC1	NAS-2719 - Getting issue details... STATUS NAS-2720 - Getting issue details... STATUS The test description requires some cleanup of i) the expected results of the various harvests and ii) use of H3 Monitor where appropriate e.g. to terminate harvests.

Regression tests

Test	Summary	Status	Tester	Comments
TEST0	Run through the Quick Start Manual	OK?	SVC	NAS-2723 - Getting issue details... STATUS NAS-2724 - Getting issue details... STATUS NAS-2725 - Getting issue details... STATUS NAS-2726 - Getting issue details... STATUS The vagrant part of the Quickstart has not been tested
TEST0 - with vagrant	Run through the Quickstart with Vagrant
TEST0 - with postgresql	Run through the quickstart where we test the procedure for using NAS with postgresql (see page Choose an Installation Scenario )	OK	SVC	createscripts and documentation has been updated A quickstart deploy for postgresql has been added Quickstart with postgresql
TEST2	Allround test	Complete with errors.	CSR	NAS-2729 - Getting issue details... STATUS NAS-2730 - Getting issue details... STATUS
TEST 3A	Database Bitpreservation	OK	SVC	NAS-2728 - Getting issue details... STATUS Fixed old issue: NAS-2078 - Getting issue details... STATUS
TEST4	Danish ADM GUI Access, indexing and caching	OK	SVC	Made some changes to the test during the test. E.g. the multi-domain harvest only generates 4 jobs, not 5. Previously we used kb-test-bar-014.bitarkiv.kb.dk and kb-test-bar-015.bitarkiv.kb.dk for windows bitarchive servers. Now we use kb-test-bar-016.bitarkiv.kb.dk and kb-test-bar-01.bitarkiv.kb.dk instead
TEST5A	Infrastructure	OK	SVC
TEST5B	Infrastructure	OK	CSR	After which I will take a look at as many of the bug-fixes/new-functionality issues as possible.
TEST6	Errorhandling	OK with errors	JRG	Step "Verify the restarted system. On devel@kb-test-adm-001" part 1 gives a: "Attempting to store file '23-metadata-1.warc.gz' with a different checksum than before". Needs follow-up. Have made JIRA-issues of the remaining errors.
TEST7	Production load job schedule database, adminDB and CS		Automatic
TEST9	Opensource modules	OK with errors	SVC	One part of the test uses /home/devel/devel-config/deployment-configs/default_deploy_config_harvester.xml We need to 1) Remove Bitpreservation sitesection from the GUI used; 2) Avoid using obsolete h3-zip (NetarchiveSuite-heritrix3-bundler-5.0-SNAPSHOT.zip) when deploying 3) Update the 1-cache.tar.bz2. It is created with Lucene 2.X. NAS now only supports indices with Lucene 3.X (workaround: use empty cache created when IndexServer starts) 4) Change the Viewerproxy test to use a freshly made index of a netarkivet.dk harvest using a standard NAS installation. (we used kb-test-acs-001.kb.dk:12-data instead of the kb-prod-udv-001.kb.dk:test-data, and a fresh index, that indexes the harvestdata in the 12-data dir) Note: We had to go kb-test-acs-001 to perform the two wget operations, otherwise the firewall blocked us NAS-2734 - Getting issue details... STATUS NAS-2737 - Getting issue details... STATUS
TEST11A	Batchjobprocessing	OK with errors	SVC	NAS-2738 - Getting issue details... STATUS NAS-2739 - Getting issue details... STATUS The last step: Run UrlAndMimeSearch with argument 'image/.' for mimetype and '.kb\.dk/.*' for url. Verify that only images from the kb domain is listed. failed to find any images at all. It could be because warcs are not supported by UrlAndMimeSearch batchjob
TEST11B	Batchjobprocessing stress test with external program	Stalled because of issue 3	SVC	Updated deploy_config_TEST11B.xml to use data from kb-test-bar-01.bitarkiv.kb.dk instead of machine kb-test-bar-012.bitarkiv.kb.dk which no longer exists. baseFileDir's E:\bitarkiv, F:\bitarkiv, G:\bitarkiv, H:\bitarkiv, I:\bitarkiv, J:\bitarkiv are replaced with baseFileDir's E:\bitarchive_1, F:\bitarchive_2, G:\bitarchive_3, H:\bitarchive_4, I:\bitarchive_5, J:\bitarchive_6 Issue 1: (FIXED in test-description) Default installation does not work with TEST11B. We want the test to use existing data in bitarchive_X/filedir But prepare_test.sh modifies the scripts to use bitarchive_X/TEST11B/filedir where there is no data. Issue 2: Description of Step "Update Checksum and FileStatus" doesn't make much sense (this issue has been ignored for this test.) `Issue 3: path test@kb-prod-udv-001.kb.dk:/home/test/test-batch/*` does not exist Have found the compiled scripts in `test@kb-test-adm-001.kb.dk:/home/test/TEST11B` but we need to have the sourcecode for these
TEST12	Wayback test	OK	CSR	Rewriting to use the latest OpenWayback with https support
BNF sanity test	Same as French UI test?			NAS-2731 - Getting issue details... STATUS NAS-2732 - Getting issue details... STATUS NAS-2733 - Getting issue details... STATUS
ONB Sanity test	German UI test		Andreas P

Test new functionality

(H3 Scripting features do not need to be tested as part of the NAS release test.)

key	summary	assignee	components	status

Loading...

Refresh

Review of Documentation

Document	Reviewer	Status	Notes
NetarchiveSuite Overview	SVC	Done	No mention of H3 Remote Access module
Quick Start Manual	KAAH	Done	NAS-2727 - Getting issue details... STATUS Replaced the picture with a new snapshot
User Manual	KAAH	Done	NAS-2727 - Getting issue details... STATUS Heritrix Control and GUI-console Access (All Running Jobs page picture needs to be updated)
Installation Manual	CSR	Done	I don't really like the state of this manual. It's very confusing and contains a considerable amount of material which should really be elsewhere. I don't how we can find time to give it the rewrite it needs.
Configuration Manual	CSR	Done
System Design	SVC	Done	The manual now assumes that this link works: https://sbforge.org/maven/netarchivesuite/apidocs/5.4/dk/netarkivet/common/utils/Settings.html Pages on JSP is no longer correct, as we now have two competing standards for how to make server-pages All links to javadoc was previously wrong: https://sbforge.org/maven/netarchivesuite/apidocs/release/dk/netarkivet/common/utils/Notifications.html The apidocs/release folder doesn't exist All javadoc links now refer to https://sbforge.org/maven/netarchivesuite/apidocs/5.4 Harvester design page contained many factual errors, which are now fixed
Additional Tools Manual	KAAH	Done	FTPValidator and MailValidator are now added.
Migrating H1 templates to H3 to use with NetarchiveSuite 5.1+	SVC	Done