Page Comparison

We have prepared a bash shell script that starts all the necessary components on one machine. We will use this script throughout this quickstart manual to allow you to get a feel for what the system can do and how it works without having to deal with issues of distributing to other servers.

Base system required

For the quick startup, NetarchiveSuite requires:

• A Linux system with a minimum of 2GB free diskspace.  The QuickStart guide was tested with Ubuntu.
• Sun/Oracle Java 6 SE. Other Java versions such as OpenJDK and Oracle Java 7 are not tested or recommended.

• A additional user is named "test". The commands to install NetarchiveSuite are run from your own login. The commands install and run the NetarchiveSuite software under user "test". This simulates the more realistic productions situation where the software runs under various logins on one or more machines in a distributed network. For convenience, it is a good idea to configure the test-user to have password-free ssh access - i.e. you should be able to execute "ssh test@localhost" in a shell without entering the test-user's password.

Setup JMS

NetarchiveSuite uses Java Messaging Service (JMS) for communication between the different components.

Download the openmq installation script:

wget https://sbforge.org/svn/netarchivesuite/trunk/scripts/openmq/mq.sh

Install the openmq broker:

sh mq.sh install

This will download openmq, install and start it. 

Note: OpenMQ will as defaut be installed to ~/openmq4.5. A alternative installdir can be defined the installdir variable prior to using the mq script, eg. 

export installdir="netarchive/openmq" 
sh mq.sh start 

Download NetarchiveSuite

Binary releases are available from https://sbforge.org/downloads/netarchivesuite/releases/.

Create a working directory and navigate to it

mkdir netarchive; cd netarchive

EITHER download the latest release (as of 2014-04-22 this is 4.2)  4.4 release.

wget -N -O NetarchiveSuite.zip https://sbforge.org/downloads/netarchivesuite/releases/stable/4.24/NetarchiveSuite-4.2.04.zip

OR - if you need the latest and untested code - download the latest successful build:

wget -N nv -O NetarchiveSuite.zip 'https://sbforge.org/jenkinsnexus/jobservice/NetarchiveSuite-unittestlocal/lastSuccessfulBuildartifact/artifact/NetarchiveSuite.zip maven/redirect?r=snapshots&g=org.netarchivesuite&a=distribution&v=LATEST&p=zip' 

Installation

Download RunNetarchiveSuite.sh and deploy_standalone_example.xml to the netarchive directory:

wget -N https://sbforge.org/svn/netarchivesuite/trunk/examples/deploy_standalone_example.xml
wget -N https://sbforge.org/svn/netarchivesuite/trunk/scripts/RunNetarchiveSuite.sh
chmod +x RunNetarchiveSuite.sh

The first script is a simple script for doing all the steps during deployment. It takes a NetarchiveSuite package ('.zip'), a configuration file (the second file), and a temporary installation directory as arguments (in the given order). The different ports used by the application for communication are included in the deploy_standalone_example.xml file.

In the configuration file all the applications are placed on one machine, the current machine (localhost).

When the installation script is run it will unpack the installation files into the netarchive/deploy directory as the current user and then - as the user "test" - install NetarchiveSuite into the /home/test/QUICKSTART directory (using ssh). 

Important: As of 2014-04-22 an Oracle Java 6 JVM is required as the default Java for the user "test" !  OpenJDK OpenJDK or Oracle Java 7 will not work.

If you already have a Quickstart installation, the existing bitarchive, database and admin.data files will be untouched. You must explicitly remove any previous installation, if you want a clean empty installation.

Now do the deployment:

./RunNetarchiveSuite.sh NetarchiveSuite.zip deploy_standalone_example.xml deploy/

Note that if you have not setup your automatic ssh test user login (using key based login), you need to login some times before the installation finishes successfully. You must also have permission to ssh and scp to test@localhost (try e.g ssh test@localhost)

The script creates a deployment folder named "QUICKSTART" in e.g. /home/test/QUICKSTART, which contains methods for starting and stopping NetarchiveSuite, and starts the whole NetarchiveSuite. The files to run the installation will be placed in the ~/netarchive/deploy directory.

Now configure your browser:

• Start a web browser. 

• Anchor
  proxy proxy

  Setup the browser to proxy on port 8070 on the host executing the netarchive, and exclude localhost and the hostname (used by the Heritrix GUI).  In Firefox it is done as follows:

Choose in the Firefox toolbar:
Edit->Preferences->Advanced->Network->Settings
Checkmark:
Manual Proxy Configuration
and add:
Http proxy: name-of-host
Port: 8070

No Proxy for: localhost, name-of-host

• Write following url in the started browser http://localhost:8074/HarvestDefinition (if running on the local machine, otherwise go to port 8074 on the host running NetarchiveSuite).  
  
  You should see the Netarchive harvest definition page:
  
  If you see a stacktrace instead of the Netarchive pages, this is most likely because the browser tries to go through the ViewerProxy.    Revisit the browser configuration and try again.
• You can now see the webinterface in the browser. You can now create, run and browse according to the following or the User Manual
• You can stop and start the entire NAS system with:

ssh test@localhost
cd QUICKSTART
./conf/killall.sh
./conf/startall.sh

• If you want to try other deploy examples, then go to "Examples of deploy configuration files" in the Installation Manual.