Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Contents

Table of Contents

Placement in NetarchiveSuite software

Every channel is named after the set of applications application instances that are expected to receive messages sent to it. All channel names are constructed privately in the dk.netarkivet.common.distribute.ChannelID class. To get a channel, you must use one of the public methods in the  dk.netarkivet.common.distribute.Channels class.

Channel Behavior

There are used to two types of channels used in the JMS protocol:

  • Queue where only one listener takes a message from the queuechannel. For example a queue where a request for doing a harvest is only received by one harvester.
  • Topic where all listeners takes a copy of the message. For example a batch job which has to be executed by all bitarchive applications. The type of channel is not affecting the channel name directly. It is indicated by the Channel Prefix, since only channel names starting with ALL are topics, while the rest are queues.

The channel type for different queues is given in a table in the next sectionIn NetarchiveSuite we use a naming convention that topic channel names include the string "_ALL_" and the suffix "_TOPIC".

Naming Conventions

The structure of channel names are is as follows:

Code Block
<Environment Name>_<ReplicaID>_[<Channel Specifier Prefix>_]<Channel Specifier>[_<Machine>][_<Application Instance>][_topicTOPIC]
  • Environment Name value must be the same for all channels on all JVMs belonging to a single NetarchiveSuite installation. It identifies the environment for the queue, e.g.: "DEV", "TEST", "RELEASETEST", "CLOVER", "PROD" or initials of developer running personal environment for e.g. sanity tests. It is read from the setting settings.common.environmentName of the NetarchiveSuite installation.
  • ReplicaID Name of the replica (ie the "location" in a distributed system), or COMMON if channel shared by all replicas.

    Type is constructed by
  • a convention for channel behavior together with denotation of the concerned application(s). For example THE_SCHED uses the convention THE for SCHED denoting the scheduler of the harvesting system.
    The conventions for channel behavior areChannel Specifier Prefix if presentis one of the following:
    • THE - communicate with singleton in the system.
    • THIS - communicate with one uniquely identified instance of a number of distributed listeners.
    • ANY - has all instances listening to the same queue.
    • ALL - is used for topics only, i.e. topics are sent to all listeners on the channel.
    • Use Replica Id will result in a channel name with the replica identifier in question (which must match possible replicas in settings). In case of no use of replica id the COMMON will be used.
    • Use IP Node will result in a channel name with the IP address of the machine whether the application in question is placed. It is read directly from the running system.
      . In case of no use of IP Node, nothing will be written.
    • Use Application Instance Id is used, if only one process is expected to listen to the queue. It will result in a channel name with an identification of the individual application instance. It consists of the application abbreviation (uppercase letters of application name) and the application instance identifier from the settings. It is read from the common settings: settings.common.applicationName and settings.common.applicationInstanceId.
      . In case of no use of Application Instance Id, nothing will be written.
    • Channel Type is only here to complete the picture. Please refer to the section on channel behavior.
      Channel names are constructed as described in the below table (columns described after the table).

Channel Prefix

Example

Description

THE_SCHED

QUICKSTART_COMMON_THE_SCHED 

JOB_PARTIAL_FOCUSED

QUICKSTART_COMMON_JOB_PARTIAL_FOCUSED 

JOB_SNAPSHOT_SNAPSHOT

QUICKSTART_COMMON_JOB_SNAPSHOT_SNAPSHOT 

THIS_REPOS_CLIENT

QUICKSTART
  • Channel Specifier simply expresses what the channel is used for. By convention this (usually) describes what kind of application should listen on this channel.

Here is a complete list of the channels in an actual realistic distributed NAS system (IP addresses have been anonymised for security reasons).

Channel NameFunction

SystemTest_COMMON_HARVESTER_STATUS_TOPIC

Topic used by the harvesters to call in ready for new jobs (to the HarvestJobManager)

SystemTest_COMMON_HARVESTMONQueue to which harvesters send messages showing crawl progress for display in the GUI
SystemTest_COMMON_HCHAN_VAL_REQ Queue for harvesters to register the harvest channel they are processing jobs from.
SystemTest_COMMON_HCHAN_VAL_RESP Queue for response from the HarvestManager to the Harvesters requesting HarvestChannel registrations.
SystemTest_COMMON_INDEX_SERVER

Queue for sending messages to the IndexServer

SystemTest_COMMON_JOB_PARTIAL_HIGHPRIORITY Queue used to send jobs to the selective harvesters in the highpriority pool.
SystemTest_COMMON_JOB_SNAPSHOT_LOWPRIORITY Queue used for sending jobs to the harvester configured for snapshot harvesting.
SystemTest_COMMON_MONITOR_TOPIC Topic for the monitor registry
SystemTest_COMMON_THE_REPOS

Queue for sending messages to the ArcRepository.

SystemTest_COMMON_THE_SCHED Queue on which HarvestControllers reply with status messages to the HarvestScheduler.
SystemTest_COMMON_THIS_INDEX_CLIENT_xxx_225_xx_140_HCS_SBHIGH_H3

Queue on which a specific HarvestControllerServer listens for responses from the IndexServer

SystemTest_COMMON_THIS_INDEX_CLIENT_xxx_226_xxx_11_HCS_KBHIGH_H3  
SystemTest_COMMON_THIS_REPOS_CLIENT_
192.168.1.7
xxx_225_xx_xxx_HCS_
FOCUSED
SBHIGH_H3One-per-client queue on which client (in this case a HarvestControllerServer) receives replies from the repository.
THE
SystemTest_COMMON_THIS_REPOS

QUICKSTART_COMMON_THE_REPOS 

Queue for sending messages to the Repository are.

THE_BAMON

QUICKSTART_A_THE_BAMON 

Queue for sending messages to bitarchive monitors.

ALL_BA

QUICKSTART_A_ALL_BA_TOPIC 

Topic for broadcasting messages to all bit archives. 

ANY_BA

QUICKSTART_A_ANY_BA 

 

THIS_INDEX_CLIENT

QUICKSTART_COMMON_THIS_INDEX_CLIENT_192.168.1.7_HCS_FOCUSED

Queue for Harvesters status messages to the HarvestManager.

INDEX_SERVER

QUICKSTART_COMMON_INDEX_SERVER

 

MONITOR

QUICKSTART_COMMON_MONITOR_TOPIC

 

ERROR

QUICKSTART_COMMON_ERROR_COMMON

Queue for errors which are not handled elsewhere. Never normally created or used.

THE_CR

QUICKSTART_
_CLIENT_xxx_226_xxx_xx_WI One-per-client queue on which client (in this case a WaybackIndexer) receives replies from the repository.
SystemTest_COMMON_THIS_REPOS_CLIENT_xxx_226_xxx_11_HCS_KBHIGH_H3  
SystemTest_COMMON_THIS_REPOS_CLIENT_xxx_226_xxx_11_HCS_KBLOW002_H3  
SystemTest_COMMON_THIS_REPOS_CLIENT_xxx_226_xxx_5_IS One-per-client queue on which client (in this case an IndexServer) receives replies from the repository.
SystemTest_COMMON_THIS_REPOS_CLIENT_xxx_226_xxx_5_VP One-per-client queue on which client (in this case a ViewerProxy) receives replies from the repository.
SystemTest_COMMON_THIS_REPOS_CLIENT_xxx_226_xxx_9_HCS_KBLOW001_H3  
SystemTest_CS_THE_CR Queue for sending messages to a specific Checksum replica (the replica named CS).
HARVESTER
SystemTest_
STATUSQUICKSTART
KB_
COMMON
ALL_
HARVESTER
BA_
STATUS_
TOPIC
 HCHAN_VAL_REQQUICKSTART_COMMON_HCHAN_VAL_REQ HCHAN_VAL_RESPQUICKSTART_COMMON_HCHAN_VAL_RESP 

The examples are using values

  • Environment name QUICKSTART
  • Possible replica identifiers A, CS
  • IP on machine 192.168.1.7
  • HCS_FOCUSED (for HarvestControllerApplication with instance id FOCUSED )
    Topic for broadcasting messages to all bit archives at KB
    SystemTest_KB_ANY_BA Queue for sending a message to any one of the bitarchive applications at KB
    SystemTest_KB_THE_BAMON  
    SystemTest_SB_ALL_BA_TOPIC  
    SystemTest_SB_ANY_BA  
    SystemTest_SB_THE_BAMONQueue for sending messages to the bitarchive monitor at SB - ie to run batch jobs on the SB replica.