Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

...

All communication via the Coordination layer starts via a Topic that is send to all. It is up to the individual subscribers to determine whether it is a message that concerns them, for example a GetTime message only concern the pillars involved in a certain service level agreement.

<TBD> An example can be found in http://wiki.statsbiblioteket.dk/BitmagasinWiki/Sekvens1

DIAGRAM NOT UPDATED!

Atomic messaging in simple operations

Operation An operation is defined as the communication that takes place in order to make an operation on the bit repository. Examples are get and put of data.

Operations must be as simple as possible, based on . They will be based on simple message primitives and data transmission. A primitive is a single request message to the system and the following replies. The primitives are defined on basis of atomic weldefined messages. This serves to give simplicity, and to get as little overlap between different operations as possible.

Operations are seen as a sequences of messages which collectively correspond to a full operation

Image Removed

a

Using Get

s example

The Get operation can begin by asking pillars who will be the fastest, or it can start by actually sending a get message for a specific default pillar.

In the strict interpretation of single operations this will be seen as two operations

  1. GetTime
  2. Get

The difference between the two get scenarios will be that the first will both consist of a GetTime and a Get operation, while the later will only consist of a Get.

The below figure illustrates the messages of the two operations, where messages belonging to the same communication in an operation is placed edge to edge. note that it only includes messages on the message bus, i.e. excluding the actual data transmission.

Since each communication is started via a (SLA) topic this means that the GetTime is sent to a topic. The Get operation is addressed to a specific pillar queue, but it usually preceded by a GetTime, such that the comunication is started via a topic. This is illustrated in the below sequence diagram

<TBD> See http://wiki.statsbiblioteket.dk/BitmagasinWiki/Sekvens1

DIAGRAM NOT UPDATED!

The advantages are:

  • the communications work similar for different operations and scenarios
  • it will be easy to handle shift in services, e.g. exchanging one primary pillar with another, exchanging pillars in a SLA etc.

The disadvantages can be:

  • there may be a heavy load of messages on the SLA topic that all subscribers must go through in order to find the ones interesting for themselves.

Reuse and overlap between operations

...

An example of the Get operation is given in the below figure.

Image Added

Here the full communication will result in an operation. The message communication is divided into two parts, each representing a primitive (identification of pillars and the actual get).

Note that the primitive for identification of pillars can cover request to several pillars, while the primitive for the actual get part only concerns one pillar.

Note also that there can be a number of different reponses, in cases where the pillar want to give information of the progress of the ioperation at pillar level. This can for instance be interesting for DVD pillars where there are several possibly time consuming phases in finding the data - even more so for a put operation.

Conversation Id

We generate a unique conversation id at initiation of a conversation such that it is possible to identify which conversation a message belongs to.

Reuse and overlap between operations

The operations are design to have as many similarities and overlapping definitions as possible. For instance the getChecksums must work simmilar to Get. Paging of data and parameters must work in the same way. See dicussion on http://kb-dk.atlassian.net/wiki/display/BITMAG/Coordination+layer+design+discussion#Coordinationlayerdesigndiscussion-Overlapmedprotokolspecifikation.

Paging af requests

It must be possible to make paging of request parameters. Paging of parameters can for instance be the case for object ids in a random check of files. It is needed since we cannot assume that clients can make prior pagination. See dicussion on http://kb-dk.atlassian.net/wiki/display/BITMAG/Coordination+layer+design+discussion#Coordinationlayerdesigndiscussion-Pagingafrequests.

Software

Independence

In the design we need to ensure that we are as independent of the software chosen as possible. If special features are used, e.g. for optimisation, then the design must be flexible enough to skip use of the special features at a later stage.

...

  • it is sufficient to identify a data transmission transaction by a token
  • the token is sufficient information as basis for completion of a data transmission

Use of tokens

Tokens are used as designation of where the data can be put to or downloaded from.

...

Paging

Paging is in the first versions managed by a simple mechanism where a segment of a file is identified by parameters. Any Get command can contain an offset along with a length parameter, which defines the segment of the file. this is also sufficient to handle receival of divided files, and thus enabling rudimentary support of restart of an interrupted transmission.

...