JWAT v0.8.0 (2012-02-09)

Released 2012-02-09.

Highlights

GZip reader/validator

Gives the ability to read a GZip file comprising of one or more concatenated GZip entries. Each entry's header is validated against the RFC specifications and the entries compressed data is made available though an InputStream. Will be refactored for v1.0.

ARC reader/validator

Read and validate v1.0 and v2.0 ARC files

v1.0 and v2.0 files are parsed and validated according to the specification.

v1.1 extension supported.

The semi-official v1.1 format is also supported.

Payload wrapper

Payload is supported and exposed to the end-user by means of a wrapper.

HttpResponse supported

HttpResponses are detected and parsed/validated and exposed to the end-user.

WARC reader/validator

Reads and validates WARC version 1.0 files. (Not all parts of the ISO are covered yet)

Multi-line headers are supported

Multi-line headers are parses and validated with the exception of UTF-8, quoted-strings and encoded words.

All WARC headers are parsed and validated

All WARC headers are parsed and validated according to type. (String, Numeric, URI, Date, IP, Content-Type, Digest)

WARC headers present in a record are validated according to the record's Warc-Type

Each WARC header is validated according to the WARC-Type of the record it is located in. According to the ISO standard header presence can either be: must, must not, shall, shall not, may or ignore.

Payload wrapper

Payload is supported and exposed to the end-user by means of a wrapper.

HttpResponse supported

HttpResponses are detected and parsed/validated and exposed to the end-user.

Full list of issues resolved in this release.