Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The goal of the JWAT library was to make a small package to read and validate WARC, ARC, and GZip and WARC files.

All the parsers were implemented on the premise that input data would be supplied in the form of streams and not files.

...

This is also the case when parsing/validating compressed WARC/ARC/WARC files where each record is GZip'ed. In which case the compressed data can also only be processed sequentially.

It is however possible to random access individual WARC/ARC/GZip/WARC records when working with the logical files and using a file offset. This is possible by using a simple RandomAccessFileInputStream present in the common package.

Since this project is intended to be of general use it also includes WARC and GZip writers.

Further documentation

If you can not find the information you seek you can always try the javadocs or look at the source code. As a last resort you are also welcome to email us with your inquiry.

...