2016-11-29 Statusmeeting

Broad crawl

The third broad crawl 2016 finished on  30 september. The same applies for the crawls of ministeries and ultra-big sites.

Event crawls

A roadmap for event crawls of parliamentary and local elections is almost in place. Thus we will be able to identify candidates rapidly and press the start button as soon as the call for election is out.

Selctive crawls

Our new crawl strategy is in place.

We gave our Social Media crawls a makeover. We have revised the list of Twitter profiles and hashtags and started to crawl Danish Instagram profiles.

NAS 5.2 is implemented in our test environment. We had a bug with the job ending – the jobs did not finish. Probably solved by now, we are testing again.

Crawls

• In November we are also starting the first crawls related to the 2017 French presidential elections, as the primaries for both the centre/right and the ecologist candidates are being held. We will have two crawls, one for the results of the ecologist primary and the build-up to the centre/right primary, and another at the end of the month for the results of the centre/right.
• Our broad crawl is still running, about to end (we expect to harvest c. 84TB).

Access

• We released a new "Parcours guidé" (collection highlight) on 1990s archives which was written by researchers.
• We are giving access to a prototype with full text search of our 1996-2000 collection using Shine.

Celebration

- On November 23, we celebrated 10th anniversary of the law that gave us our legal mandate and the 20th anniversary of the very first web archives. We organised with Ina a one-day conference with 150 participants and 40 speakers.

 

• The crawl about our presidential elections still running

• Compressing of all  metadata arc Files by using the jwat tools is finished. We did gain around 1,7 TB which is more than expected and this helps us a lot.
• In Production we switched now to NAS 5.2 stable.
  
  

• On November, 7th, we celebrated a working meeting at the National Library with web curators from the Library and from the regional libraries to update our collaborative work on non-print legal deposit and web archiving. Regional web curators and more and more involved and committed with the web archiving Project.
• All of them are using intensively the Spanish version of BCWeb (what we renamed CWeb), but in a preproduction environment to avoid eventual dangers. And then we export to production CWeb those seeds they want to run and be crawled. So many of our regional libraries are building now there own web collections. I include two screenshots of our CWeb.

 

• Our General Elections crawl lasted 11 months, because we had two elections one in december 2015 and the second one in June 2016. Once we had a new Government and Prime Minister the crawl was closed.
• The event collection on the Elections in Galicia was closed also recently and was fully managed by regional web curators from Galicia.
• Regarding thematic crawls, we have been requesting collaboration from the main research institution in Spain (the Council of Scientific Research) and the University Libraries Network. Both of them showed much interest in participating, but as they both tie together many libraries, we are waiting for their answers to materialise the collaboration in specific measures. We hope to widen soon our web curators working group.
• The National Library of Colombia asked us for support, because they want to build a web archiving Project, starting with a collection on the Peace Process in their country. We have had two long meetings via Skype and they want to try NAS. I'll invite them to register in the user mailing list, and keep you informed about their progress.