2021-04-06 Statusmeeting

• Broad crawl step 2 progressing nicely - Fixed crawl time pr. job – 15 days!
• We are are doing a market analysis regarding outsourcing the harvest part of web archiving and have already had 2 SWOT-workshops. 
• Bitmagasin-development/testing is progressing
• Stephen on paternity leave until may
• KB is looking for an IT-specialist: 
  

  https://app.slack.com/client/T1U8G0CA0/C1U8Y0117 (in english)
  https://candidate.hr-manager.net/ApplicationInit.aspx?cid=5001&ProjectId=141080&DepartmentId=6151&MediaId=5&SkipAdvertisement=False (in Danish)
  Will be around 50% web archiving and 50% other digital cultural heritage-tasks

We are pleased to announce that, last month, we published our selective crawls seed lists on the new version of the BnF website dedicated to APIs and datasets. These lists are created from BCWeb exports including some crawl settings and descriptive elements like themes and keywords.
In 2020, three new crawls were launched and added on the website: Instagram, Artificial Intelligence and Environnemental Issues.
You can consult all these lists at this address: https://api.bnf.fr/fr/liste-des-adresses-url-des-collectes-ciblees-du-web-francais-par-la-bnf
Another page which is focused on Covid-19 selections can be consulted at this address: https://api.bnf.fr/fr/node/176

For the second consecutive year, we launched an Instagram crawl. We plan to make five Instagram crawls, some of them are about specific subjects like the Olympic games or the regional and departmental elections in France.
Just like last year, we had to crawl picuki.com. Actually, in spite of many tests, we always end up being blocked by Instagram.

And finally, our in-house harvesting workshop about Flash is going to finish. It was complicated to find a way to harvest automatically some of the websites with Flash animations because some URLs are dynamically generated or relative, and so they are inaccessible to Heritrix. So we will try to discover all the URLs with the help of a human hand and we will launch the harvest in a second time.
In case of successful crawl, we will sometimes have an issue with compatibility of Flash plugin used with the Wayback.

• New contribution to coronavirus international crawl with a selection of 200 seeds
• Last month we had two meetings with our regional web curators from different part of Spain. We worked on the selection of seeds.
• This month we are working with our collaborators on a new event collection for the regional election in Madrid
• We continue to work on regional election in Catalonia