Repository
From the large repository list at re3data.org we have investigated the following repositories
- Global repositories
- Danish repositories
- KB (based on Dataverse)
- Dansk Data Arkiver
Category
We have established the following criteria that we will use to evaluate each repository
- Price
- Initial subscription price
- Free storage size limit
- Either total price or price per year
- Size limit
- Default uploaded file size
- Uploaded file size on request
- Number of files that can be stored
- Total allowed storage capacity
- Allowed data types
- Data types
- What is the plan for ensuring that the data type can be read in the future?
- Research restrictions
- Material not allowed
- Research phase
- Research topic requirement (Like astronomy, biology, archeology and so on )
- Access to data
- Conditions that the files have to fulfill
- Tracking users and statistics
- Restricted Access
- Data security
- How many copies of the data are present?
- How protected are the data?
- Location
- Accept material from
- Ownership
- Ability to have versions
- Safety
- Sustainability
- What is the plan in case the service gets terminated?
- What funding is there?
- How much do the users of the service contribute with
- How high is the risk of the service getting terminated
- Withdrawal of data
- Revocation of DOIs
- Data privacy
- Virus check
- Encryption
- Form restrictions (like corpus, survey etc)
- Policy/Law
- Preservation of data
- Retention period
- Functional preservation
- File preservation
- Fixity and authenticity
- Metadata
- Access and reuse
- License
- Requirement/wishes (Ingest, Data Management, Preservation, Administration, Archival Storage and General)
- Ingest
- Submission Information Package (SIP)
- Record decision relating to SIP
- Identify data objects
- Validate files
- Automated extraction of metadata
- Virus check files on ingest.
- Record related physical material
- Select and configure automation
- Process large numbers of large files
- Data Management
- Persistent unique internal identifiers
- PDI associated with content information
- PREMIS metadata schema
- Describe data at different levels of granularity
- Maintain relationship between different file representations
- Store technical metadata extracted from files
- Fulfill Dublin Core metadata
- Guarantee long-term preservation and integrity of the data
- Certified to the Trusted Repositories Audit and Certification standard
- Restrict access to sensitive data if this is necessary
- Preservation
- Preservation plans for individual or groups of files
- Automatic checking performed post-migration
- Record actions, migrations and administrative processes
- Administration
- Disposal of data where appropriate
- Record of what, when and why material got disposed
- Reporting capabilities so statistics can be collated
- Archival Storage
- Monitor the integrity using checksums
- Error report when data are lost or corrupted
- Compliant with OAIS
- General
- Integrate with a range of repository or content management systems
- The digital archive will integrate with our archival management systems.
- APIs for integrating with other systems
- Incorporate new digital preservation tools as they become available
- Extracting and exporting data and metadata
- Support and technical help is available
- Under active development
- Community of users exists
- Ingest
Evaluation Grid
Green - The system fulfills this requirement
Yellow - The system partially fulfills this requirement
Pink - The system does not fulfills this requirement
Category I | Category II | Zenodo | figshare | Dataverse | KB | Rigsarkivet |
---|---|---|---|---|---|---|
Price | Initial subscription price
| 0 | 0 | 0 | 0
|
|
Free storage size limit | No limits | 1GB | No limits | 10GB | ||
Price per GB above this limit | 0 | Not allowed for non-paying customers | 0 | 10kr | ||
Price per year | 0 | Different pricemodels (see figshare pricing) | 0 | GB/yearly 5kr | ||
Size limit | Default uploaded file size limit | 2GB | 250MB End of 2015 5GB | 10GB | 10GB |
|
File size limit on request | 10GB (in November) 1TB | 250MB End of 2015 5GB | 10GB | 10GB | ||
Number of files that can be stored | No limits | 1GB limit End of 2015 20GB | 1TB limit | No limits | ||
Total allowed storage capacity | No limits | No limits public 1GB private End of 2015 20GB | 1TB | No limits | ||
Data types |
| All formats are allowed - even preservation unfriendly. | All formats are allowed | All. SPSS-filer format are, though, changed from SAW/POR to TAB. | All. SPSS-filer format are, though, changed from SAW/POR to TAB. | Text, which get photographed and saved as TIFF files |
Research restrictions | Allowed researchers | All researchers. | All researchers | All researchers | All Danish researchers | Danish researchers. The material has to have substantial relevance for the danish state |
Material not allowed | Illegal material and illegal storage of copyrighted material will not be allowed. | Privacy-sensitive material | Material that are not important for the Danish society | |||
Research | Phase | Any document/dataset from any phase of the research lifecycle. Because each dataset gets a DOI and is difficult to remove, the managing active data phase is sort of tedious to use. | All phases | All phases | All phases Both the managing active data phase and the data repositories phase have focus, but the recommended preservation time period is 5-10 years | Long-term preservation |
| Disciplinary | All disciplines are accepted | All disciplines are accepted | ? |
| |
Access to data | Open data | Yes | Yes | Yes | Yes |
|
Embargoed data | Yes | Yes (end of 2015) | Yes | Yes | ||
Closed data | Yes | Yes | Yes | Yes | ||
Tracking users and statistics | Zenodo does not track, collect or retain personal information | ? | ||||
Restricted Access | Users may deposit restricted files with the ability to share access with others if certain requirements are met. | Metadata can always be seen by anybody, but users may restrict access to the actual data. | Metadata can always be seen by anybody, but users may restrict access to the actual data. | |||
Data security | Multiple redundant copies | Yes | Yes | Yes |
| |
How protected are the data? | Zenodo will take all reasonable measures to protect the privacy of its users and to resist service interruptions, intentional attacks, or other events that may compromise the security of the Zenodo website. | |||||
Location |
| Europe | Denmark | Denmark | ||
Accept material from | International researchers | Danish researchers | Danish researchers | |||
Ownership | All uploaded content remains the property of the parties prior to submission | All uploaded content remains the property of the parties prior to submission | it offers distributed ownership for data authors through virtual web archives named "dataverses." | it offers distributed ownership for data authors through virtual web archives named "dataverses." | ||
Ability to have versions | Data files are versioned. Records are not versioned. Records can be retracted from public view; however, the data files and record are preserved. | Yes - in end of 2015 | Yes | ? | ||
Safety | Data files are kept in multiple replicas in a distributed file system, which is backed up to tape on a nightly basis. | Data files are kept in the Amazon data network storage |
| |||
Sustainability | What is the plan in case the service gets terminated? | If terminated researchers are advised where they can store their research best | No plan | No plan | ? |
|
What funding is there | Funded by EU | None | Funded by the danish state | |||
How high is the risk of the service getting terminated | No risk | Low risk | Low risk | High risk till more researchers use it | No risk | |
Data | Withdrawal | If the uploaded research object must later be withdrawn, the reason for the withdrawal will be indicated on a tombstone page, which will henceforth be served in its place. Withdrawal is considered an exceptional action, which normally should be requested and fully justified by the original uploader. In any other circumstance reasonable attempts will be made to contact the original uploader to obtain consent. The DOI and the URL of the original object are retained. | Easy to delete a private file, but almost impossible to delete a published file | ? | ||
Revocation of DOIs | Content not considered to fall under the scope of the repository will be removed and associated DOIs issued by Zenodo revoked. User access may be revoked on violation of Terms of Use. | ? | ||||
| Encryption | No | Yes | Yes | Yes |
|
Virus check | Yes | |||||
| Storage of sensitive data | In so far that the data can be closed for access data are secure, so for instance social security numbers cannot be read by everyone | Launched in November 2015 Confidential files – Enables new collaboration opportunities by releasing a metadata-only record for files which are commercially or personally sensitive | Data can be selected to be viewable to only a number of people, so data are secure | Data can be selected to be viewable to only a number of people, so data are secure |
|
Policy/law | Access | Open for all, but users must specify a license for all publicly available files. Licenses for closed access files may be specified in the description field. | Open for danish researcher | Open for danish researcher | ||
Mandate | Government | Commercial | Institutional | Institutional | Government | |
Preservation | Retention period | Items will be retained for at least 20 years | Minimum of 10 years. | 5-10 years if enough researchers use it | The TIFF-files will be retained for at least 20 years | |
Functional preservation | Zenodo makes no promises of usability and understandability of deposited objects over time. | Planning to allow data consumers to download the files in original formats | Yes | |||
File preservation | Data files and metadata are backed up nightly and replicated into multiple copies in the online system. | Data files and metadata are backed up nightly and replicated into multiple copies in the online system. | ||||
Fixity and authenticity | All data files are stored along with a MD5 checksum of the file content. | All data files are stored along with a MD5 checksum of the file content. | ||||
Logical preservation | No | No | No | No | ||
Curation | Yes | No | No | ? | Yes | |
Metadata | Access and reuse | All metadata is exported via OAI-PMH and can be harvested. All metadata is stored internally in MARC according to the schema defined in Zenodo metadata. Metadata is exported in several standard formats such as MARCXML, Dublin Core, and DataCite Metadata Schema according to OpenAIRE Guidelines. | Each dataset gets a DOI that ensures future harvesting |
| ||
License | Metadata is licensed under CC0, except for email addresses. | |||||
Digital preservation requirement
| ||||||
INGEST
M – must
S – should
C – could | Requirement | Zenodo | figshare | Dataverse | KB | Rigsarkivet |
S | The digital archive will enable us to store administrative information relating to the Submission Information Package (SIP) (information and correspondence relating to receipt of the SIP). | The uploaded data is archived as a Submission Information Package | None | |||
S | The digital archive will include a means for recording appraisal decisions relating to the Submission Information Package and individual elements within it. | Yes | ||||
M | The digital archive will be able to identify and characterise data objects (where appropriate tools exist). | DOI | DOI | DOI | DOI | |
S | The digital archive will be able to validate files (where appropriate tools exist). | All data files are stored along with a MD5 checksum of the file content | ||||
S | The digital archive will support automated extraction of metadata from files. | Upcoming feature: Metadata extraction | ||||
S | The digital archive will virus check files on ingest. | No | ||||
C | The digital archive will be able to record the presence and location of related physical material. | |||||
S | It will be possible to select and configure the required level of automation within the ingest workflow. | |||||
M | The digital archive will be able to process large numbers of files and files that are large in size. | Large number of files that each are max. 10GB (soon to be 1TB) | Large number of files that each are max. 2GB | |||
DATA MANAGEMENT | ||||||
M | The digital archive will generate persistent, unique internal identifiers. | DOI | DOI | DOI | DOI | |
M | The digital archive will ensure that Preservation Description Information (PDI) is persistently associated with the relevant content information. The relationship between a file and its metadata/documentation must be permanent. | No | ||||
M | The digital archive will support the PREMIS metadata schema and use it to store preservation metadata. | No | ||||
S | The digital archive will enable us to describe data at different levels of granularity – for example metadata may be attached to a collection, a group of files or an individual file. | Yes - in end of 2015 | ||||
M | The digital archive will accurately record and maintain relationships between different representations of a file (for example, from submitted originals to dissemination and preservation versions that will be created over time). | |||||
M | The digital archive will store technical metadata extracted from files (for example that which is created as part of the ingest process). | |||||
M | The digital archive will fulfill Dublin Core metadata | Yes | ||||
M | The digital archive will guarantee long-term preservation and integrity of the data | Yes | ||||
M | The digital archive will be certified to the Trusted Repositories Audit and Certification standard | Will be in 2016 | ||||
M | The digital archive will have access controls that can be used to restrict access to sensitive data if this is necessary | Yes | ||||
PRESERVATION PLANNING | ||||||
M | The digital archive will allow preservation plans (such as file migration/normalisation) to be enacted on individual or groups of files. | |||||
C | Automated checking of significant properties of files will be carried out post-migration to ensure these properties are adequately preserved (where appropriate tools exist). | |||||
M | The digital archive will record actions, migrations and administrative processes that occur whilst the digital objects are contained within the digital archive. | |||||
ADMINISTRATION | ||||||
M | The digital archive will allow for disposal of data where appropriate. | |||||
S | A record must be kept of data disposal including what was disposed of, when it was disposed of and reasons for disposal. | |||||
S | The digital archive will have reporting capabilities so statistics can be collated. For example it would be useful to be able to report on numbers of files, types of files, size of files, preservation actions carried out. | |||||
ARCHIVAL STORAGE | ||||||
M | The digital archive will actively monitor the integrity of digital objects on a regular and automated schedule with the use of checksums. | Yes | ||||
M | Where problems of data loss or corruption occur, The digital archive will have a reporting/notification system to prompt appropriate action. | Yes | ||||
S | The digital archive will be able to connect to, and support a range of storage systems | |||||
GENERAL | ||||||
S | The digital archive will be compliant with the Open Archival Information System (OAIS) reference model. | No | No | Yes | ||
M | The digital archive will integrate with a range of repository or content management systems | |||||
S | The digital archive will integrate with our archival management systems. | |||||
S | The digital archive will have APIs or other services for integrating with other systems. | Yes | Yes | Yes | Yes | |
S | The digital archive will be able to incorporate new digital preservation tools (for migration, file validation, characterisation etc) as they become available. | No | No | No | No | |
M | The digital archive will include functionality for extracting and exporting the data and associated metadata in standards compliant formats. | |||||
S | The software or system chosen for the digital archive will be supported and technical help should be available. | |||||
S | The software or system chosen for the digital archive will be under active development. | Yes | Yes | Yes | ||
S | A community of users will exist around the software or system to enable sharing of use cases, workflows and to promote developments in line with changes and innovations in the discipline of digital preservation. | No | No | Yes |
1 http://digital-archiving.blogspot.co.uk/2013/12/my-digital-preservation-christmas-wish.html
Links
The data archive in UK's FAQ, which asks and answers some interesting questions.
Similar question and recommendation: Where to archive and Choose Data Archive
A large register of different repository possibilities: Registry of Research Data Repositories