DataONE Architecture

The Data Observation Network for Earth (DataONE) is an NSF funded project in the DataNet program charged with implementing and deploying core services and infrastructure to improve the long term accessibility and re-use of data collected through NSF funded and other research projects.

These documents provide the reference for implementation of services and tools that together form the DataONE cyberinfrastructure.

The source for these documents is located in the subversion repository. For hints on editing, please review About These Documents. The HTML version of the document is generated from the current version of the source documents and is available from:

https://purl.dataone.org/architecture-dev

Design Documents

• DataONE Overview
• User Scenarios
• What is Data (DataONE Perspective)?
• Data Packaging
• Content Discovery
• DataONE Preservation Strategy
• Replication Overview
• Mutability of Content in DataONE
• Reassignment of AuthoritativeMemberNode field for unhosted versions
• Replication of unhosted back-versions
• Immutability of Content in DataONE
• Summary
• Identifiers in DataONE
• Identity Management and Authenticated Session Management
• Authorization in DataONE
• Authorization and Authentication in DataONE
• Supporting Access Control in Search
• System Metadata
• Getting a Handle on Systems Metadata for the Long Haul
• Natural History of System Metadata
• Analysis and Modeling Tools
• Investigator Toolkit
• Use Cases
• Serialization of Types for Transfer Over HTTP
• Event Logging and Reporting
• NodeList
• Querying DataONE
• DataONE Cybersecurity Plan
• Coordinating Node Internals
• Log Aggregation Overview
• (Proposal) Member Node Service Registration
• What is This Thing?
• Supporting multiple API versions
• Very Large Data Packages
• DataONE CN OS Core postinst and Configuration
• Spatial Search and Plotting Using Geohashes
• Introduction
• DataONE Search Index and Geohashes
• Using Geohashes for plotting
• Using Geohashes for searching
• Geohash algorithm
• User Scenarios
• Use Cases
• Node Identity and Registration
• What is Data (DataONE Perspective)?
• Data Packaging
• DataONE Preservation Strategy
• Replication Overview
• Mutability of Content in DataONE
• Reassignment of AuthoritativeMemberNode field for unhosted versions
• Replication of unhosted back-versions
• Identifiers in DataONE
• Content Discovery
• Querying DataONE
• Supporting Online Citation Managers through COinS
• DataONE Cybersecurity Plan
• Identity Management and Authenticated Session Management
• Authorization in DataONE
• Supporting Access Control in Search
• System Metadata
• Getting a Handle on Systems Metadata for the Long Haul
• Natural History of System Metadata
• Serialization of Types for Transfer Over HTTP
• NodeList
• Coordinating Node Internals
• Event Logging and Reporting
• Log Aggregation Overview
• DataONE Usage Statistics
• Analysis and Modeling Tools
• Investigator Toolkit
• What is This Thing?

API Reference

• DataONE APIs
  • REST Interface Overview
    • Message Serialization
    • Parameters in Requests
  • Member Node APIs
    • Core API
    • Read API
    • Query API
    • View API
    • Package API
    • Authorization API
    • Storage API
    • Replication API
  • Coordinating Node APIs
    • Diagnostic API
    • Core API
    • Read API
    • View API
    • Authorization API
    • Identity API
    • Replication API
    • Register API
  • Investigator Toolkit APIs
    • Java Tools
    • Python Tools
  • Version 1.0 Data Types
  • Version 1.1 Data Types (extend v1.0)
  • Version 2.0 Data Types (extend v1.1)
  • Exceptions Raised by MN and CN APIs
    • General
    • HTTP Exception Handling Codes
    • Serializing Exceptions
    • The Exceptions
  • Infrastructure Monitoring APIs
  • APIs Internal to Coordinating Nodes

Implementation Notes

• Implementation Notes
  • Implementation Roadmap
  • Components of the DataONE Infrastructure
  • Implementation Priority
  • Integration Testing
  • Infrastructure Versions

Notes and Miscellanea

• General Design and Implementation Notes
  • Logging and Privacy concerns
  • Cross Domain Indexing and Access for Data and Metadata
  • Replication Notes
  • Notes for MN API
  • Coordinating Node Requirements
  • Apache Configuration for DataONE Services
  • Mutability of Metadata
  • Selectors for Data Package Components
  • Time and Bandwidth Constraints
  • Proposal for API Refactoring
• License and Copyright Policy
  • Open source licensing
  • Copyright
  • Copyright and license notification
• Acknowledgements
• About These Documents
  • Editing Content
  • Building the Documentation
• Glossary
• Change Log
  • 2014-02-12: Version 1.2. Clarification edits
  • 2013-10-17: Version 1.2. Minor edits
  • 2012-10-09: Version 1.1. Clarifying count parameter for slicing
  • 2012-09-28: Version 1.1. Adding MNQuery API
  • 2012-09-25: Version 1.1. Revising proposed Query API
  • 2012-08-20: Version 1.1. Augmenting Search API
  • 2012-05-31: Version 1.0, Correction to URIs in Resource Maps
  • 2012-05-02: Version 1.0, MethodCrossReference.xls
  • 2012-05-01: Version 1.0, MethodCrossReference.xls
  • 2012-03-26: Version 1.0, MethodCrossReference.xls
  • 2012-03-26: Version 1.0, MethodCrossReference.xls
  • 2012-03-25: Version 1.0, MethodCrossReference.xls
  • 2012-02-27: Version 1.0, MethodCrossReference.xls
  • 2012-02-24 : dataoneTypes.xsd
  • 2012-02-17 : Version 1.0, MethodCrossReference.xls
  • 2012-02-15 : Version 1.0, MethodCrossReference.xls
  • 2012-02-02 : Version 1.0, MethodCrossReference.xls
  • 2012-02-01 : Version 1.0, MethodCrossReference.xls
  • 2012-01-20 : Version 1.0, MethodCrossReference.xls
  • 2012-01-19 : Version 1.0, MethodCrossReference.xls
  • 2012-01-12 : Version 1.0, MethodCrossReference.xls
  • 2012-01-10 : Version 1.0, MethodCrossReference.xls
  • 2012-01-03 : Version 1.0, MethodCrossReference.xls
  • 2011-12-19 : Version 1.0, dataoneTypes.xsd
  • 2011-12-13 : Version 1.0
  • 2011-12-12 : Version 1.0
  • 2011-12-08 : Version 1.0
  • 2011-12-06 Version 1.0.0 Branch
  • 2011-12-05 Version 1.0.0 Branch
  • 2011-11-08 Version 1.0.0 Branch
  • 2011-11-03 Version 1.0.0 Branch
  • 2011-11-02 Version 1.0.0 Branch
  • 2011-11-01 Version 1.0.0 Branch
  • 2011-10-26 Version 1.0.0 Branch

Indices, Tables, and Change Log

• Index
• Module Index
• Search Page