Harvesting metadata from catalogs
The ability to 'harvest' or transfer collections of metadata records between catalogs to keep them synchronized is a desirable operation in a federated catalog system. The major reasons are performance and reliability. Although the OGC catalog service specification (CSW) includes provisions for federated catalogs to propagate requests to other servers, most people that we have talked to who actually have implemented such functionality report that perfomance is too slow and unreliable. If one of the servers a request is cascaded to is not functioning, it can freeze the entire process, the response is only as fast as the slowest server, and the client must determine how to identify duplicate result records. Harvest and cache of metadata records allows particular metadata registries to specialize on particular kinds of content, and to index records and create stored views to optimize performance with the records held in that registry.
Discussions with developers of Stratigraphy.net indicate that there are problems using the CSW metadata harvesting operations in the context of geoscience metadata resources. They recommend use of the Open Archives Initiative Profile for Metadata Harvesting (OAIPMH) for Harvesting services. They report that based on their experience and that of others, that a distributed metadata catalog architecture based on a collection of metadata providers and portal servers that harvest and cache metadata records is a more viable design than a real time distrubuted query system.
Latest News
| Related Community Groups |
|---|
|
CSW Debug Blog | 17 Posts | Join A group blog to discuss metadata Catalog Service for the Web (CSW) implementation experiences |
|
Building a GeoSciML WFS Server | 11 Posts | Join Development, testing and implementation of a WFS service that returns GeoSciML documents |
|
ETL Debug Blog | 12 Posts | Join A group blog on implementing and debugging Extract-Transform-Load (ETL) efforts. |
|
Presentations and Posters | 4 Posts | Join Post your posters and presentations related to USGIN topics. |
|
Metadata interest group | 5 Posts | Join group for general posting on metadata content, standards, tools |
|
USGIN Amazon Virtual Server Development | 17 Posts | Invite only Documenting the process of development of a Web Server in the Amazon EC2 environment. Software installations tailored to the requirements for USGIN |
|
GeoNetwork configuration and development | 2 Posts 1 | Join Discussion on GeoNetwork setup, configuration, and development. |
|
USGIN ISO19139 profile discussion summary | 0 Posts | Request membership Comment and discussion on the proposed USGIN ISO19139 profile will be posted here |
|
Student Projects | 0 Posts | Join Discussion of student projects related to USGIN |
|
Drupal Development | 6 Posts | Join All about bending Drupal to your needs |
