A common problem we run into is the unique identification of resources. This is a very broad statement because it represents a very broad problem. We have identification schemes for all kinds of things: people have names, web pages have URLs, books have ISBN numbers, etc. For the USGIN system to work smoothly, we need an identification scheme for resources that are part of the network. In USGIN we use Universal Resource Identifiers (URIs) which follow a specific format. For more information about the format, and for more background, see this post on lab.usgin.org.
Once we have this identification scheme built, it is important that the URIs can be dereferenced, or in other words that they are resolvable. We would like a user to be able to put the URI of a particular feature into the address bar of their web browser, push enter, and be presented with some sort of representation of their resource of interest. This is the niche that our Django-based redirection engine it trying to fill.
I've been building a Django app that essentially allows for the dereferencing of URIs that follow the USGIN Scheme. The engine also does some content-negotiation by reading HTTP GET accept-headers and responding with an appropriate file located somewhere on the internet. I had never even heard about content negotiation before embarking on this, and here are a few things that I learned:
I'm relatively new to Django, and maybe to servers in general, but setting up Django is a chore. Or at least figuring out how to set it up is. There is no installer. You have to have a pretty good understanding of how your HTTP server (IIS, Apache, whatever) works before you're going to be able to get anywhere. Then you'll have to know how to set up quite a few ancillary applications (e.g. Python, WSGI, FastCGI, Flup, MySQL, PostgreSQL). The details of which other applications you want to set up depend entirely on the evironment you're building - so that means it is a little different every time - and that means it is pretty hard to write one single "walkthrough" for how you should do it.
What I want to do here is detail my setup, and along the way make some suggestions about what seems to make things easier. So here goes:
Suggestion: Use Ubuntu (Most recent stable version, always.)
I say this because that apt system really makes installations a breeze. I fumbled for a while to try and setup WSGI (a module for Apache) to work on my Windows machine, to no avail. In Ubuntu it is so easy:
Version/Revision/Source Control System on local network:
You can install
the Git Windows portable 7z autounzip file and add environment variable for gitdir
and add your gitdir\cmd\ to your path, or you can install the Git Windows exe
for full Windows GUI install.
By default, metadata records that are harvested into the Geoportal are not editable. This includes any records in the Geoportal that were not created explicitly with interface provided by the web application ("Use dedicated editor to create metadata manually"). The SQL below creates a database trigger that allows harvested records to be edited.
Be aware that there is the potential for a lossy data translation when you do this. The harvested record is completely rewritten using the schema definitions that your Geoportal implements. If the original harvested record has any data that is not included in your schema implementation, that data will be lost after the original is edited. Another way to put that is, if you edit a harvested record, the edited version will only contain information that you see in the editor interface. If the original started with anything else in it, that will be lost.
Now that we have a better grasp of the IT and informatics challenges of a distributed and interoperable information network, we can think more again about its adoption - how to make it work and useful on the ground. Following was our rough chain of thought on metadata:
It was easy to connect to via SSH Tunnel to port 10389. The next step would be to fix some issues with the instance and its data-store, remove the extra MySQL installation, and then move on to configuration of the PostgreSQL database.
This setup guide for downloading GeoNetwork using Subversion,
building using Maven 2, developing, deploying and debugging on Tomcat
and Eclipse on Windows 7 64-bit still uses the MckoiDB and should be
updated to use MySQL.
Pylint is a good python code checking tool. Pydev is a good plugin for Eclipse for editing and running
python programs. Much of this info is taken from
Ryan Fraser's page
. Wolfgang has an alternative description for installing pylint in the Komodo python IDE
The <typeMappings> section is where you indicate what fields in your
source data (NCGMP09) go into which fields in your output data
(GeoSciML). Here I want to describe the different levels of
complexity you can undergo when making a mapping file.
This post describes a series of Python scripts to convert FGDC XML metadata for WMS and WFS services to ISO 19139 service metadata via GeoNetwork's OGC harvest service. Here are the steps:
Extract WMS and WFS GetCapabilities URLs from FGDC XML metadata and write them to a file.
Create a GeoNetwork Harvest Node from extracted GetCapabilities URLs and let GeoNetwork's OGC WMS/WFS harvester create ISO 19139 metadata from GetCapabilities response. Note: the resulting ISO 19139 metadata is only as good as the one in the GetCapabilities response. In addition, GetCapabilities responses do not include all fields necessary for minimum ISO 19139 metadata.
Copy some FGDC metadata entries (Title, Abstract, etc.) to newly created ISO 19139 metadata.
Our goal is to have working WMS and WFS metadata records in our CSW catalog that can be used to add the described services to analytical software such as ESRI's C-SW Client for ArcGIS Desktop. Following are the reasons why I am currently choosing this convoluted process instead of a direct FGDC to ISO 19139 metadata conversion:
GeoNetwork offers a harvest service that, among other, follows a OGC WMS or WFS GetCapabilities URL and transforms the response into an ISO 19139 metadata record. Bear in mind that the resulting ISO metadata record is only as good as the GetCapabilities response and that the metadata can never be entirely ISO 19139 conformant without dummy values due to limitations of the OGC GetCapabilities schema.
GeoNetwork offers a handy user interface to add those "GeoNetwork Harvesting Nodes" and also exposes their functionality through a even handier XML service. See chapter 19.3 on "Harvesting Services" of the GeoNetwork opensource V 2.4 The Complete Manual. Following are my notes on creating an OGC harvesting node through GeoNetwork's harvesting service.
This is a Python example script for importing GeoNetwork Metadata Exchange Format 1.1 (MEF) archives to GeoNetwork 2.4.2's mef.import service. The mef.import service requires a multipart/form-data POST through a modified MultipartPostHandler.py library which now supports Unicode (urllib2). It has been tested in Windows XP and Python 2.6.
This is an example Python script that showcases the creation of GeoNetwork Metadata Exchange Format 1.1 (MEF) archives from ISO 19139 metadata XML files.It has been tested in Windows XP and Python 2.6.
Following is a list of Web Service client, server, and hybrid
applications we are aware of that are supposed to handle the CSW 2.0.2
protocol. Check out OGC's registry of products for more CSW implementations.
We recently struggled through figuring out how to make Drupal run on a Windows Server 2008 machine that was running IIS. Every Drupal site that we had dealt with prior to this was hosted on a Linux machine running Apache, MySQL and PHP, and so we needed to figure out what could be done differently to make it work on our in-house server, which was already running a production instance of ArcGIS Server under IIS7.
I am working on transforming FGDC XML metadata records to the USGIN 1.1 version of ISO 19139 metadata and having a hard time finding formal FGDC schema, name spaces, and schema locations. This is what I found so far:
USGIN Amazon Virtual Server Development | 18 Posts | Invite only Documenting the process of development of a Web Server in the Amazon EC2 environment. Software installations tailored to the requirements for USGIN