USGIN Community Homepage

Welcome to the USGIN Community page! This is a great place to start getting involved in the USGIN project.

Community Tags

Latest Group Posts

Redirection Engine for USGIN-Compliant URI Resolution

Ryan Clark's picture
Ryan Clark 01/21/2011 - 4:41pm | Comments 0

A common problem we run into is the unique identification of resources. This is a very broad statement because it represents a very broad problem. We have identification schemes for all kinds of things: people have names, web pages have URLs, books have ISBN numbers, etc. For the USGIN system to work smoothly, we need an identification scheme for resources that are part of the network. In USGIN we use Universal Resource Identifiers (URIs) which follow a specific format. For more information about the format, and for more background, see this post on lab.usgin.org.  

Once we have this identification scheme built, it is important that the URIs can be dereferenced, or in other words that they are resolvable. We would like a user to be able to put the URI of a particular feature into the address bar of their web browser, push enter, and be presented with some sort of representation of their resource of interest. This is the niche that our Django-based redirection engine it trying to fill.

» Read more Using Django for USGIN

URI dereferencing with Django / HTTP Content Negotiation

Ryan Clark's picture
Ryan Clark 01/21/2011 - 12:36pm | Comments 0

I've been building a Django app that essentially allows for the dereferencing of URIs that follow the USGIN Scheme. The engine also does some content-negotiation by reading HTTP GET accept-headers and responding with an appropriate file located somewhere on the internet. I had never even heard about content negotiation before embarking on this, and here are a few things that I learned:

 

» Read more Using Django for USGIN

Django Setup

Ryan Clark's picture
Ryan Clark 01/17/2011 - 10:42am | Comments 0

Introduction

I'm relatively new to Django, and maybe to servers in general, but setting up Django is a chore. Or at least figuring out how to set it up is. There is no installer. You have to have a pretty good understanding of how your HTTP server (IIS, Apache, whatever) works before you're going to be able to get anywhere. Then you'll have to know how to set up quite a few ancillary applications (e.g. Python, WSGI, FastCGI, Flup, MySQL, PostgreSQL). The details of which other applications you want to set up depend entirely on the evironment you're building - so that means it is a little different every time - and that means it is pretty hard to write one single "walkthrough" for how you should do it.

What I want to do here is detail my setup, and along the way make some suggestions about what seems to make things easier. So here goes:

Suggestion: Use Ubuntu (Most recent stable version, always.)

I say this because that apt system really makes installations a breeze. I fumbled for a while to try and setup WSGI (a module for Apache) to work on my Windows machine, to no avail. In Ubuntu it is so easy:

» Read more Using Django for USGIN

Installing/Using Git Version/Revision/Source Control System on local network

lwolfe 01/13/2011 - 9:52am | Comments 0

Installing/Using Git Version/Revision/Source Control System on local network:

You can install the Git Windows portable 7z autounzip file and add environment variable for gitdir and add your gitdir\cmd\ to your path, or you can install the Git Windows exe for full Windows GUI install.

» Read more

PostgreSQL Trigger - make harvested/uploaded metadata records editable in Geoportal

Ryan Clark's picture
Ryan Clark 12/27/2010 - 12:03pm | Comments 0

By default, metadata records that are harvested into the Geoportal are not editable. This includes any records in the Geoportal that were not created explicitly with interface provided by the web application ("Use dedicated editor to create metadata manually"). The SQL below creates a database trigger that allows harvested records to be edited.

Be aware that there is the potential for a lossy data translation when you do this. The harvested record is completely rewritten using the schema definitions that your Geoportal implements. If the original harvested record has any data that is not included in your schema implementation, that data will be lost after the original is edited. Another way to put that is, if you edit a harvested record, the edited version will only contain information that you see in the editor interface. If the original started with anything else in it, that will be lost.

» Read more Geoportal on an Amazon Virtual Machine

Usgin Draft Specifications For Geoscience Resource Metadata And Identifiers

srichardAzgs's picture
srichardAzgs 11/18/2010 - 5:11pm | Comments 0

Poster presented at Geological Society of America Meeting, Monday, Nov. 1, 2010 in Denver, Colorado. Overview of metadata recommendations, catalog system architecture, and proposed URI scheme.

» Read more Presentations and Posters

Recommendations for metadata content to describe geoscience resources

wgrunberg's picture
wgrunberg 11/18/2010 - 9:32am | Comments 1

Now that we have a better grasp of the IT and informatics challenges of a distributed and interoperable information network, we can think more again about its adoption - how to make it work and useful on the ground. Following was our rough chain of thought on metadata:

» Read more Metadata interest group

Install Apache Directory Server

Ryan Clark's picture
Ryan Clark 10/28/2010 - 3:38pm | Comments 0

ESRI's Geoportal wants an LDAP server, and suggests using Apache Directory Server.

I started a new VM using the image usgin2-atpms. Installation was as easy as:

wget http://apache.imghat.com//directory/apacheds/unstable/1.5/1.5.7/apacheds-1.5.7-i386.deb
sudo dpkg -i /home/ubuntu/apacheds-1.5.7-i386.deb

It was easy to connect to via SSH Tunnel to port 10389. The next step would be to fix some issues with the instance and its data-store, remove the extra MySQL installation, and then move on to configuration of the PostgreSQL database.

» Read more Geoportal on an Amazon Virtual Machine

Windows 7 64-bit guide for GeoNetwork development within Eclipse

wgrunberg's picture
wgrunberg 09/16/2010 - 3:12pm | Comments 0

This setup guide for downloading GeoNetwork using Subversion, building using Maven 2, developing, deploying and debugging on Tomcat and Eclipse on Windows 7 64-bit still uses the MckoiDB and should be updated to use MySQL.

Introduction

This document was developed for GeoNetwork’s new Maven 2 based setup. See http://trac.osgeo.org/geonetwork/wiki/Maven for more information.

Note that several installation and configuration options may not be necessary for GeoNetwork development but can be necessary for other software development projects.

This document was compiled from experiments, notes, and web sites by the following contributors and sources (in alphabetical order):

» Read more GeoNetwork configuration and development

Python Pylint/Pydev Code Checking in Eclipse

lwolfe 09/16/2010 - 2:07pm | Comments 0

Pylint is a good python code checking tool.  Pydev is a good plugin for Eclipse for editing and running python programs.  Much of this info is taken from Ryan Fraser's page .  Wolfgang has an alternative description for installing pylint in the Komodo python IDE here

» Read more GeoNetwork configuration and development

NCGMP to GeoSciML: MappedFeature

Ryan Clark's picture
Ryan Clark 09/03/2010 - 8:30am | Comments 0

I posted an example of the <typeMappings> section of a GeoServer/app-schema mapping file here: A Sample GeoServer Mapping file for NCGMP09 to GeoSciML. Checkout this other post to see where your mapping file should be, and how your datastore will refer to it: GeoServer Complex-Feature WFS Configuration: Directory Structure.

The <typeMappings> section is where you indicate what fields in your source data (NCGMP09) go into which fields in your output data (GeoSciML). Here I want to describe the different levels of complexity you can undergo when making a mapping file.

The Simplest Case: From one Field to Another

» Read more Building a GeoSciML WFS Server

Converting FGDC XML metadata records for WMS and WFS services to ISO 19139 via Python and GeoNetwork

wgrunberg's picture
wgrunberg 09/02/2010 - 2:04pm | Comments 0

This post describes a series of Python scripts to convert FGDC XML metadata for WMS and WFS services to ISO 19139 service metadata via GeoNetwork's OGC harvest service. Here are the steps:

  1. Extract WMS and WFS GetCapabilities URLs from FGDC XML metadata and write them to a file.
  2. Create a GeoNetwork Harvest Node from extracted GetCapabilities URLs and let GeoNetwork's OGC WMS/WFS harvester create ISO 19139 metadata from GetCapabilities response. Note: the resulting ISO 19139 metadata is only as good as the one in the GetCapabilities response. In addition, GetCapabilities responses do not include all fields necessary for minimum ISO 19139 metadata.
  3. Copy some FGDC metadata entries (Title, Abstract, etc.) to newly created ISO 19139 metadata.

Our goal is to have working WMS and WFS metadata records in our CSW catalog that can be used to add the described services to analytical software such as ESRI's C-SW Client for ArcGIS Desktop. Following are the reasons why I am currently choosing this convoluted process instead of a direct FGDC to ISO 19139 metadata conversion:

» Read more ETL Debug Blog, GeoNetwork configuration and development

Create a GeoNetwork OGC Harvest Node (WMS or WFS GetCapabilities to ISO 19139 metadata) through xml.harvesting.add request

wgrunberg's picture
wgrunberg 08/17/2010 - 9:17am | Comments 0

GeoNetwork offers a harvest service that, among other, follows a OGC WMS or WFS GetCapabilities URL and transforms the response into an ISO 19139 metadata record. Bear in mind that the resulting ISO metadata record is only as good as the GetCapabilities response and that the metadata can never be entirely ISO 19139 conformant without dummy values due to limitations of the OGC GetCapabilities schema.

GeoNetwork offers a handy user interface to add those "GeoNetwork Harvesting Nodes" and also exposes their functionality through a even handier XML service. See chapter 19.3 on "Harvesting Services" of the GeoNetwork opensource V 2.4 The Complete Manual. Following are my notes on creating an OGC harvesting node through GeoNetwork's harvesting service.

» Read more ETL Debug Blog, GeoNetwork configuration and development

Import MEF metadata archive into GeoNetwork through Python

wgrunberg's picture
wgrunberg 08/17/2010 - 8:53am | Comments 0

This is a Python example script for importing GeoNetwork Metadata Exchange Format 1.1 (MEF) archives to GeoNetwork 2.4.2's mef.import service. The mef.import service requires a multipart/form-data POST through a modified MultipartPostHandler.py library which now supports Unicode (urllib2). It has been tested in Windows XP and Python 2.6.

» Read more ETL Debug Blog, GeoNetwork configuration and development

Create GeoNetwork MEF files from ISO 19139 XML through Python

wgrunberg's picture
wgrunberg 08/17/2010 - 8:53am | Comments 0

This is an example Python script that showcases the creation of GeoNetwork Metadata Exchange Format 1.1 (MEF) archives from ISO 19139 metadata XML files.It has been tested in Windows XP and Python 2.6.

» Read more ETL Debug Blog, GeoNetwork configuration and development

Debugging Python scripts with Pylint

wgrunberg's picture
wgrunberg 08/05/2010 - 11:16am | Comments 0

I am currently using Pylint to check for bugs and formatting errors in my Python scripts. I used Brandon Corfman's nice Komodo Hacks: Integrating Pylint tutorial to set up Pylint and how to integrate it with the Komodo Edit IDE (see Easy setup of a Python 2.6 development environment on Windows about setting up Python).

Here is my updated Pylint execution macro for Komodo Edit 5 which includes a verbose and an errors only reporting command.

» Read more ETL Debug Blog

CatalogConnector CSW Client

wgrunberg's picture
wgrunberg 08/04/2010 - 12:24pm | Comments 2

USGIN CatalogConnector test client http://catalog.usgin.org/catalogConnector/ (version 0.5)

This promising, stand-alone, Java-based, CSW web client is currently being developed by SDI Catalonia (IDEC) and USGIN at http://sourceforge.net/projects/catalogconnecto/

» Read more CSW Debug Blog

CSW Clients and Servers we are testing

wgrunberg's picture
wgrunberg 07/31/2010 - 9:23am | Comments 4

Following is a list of Web Service client, server, and hybrid applications we are aware of that are supposed to handle the CSW 2.0.2 protocol. Check out OGC's registry of products for more CSW implementations.

» Read more CSW Debug Blog

Running Drupal on a Windows Server 2008 machine

Ryan Clark's picture
Ryan Clark 07/15/2010 - 12:36pm | Comments 0

We recently struggled through figuring out how to make Drupal run on a Windows Server 2008 machine that was running IIS. Every Drupal site that we had dealt with prior to this was hosted on a Linux machine running Apache, MySQL and PHP, and so we needed to figure out what could be done differently to make it work on our in-house server, which was already running a production instance of ArcGIS Server under IIS7.

From the outset, we had two options:

» Read more Drupal Development

FGDC XML schema woes

wgrunberg's picture
wgrunberg 06/02/2010 - 3:08pm | Comments 2

I am working on transforming FGDC XML metadata records to the USGIN 1.1 version of ISO 19139 metadata and having a hard time finding formal FGDC schema, name spaces, and schema locations. This is what I found so far:

» Read more ETL Debug Blog
Related Community Groups
CSW Debug Blog | 17 Posts | Join
A group blog to discuss metadata Catalog Service for the Web (CSW) implementation experiences
Building a GeoSciML WFS Server | 11 Posts | Join
Development, testing and implementation of a WFS service that returns GeoSciML documents
ETL Debug Blog | 12 Posts | Join
A group blog on implementing and debugging Extract-Transform-Load (ETL) efforts.
Presentations and Posters | 12 Posts | Join
Post your posters and presentations related to USGIN topics.
Metadata interest group | 13 Posts | Join
group for general posting on metadata content, standards, tools
USGIN Amazon Virtual Server Development | 18 Posts | Invite only
Documenting the process of development of a Web Server in the Amazon EC2 environment. Software installations tailored to the requirements for USGIN
GeoNetwork configuration and development | 7 Posts | Join
Discussion on GeoNetwork setup, configuration, and development.
Student Projects | 0 Posts | Join
Discussion of student projects related to USGIN
Drupal Development | 6 Posts | Join
All about bending Drupal to your needs
Geoportal on an Amazon Virtual Machine | 3 Posts | Closed
Installation, configuration, etc.
Using Django for USGIN | 7 Posts | Request membership
Thought and ideas about using Django to accomplish USGIN-related... things.
ArcGIS Server and OGC Services | 3 Posts | Join
Tips on using ArcGIS Server to provide OGC web services
Content model discussion | 0 Posts | Request membership
Community site for comments on development of content models and encoding for information intechange
Making Web Maps | 2 Posts | Request membership
For information about the myriad of mechanisms for showing service data on a web page.
Troubleshooting Web Service Deployment - Blog | 5 Posts | Join
This blog is for documenting our group's experiences with web service deployment.
Best Practices for USGIN Web Service Hosting | 10 Posts | Join
Tips, techniques, and frequently asked questions for hosting AASG Geothermal Data Web Map Services and Web Feature Services
Hub Disaster Recovery | 0 Posts | Request membership
Discussions around how to harden a distributed federated system against disaster; setting up a system to mirror hub VMs at other hubs.