Previous schema resolution had been achieved running GeoServer on a Linux platform, using Oracle VM VirtualBox. The previously attached files were specifically for app-schema set-up running on this platform. Though not a terribly different process, we also wish to create app-schema documentation for running GeoServer on a Windows platform. A popular software package for Windows is OpenGeo Suite (http://opengeo.org/)
which includes Postgres and PostGIS (database management systems) and
OpenLayers. Using the
PostGIS interface, a shapefile can be imported then converted to a
table within the PostGIS database. PgAdminIII is also included for managing PostGIS databases and data tables.
One obvious difference between the two platforms is the file location of the all-important 'workspaces' folder where the mapping and datastore.xml files must be located:
In the Oracle VM VirtualBox on Linux platform, the folder path to the workspaces folder is:
Working from either patform, each workspace created in the GeoServer GUI will have its own file in the workspaces folder. Within that file is the store name file, and within that file is the layer name file. The mapping and datastore.xml files must be in the store name folder, as shown in the graphic below.
where:
hf is the name of the Workspace (created in the GeoServer interface)
azheatflow is the name of the Store (created in the GeoServer interface)
HeatFlow is the name of the Layer (SQL view was created in the GeoServer interface)
HeatFlow.xml is the name of the mapping file (stored in the same location as the datastore.xml file)
Do not edit the namespace.xml or workspace.xml files, nor
the files that are in the HeatFlow folder (which are the featuretype.xml
and layer.xml files). When working on Windows platform, additional files will be added automatically to the workspace folder, called wfs.xml, wms.xml, and settings.xml. These also must remain unedited.
For a complete, working Windows platform directory, see the "HeatFlow" file (named after the original HeatFlow workspace) at the following GitHub directory: https://github.com/ccaudill/GeoserverWorkspace.
The
document linked to below is meant to assist in simple GeoServer set-up using OpenGeo
Suite as well as deployment of NGDS data services using Geoserver, with
some additional configuration considerations for use of application
schemas in Geoserver. It is assumed that schema-valid data tables
already exists in an object-relational database management system
(Postgres) and/or an open-source software program adding support to that
system (PostGIS). (For more information on schemas, visit http://schemas.usgin.org/models/.)
It must be noted that one important caveat to creating tables in any database structure using ArcMap is that a primary key is not indicated. We found this when the return of a GetFeature request in Geoserver using app-schemas showed that the data was not correctly mapped to the fields (or columns from the PostGIS data table). It appeared that the first column, the OBJECTID, was acutally being mapped as data and therefore put into the first column from the mapping file, which was ObservationURI. This caused more choas; as the schema was being read by Geoserver with a one-field offset, it misinterpreted some fields as not required and completely left them out of the GetFeature request. To fix this, we used PGAdmin to specify the primary key as the objectid in the table. Then, Geoserver was able to acknowledge this and leave that field's integer out of the data when the GetFeature was returned and thus giving a correct return of the data from that PostGIS table.
See the image below for the final GetFeature request example. For those using PGAdmin, specify a primary key by right-clicking the PostGIS database table and selecting "Properties". Under the "Constraints" tab, click the Add button beside "Primary Key". Give a name, such as "bht_p_key" then click the "Columns" tab to choose the appropriate field (here, the objectid). Click OK.
One thing yet to be resolved is our want for the objectid to appear in the gml:id. Doing this in the mapping file requires writing a short CQL statement (see examples in the previous comment), which did not even give us the results we wanted. It turns out that, according to Geoserver developers, "For performance reasons, we strongly recommend avoiding using CQL to
generate gml:id in the mapping file as this prevents the use of database
indices and the generation of efficient SQL queries."
So for now, we will continue to use the "getId()" expression recommended until a solution to using our original objectid can be found.
Also note that the name of your layer must match exactly the name of the targetElement in your mapping file, which is the element indicated in the schema. Thus, the layer name in Geoserver used for app-schema configuration must match the name specified in your schema document.
It seems that the 'New Mapping File' (above) will allow for schema resolution, which indicates the correct namespace and online schema location in the GetFeature request, along with the features of course. We're not finished yet, though; the idExpression in the mapping file now indicates for Geoserver to give each feature a gml:id, but what we really want is this ID to be indicated by the objectid that already exists in our PostGIS datatables.
"java.lang.IllegalArgumentException: Filter Function problem for function strConcat argument #1 - expected type String" when preforming a GetFeature request."
Other errors resolved along with way:
"java.lang.VerifyError: Cannot inherit from final class"
Be certain that the version of the app-schema download is the same version is the same version of Geoserver you're using. Those specific .jar files in the app-schema plug-in are essential. Using all the same versions cleared this error. See this page http://sourceforge.net/projects/geoserver/files/GeoServer%20Extensions/ for app-schema versions and downloads.
"Can't parse complex datastore config."
This error was resolved with an awful lot of tweaking the mapping file. See the attached for a polished example.
"org.xml.sax.SAXParseException:
Content is not allowed in prolog."
This error was caused by characters that were unintentionally added to the first line of the mapping file. This had two causes - one was the copy and paste method I used to create the file, and the second was the use of XML Copy Editor. This free XML editor evidently inserts it's own code into the file. Every time I opened and saved the file with this software, I would get this error. Removing characters in the mapping file that were before the first line ( i.e., <?xml version="1.0" encoding="UTF-8"?>) and using another free editor solved these problems for me.
When entering in the Service Description at the beginning of the Add New Service wizard, make this a robust description as it will give an opportunity for those visiting the service rest page to read about the service properties (software versions), the data specifications (OGC-compliance for AASG services), and description on viewing the WFS using the ArcGIS data interoperability extension.
Additional text appearing on the rest page are culled by ArcGIS from the properties of the layer and map document in the .mxd. Under the layer Properties, be sure to add a description that is specific to that service. For AASG services, this will indicate the state obtained from, the nature of the data, cross-referening fields (i.e., HeaderURI), symbology, and associated metadata or downloadable files (link to the repository page for AASG services). Under Map Document Properties in your .mxd, be sure to indicate the service title, author of the data, keywords, and credits.
Where I'd really like to see this go is the ability to use the commercial search engines (google, yahoo, bing, etc...) to index metadata that we (as data providers) expose using standard metadata formats and present search results in such a way that a search client can find and access data from within a user's application. The search client wouldn't be any more complicated that the standard file open dialogs (with a search box) currently presented by standard GUI interfaces-- maybe with an option to access a map-based interface for geospatial data.
The idea is that from within a web application like USGS the national map , or ArcGIS online, ESRIs ArcMap, MS Excel, ModFlow, GOCAD, or whatever data analysis or map browsing program you're using, you can search for data (pick your search engine --like OpenSearch enables...), get results, look at descriptions of what you found to pick the one you need, and 'get' it. Bingo, you have a new worksheet in your workbook or layer in your map layout.
To do this the search result has to return machine actionable links that client software can parse. The links need to include sufficient information that the client can pick the link that accesses a service it knows how to use and get data in a format it can use. There are a variety of specs out there for smarter links (Web Linking RFC-5988, XLink, ESIP service cast, OGC OWS context documents, ISO19115 CI_OnlineResource, atom:link...), we just need some convergence on what properties need to be associated with a link to make it machine actionable...
Here's a proposal for the properties of interest, based on the specs mentioned above...
href
the
link URL-this is what the client will actually ‘get’
title
free text to label link in GUI
type
MIME type of response. Specifies encoding and optionally the native
software application environment
rel
URI from IANA rel
vocabulary for consistency with IETF5988. Semantics of link from global
vocabulary for interoperability. Semantics in this context means calculatable
(see discussion in Coyle, 2010 p. 19).
purpose
analogous to ISO19115 CI_OnlineFunctionCode. Controlled
vocabulary—tells client why they’d use this link. Purpose property provides
mechanism for more granular, application specific indication of link
semantics.
protocol
ftp, http, …. Default is http if attribute not
specified
serviceType
URI that identifies a service protocol and version (one attribute or
two?). This specifies protocols for network layers above http/ftp etc. Should
be a URI that can dereference to some kind of service specification document.
outputscheme
profile for content of message retrieve by href URL;
URI for xml schema or JSON scheme, other description of data structure and
content? Need conventions, clear definition. Clients look at this to pick the
link that will get a representation they can use. This is the information
scheme in the layers on top of the MIME type encoding; note that the same
output scheme might be encoded using different MIME types, so the two are
somewhat orthogonal. MIME types have been conflating this property with the
lower level encoding (.vnd, +xml … stuff).
MDweb is a free tool for cataloging and
locating resources (data, documents and services) that are accessible
over the web. It is based on current geographic-information metadata
(ISO 19115, 19119) and communication (OGC’s CSW-2) standards and
conforms to the rules for implementing metadata and the associated
discovery services of the INSPIRE directives.
This activity is based in France:
Company IRD jean-christophe.desconnets@ird.fr dorian.ginane@ird.fr Geomatys vincent.heurteaux@geomatys.fr
Project is active, developing CSW client and Server. Current release 2.1.0, on 2010 July 30.
http://spatiumcube.sourceforge.net/components.htm
Project based in Spain, University of Zaragoza, associated with CatMDedit
Notes from their web site:
* Easy way of metadata loading. The storage of metadata records is managed directly through the file system.
* The server side of the product supports:
o the compulsory operations GetCapabilities, DescribeRecord, GetRecords and GetRecordById;
o transactional operations to allow users to add, delete and modify the catalog's contents.
* implements ISO19115 and DublinCore output schemas
* Ranked searching. Metadata are indexed based on Lucene search engine http://lucene.apache.org/java/docs/, so best results are returned first.
* Tested compatibility with other catalog tools such as GI-Cat/GI-Go http://zeus.pin.unifi.it/cgi-bin/twiki/view/GIcat/WebHome, Geonetwork http://geonetwork-opensource.org/ or CatalogConnector http://sourceforge.net/projects/catalogconnecto/.
* integrated Web Client application to test catalog services compliant with CSW 2.0.2. This client offers a guided interface to facilitate the definition of CSW requests (including maps and other graphic components for query filter expressions) using both KVP, XML and SOAP protocols. This client interface is multilingual (currently available in Spanish, English, French, Portuguese, Catalan, Galician and the Basque language; it can be customized for other languages) and can be configured to request other catalog services compliant with CSW 2.0.2 developed with different technologies like GI-Cat or GeoNetwork.
If you followed the installation instructions for PostgreSQL and PostGIS that I wrote up, the shp2pgsql command is not available globally from the command line. Browse to /opt/postgis/loader before running the command shown above. This is the case for AMI's usgin-2.0 and up.
Apparently, there is a limit to how many files the rm command can handle. It chokes on over 6000 files in Ubunto and throws an "unable to execute /bin/rm: Argument list too long" error. Here is work around:
Two problems that exist with this setup are the Help page does not load correctly and the map does not display very nicely. I've found the solution to the map problem:
Open web\intermap\WEB-INF\mapServers.xml. Locate the <default name="Layers for default map -- DUMMY NAME:not used"> element. Update it to look like this:
There are a few fixes that I have not yet placed in the PDF:
You need to place the arguments
-XX:MaxPermSize=256m -XX:PermSize=128m
in your project run configuration in Eclipse to avoid OutOfMemory errors.
Also, to debug your project, you need to add your project in the Source tab within the run configuration for your project. To do so, navigate to Run Configurations or Debug Configurations for your project and then select the Source tab. Next, click Add and then select Java Project and select your project.
Lastly, the Eclipse/Tomcat debugger will not detect any changes to your Java code unless you run build.xml and restart Tomcat. I am not quite sure why this is true, but it is a minor annoyance.
Until some use case(s) for the DCP property can be identified, and a logically coherent codelist that applies to the use cases, it is probably not worth worring about this!
I have copied this post as a discussion paper to the ISO 19139 USGIN Profile section (OGC Schema Issues in Relation to CSW ) because it is referenced in the USGIN Profile document.
Previous schema resolution had been achieved running GeoServer on a Linux platform, using Oracle VM VirtualBox. The previously attached files were specifically for app-schema set-up running on this platform. Though not a terribly different process, we also wish to create app-schema documentation for running GeoServer on a Windows platform. A popular software package for Windows is OpenGeo Suite (http://opengeo.org/) which includes Postgres and PostGIS (database management systems) and OpenLayers. Using the PostGIS interface, a shapefile can be imported then converted to a table within the PostGIS database. PgAdminIII is also included for managing PostGIS databases and data tables.
Working from either patform, each workspace created in the GeoServer GUI will have its own file in the workspaces folder. Within that file is the store name file, and within that file is the layer name file. The mapping and datastore.xml files must be in the store name folder, as shown in the graphic below.
where:
hf is the name of the Workspace (created in the GeoServer interface)
azheatflow is the name of the Store (created in the GeoServer interface)
HeatFlow is the name of the Layer (SQL view was created in the GeoServer interface)
Do not edit the namespace.xml or workspace.xml files, nor the files that are in the HeatFlow folder (which are the featuretype.xml and layer.xml files). When working on Windows platform, additional files will be added automatically to the workspace folder, called wfs.xml, wms.xml, and settings.xml. These also must remain unedited.
For a complete, working Windows platform directory, see the "HeatFlow" file (named after the original HeatFlow workspace) at the following GitHub directory: https://github.com/ccaudill/GeoserverWorkspace.
The GetFeature request example URL:
http://localhost:8080/geoserver/wfs?service=wfs&version=1.1.0&request=GetFeature&typeName=HeatFlow:HeatFlow
returned the following with schema resolution (as indicated by the schema location):
The document linked to below is meant to assist in simple GeoServer set-up using OpenGeo Suite as well as deployment of NGDS data services using Geoserver, with some additional configuration considerations for use of application schemas in Geoserver. It is assumed that schema-valid data tables already exists in an object-relational database management system (Postgres) and/or an open-source software program adding support to that system (PostGIS). (For more information on schemas, visit http://schemas.usgin.org/models/.)
http://repository.stategeothermaldata.org/resources/documents/NGDS/GeoserverDocumentation.docx
I hope this will provide some background for using GeoServer!
Schema resolution is a thing of beauty.
It must be noted that one important caveat to creating tables in any database structure using ArcMap is that a primary key is not indicated. We found this when the return of a GetFeature request in Geoserver using app-schemas showed that the data was not correctly mapped to the fields (or columns from the PostGIS data table). It appeared that the first column, the OBJECTID, was acutally being mapped as data and therefore put into the first column from the mapping file, which was ObservationURI. This caused more choas; as the schema was being read by Geoserver with a one-field offset, it misinterpreted some fields as not required and completely left them out of the GetFeature request. To fix this, we used PGAdmin to specify the primary key as the objectid in the table. Then, Geoserver was able to acknowledge this and leave that field's integer out of the data when the GetFeature was returned and thus giving a correct return of the data from that PostGIS table.
See the image below for the final GetFeature request example. For those using PGAdmin, specify a primary key by right-clicking the PostGIS database table and selecting "Properties". Under the "Constraints" tab, click the Add button beside "Primary Key". Give a name, such as "bht_p_key" then click the "Columns" tab to choose the appropriate field (here, the objectid). Click OK.
One thing yet to be resolved is our want for the objectid to appear in the gml:id. Doing this in the mapping file requires writing a short CQL statement (see examples in the previous comment), which did not even give us the results we wanted. It turns out that, according to Geoserver developers, "For performance reasons, we strongly recommend avoiding using CQL to generate gml:id in the mapping file as this prevents the use of database indices and the generation of efficient SQL queries."
So for now, we will continue to use the "getId()" expression recommended until a solution to using our original objectid can be found.
Also note that the name of your layer must match exactly the name of the targetElement in your mapping file, which is the element indicated in the schema. Thus, the layer name in Geoserver used for app-schema configuration must match the name specified in your schema document.
Its still in the web archive at
http://web.archive.org/web/20090801194916/http://www.nabble.com/Geoserver-2.0-availability---included-app-schema-capabilities-td24624783.html
It seems that the 'New Mapping File' (above) will allow for schema resolution, which indicates the correct namespace and online schema location in the GetFeature request, along with the features of course. We're not finished yet, though; the idExpression in the mapping file now indicates for Geoserver to give each feature a gml:id, but what we really want is this ID to be indicated by the objectid that already exists in our PostGIS datatables.
The 'working' solution using getId() :
Some of our ideas to use the object id field:
</idExpression>
And we also tried:
<idExpression>
<OCQL>strConcat('BT', strTrim(objectid))</OCQL>
</idExpression>
When entering in the Service Description at the beginning of the Add New Service wizard, make this a robust description as it will give an opportunity for those visiting the service rest page to read about the service properties (software versions), the data specifications (OGC-compliance for AASG services), and description on viewing the WFS using the ArcGIS data interoperability extension.
Additional text appearing on the rest page are culled by ArcGIS from the properties of the layer and map document in the .mxd. Under the layer Properties, be sure to add a description that is specific to that service. For AASG services, this will indicate the state obtained from, the nature of the data, cross-referening fields (i.e., HeaderURI), symbology, and associated metadata or downloadable files (link to the repository page for AASG services). Under Map Document Properties in your .mxd, be sure to indicate the service title, author of the data, keywords, and credits.
Where I'd really like to see this go is the ability to use the commercial search engines (google, yahoo, bing, etc...) to index metadata that we (as data providers) expose using standard metadata formats and present search results in such a way that a search client can find and access data from within a user's application. The search client wouldn't be any more complicated that the standard file open dialogs (with a search box) currently presented by standard GUI interfaces-- maybe with an option to access a map-based interface for geospatial data.
The idea is that from within a web application like USGS the national map , or ArcGIS online, ESRIs ArcMap, MS Excel, ModFlow, GOCAD, or whatever data analysis or map browsing program you're using, you can search for data (pick your search engine --like OpenSearch enables...), get results, look at descriptions of what you found to pick the one you need, and 'get' it. Bingo, you have a new worksheet in your workbook or layer in your map layout.
To do this the search result has to return machine actionable links that client software can parse. The links need to include sufficient information that the client can pick the link that accesses a service it knows how to use and get data in a format it can use. There are a variety of specs out there for smarter links (Web Linking RFC-5988, XLink, ESIP service cast, OGC OWS context documents, ISO19115 CI_OnlineResource, atom:link...), we just need some convergence on what properties need to be associated with a link to make it machine actionable...
Here's a proposal for the properties of interest, based on the specs mentioned above...
href
the link URL-this is what the client will actually ‘get’
title
free text to label link in GUI
type
MIME type of response. Specifies encoding and optionally the native software application environment
rel
URI from IANA rel vocabulary for consistency with IETF5988. Semantics of link from global vocabulary for interoperability. Semantics in this context means calculatable (see discussion in Coyle, 2010 p. 19).
purpose
analogous to ISO19115 CI_OnlineFunctionCode. Controlled vocabulary—tells client why they’d use this link. Purpose property provides mechanism for more granular, application specific indication of link semantics.
protocol
ftp, http, …. Default is http if attribute not specified
serviceType
URI that identifies a service protocol and version (one attribute or two?). This specifies protocols for network layers above http/ftp etc. Should be a URI that can dereference to some kind of service specification document.
outputscheme
profile for content of message retrieve by href URL; URI for xml schema or JSON scheme, other description of data structure and content? Need conventions, clear definition. Clients look at this to pick the link that will get a representation they can use. This is the information scheme in the layers on top of the MIME type encoding; note that the same output scheme might be encoded using different MIME types, so the two are somewhat orthogonal. MIME types have been conflating this property with the lower level encoding (.vnd, +xml … stuff).
I made some minor edits to Ryan's post-- details on the title metadata element, and more about the attributes for 'URL for data access'
The scripts are referenced on the various geoportal pages by adding them to www/catalog/skins/themes/lookAndFeel.jsp
In addition to other new features and stability fixes, the latest release of the CatalogConnctor can now be accessed through OpenSearch.
New release 2010 July 27.
MDweb is a free tool for cataloging and locating resources (data, documents and services) that are accessible over the web. It is based on current geographic-information metadata (ISO 19115, 19119) and communication (OGC’s CSW-2) standards and conforms to the rules for implementing metadata and the associated discovery services of the INSPIRE directives.
This activity is based in France:
Company
IRD
jean-christophe.desconnets@ird.fr
dorian.ginane@ird.fr
Geomatys
vincent.heurteaux@geomatys.fr
Websites
MDweb -- http://www.mdweb-project.org
IRD -- http://www.espace.ird.fr
GEOMATYS -- http://www.geomatys.com
Project is active, developing CSW client and Server. Current release 2.1.0, on 2010 July 30.
http://spatiumcube.sourceforge.net/components.htm
Project based in Spain, University of Zaragoza, associated with CatMDedit
Notes from their web site:
* Easy way of metadata loading. The storage of metadata records is managed directly through the file system.
* The server side of the product supports:
o the compulsory operations GetCapabilities, DescribeRecord, GetRecords and GetRecordById;
o transactional operations to allow users to add, delete and modify the catalog's contents.
* implements ISO19115 and DublinCore output schemas
* Ranked searching. Metadata are indexed based on Lucene search engine http://lucene.apache.org/java/docs/, so best results are returned first.
* Tested compatibility with other catalog tools such as GI-Cat/GI-Go http://zeus.pin.unifi.it/cgi-bin/twiki/view/GIcat/WebHome, Geonetwork http://geonetwork-opensource.org/ or CatalogConnector http://sourceforge.net/projects/catalogconnecto/.
* integrated Web Client application to test catalog services compliant with CSW 2.0.2. This client offers a guided interface to facilitate the definition of CSW requests (including maps and other graphic components for query filter expressions) using both KVP, XML and SOAP protocols. This client interface is multilingual (currently available in Spanish, English, French, Portuguese, Catalan, Galician and the Basque language; it can be customized for other languages) and can be configured to request other catalog services compliant with CSW 2.0.2 developed with different technologies like GI-Cat or GeoNetwork.
The tutorial is deprecated since GeoNetwork has moved to a Maven development environment. See http://trac.osgeo.org/geonetwork/wiki/Maven for the changes.
should there be a browse graphic link in this profile?
There appears to be now a official FGDC CSDGM XML schema location at http://www.fgdc.gov/schemas/metadata/
If you followed the installation instructions for PostgreSQL and PostGIS that I wrote up, the shp2pgsql command is not available globally from the command line. Browse to /opt/postgis/loader before running the command shown above. This is the case for AMI's usgin-2.0 and up.
Apparently, there is a limit to how many files the rm command can handle. It chokes on over 6000 files in Ubunto and throws an "unable to execute /bin/rm: Argument list too long" error. Here is work around:
or if you need root privileges:
The document has been updated to include the changes mentioned below.
Two problems that exist with this setup are the Help page does not load correctly and the map does not display very nicely. I've found the solution to the map problem:
Open web\intermap\WEB-INF\mapServers.xml. Locate the <default name="Layers for default map -- DUMMY NAME:not used"> element. Update it to look like this:
<default name="Layers for default map -- DUMMY NAME:not used">
<server url="http://localhost:8080/geoserver/wms" type="2">
<layer name="gn:world" />
</server>
<server url="http://localhost:8080/geoserver/wms" type="2">
<layer name="gn:gboundaries" />
</server>
</default>
There are a few fixes that I have not yet placed in the PDF:
You need to place the arguments
-XX:MaxPermSize=256m
-XX:PermSize=128m
in your project run configuration in Eclipse to avoid OutOfMemory errors.
Also, to debug your project, you need to add your project in the Source tab within the run configuration for your project. To do so, navigate to Run Configurations or Debug Configurations for your project and then select the Source tab. Next, click Add and then select Java Project and select your project.
Lastly, the Eclipse/Tomcat debugger will not detect any changes to your Java code unless you run build.xml and restart Tomcat. I am not quite sure why this is true, but it is a minor annoyance.
I could not find any authoritative and XML-editor-accessible XSD location for FGDC's fgdc-std-001-1998-ann schema. Until I do, I have added them to the USGIN repository at http://resources.usgin.org/standards/fgdc/ For example http://resources.usgin.org/standards/fgdc/fgdc-std-001-1998-ann/fgdc-std-001-1998-ann.xsd is our interim path to the FGDC-STD-001-1998 schema.
Until some use case(s) for the DCP property can be identified, and a logically coherent codelist that applies to the use cases, it is probably not worth worring about this!
I have copied this post as a discussion paper to the ISO 19139 USGIN Profile section (OGC Schema Issues in Relation to CSW ) because it is referenced in the USGIN Profile document.