Geoserver App-Schema Set-Up: In Progress
One hurdle to using Geoserver to deploy AASG Geothermal Data services is the resolution of schemas. Creating additonal configuration files for Geoserver allows the use of "app-schemas" which will automatically download the indicated web-accessible schema (.xsd) and be referenced in the GetFeature request. This would solve a problem with service deployment using ArcGIS, where custom schema locations is necessary.
This blog will document the problems and break-throughs I've had so far in implementing Geoserver app-schemas. I have not yet been able to get schema resolution, but would like to make a post so that others can take into account my work thus far and chime in about other ideas or advances. This work is based on Tomcat running Geoserver, set up on an Oracle VM VirtualBox. The data for the services are pulled from a PostGIS database.
A few housekeeping tips I learned the hard way, for starters:
catalina.sh
Is Geoserver thinking forever to take you to the Layers page or on a request, or not able to return a request at all? Try editing the catalina.sh file in the Geoserver data directory to indicate that more of Tomcat's memory needs to be allocated to running Geoserver. To this file, add:
JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms512m -Xmx1024m -XX:NewSize=256m -XX:MaxNewSize=256m -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+DisableExplicitGC"
So that a portion of the file will now look like this:
# JSSE_HOME (Optional) May point at your Java Secure Sockets Extension
# (JSSE) installation, whose JAR files will be added to the
# system class path used to start Tomcat.
#
# CATALINA_PID (Optional) Path of the file which should contains the pid
# of catalina startup java process, when start (fork) is used
#
# $Id: catalina.sh 609438 2008-01-06 22:14:28Z markt $
# -----------------------------------------------------------------------------
JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms512m -Xmx1024m -XX:NewSize=256m -XX:MaxNewSize=256m -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+DisableExplicitGC"
# OS specific support. $var _must_ be set to either true or false.
cygwin=false
os400=false
darwin=false
Now that the PermSize is taken care of, on to the business of setting up Geoserver to connect to a PostGIS database. First, it is a good idea to use a data management program to better understand your file structure. I've been using pgAdmin3. When you understand the file structure and extactly where the data tables are in your PostGIS database, you will be better able to input the correct parameters to connect to that database in the Store page on Gerserver.
Set up a Workspace for your a service in Geoserver. NOTE: One limitation of Geoserver app-schemas is that the prefix used by your service must be the same as the workspace name. Factor this is when creating your workspace. Enter your namespace here. Next, create a Store for your data. Input the parameters to connect to your PostGIS database. Go to Layers and click "Add a new resource"; choose the workspace:data store you just created. This should produce a display of all the tables in your chosen PostGIS folder; scroll through until you see the correct table and click "publish".
SRID
I found that the field names from the data showed up on the Layer page, and thus had connected to the database, but GetFeature requests did not return actual data. After some digging, I found that the problem was that the SRID (Spatial Reference System Identifier) was not recognized by Geoserver. This needs to be indicated manually by creating a new SQL view of the layer.
Go to the Layer page and click "Add a new resource". Choose the workspace:datastore you just created. Click "Configure new SQL view...". Choose a name for the new layer (which may be similar to the original layer, but not exactly the same). In the text box, write a SQL statement that will select all the fields from a given table in your PostGIS database, as follows:
select * from azthermalsprings
Click "Refresh". All fields should be added. Scroll down to see that the shape field type is Geometry and manually enter your SRID (i.e., 4326). Save. This is your new layer, which should return actual data from the service in a GetFeature request.
App-Schemas
Very helpful Geoserver documentation: http://docs.geoserver.org/stable/en/user/data/app-schema/index.html
Two files need to be created to configure app-shemas, mapping-file.xml and datastore.xml (I acutally found that using Linux, Geoserver will automatically create this datastore.xml file, it just needs to be edited for app-schema configuration).
This file will contain the database connection information, the web address to the schema (.xsd), and the TypeMapping, which correlates the fields in your layer to those in your schema. Please see these two blogs for FeatureTypeMapping examples: http://lab.usgin.org/node/116 and http://lab.usgin.org/groups/building-geosciml-wfs-server/ncgmp-geosciml-mappedfeature
datastore.xml file: http://docs.geoserver.org/stable/en/user/data/app-schema/data-stores.html#app-schema-data-stores
Also see this blog http://lab.usgin.org/node/117 for an example Geoserver directory structure, which is very important in app-schema configuration. The datastore.xml file points to the file location of the mapping file, and also indicates that an app-schema is to be used. A few things I've learned thus far in an attempt to create the datastore.xml file:
- The datastore.xml file must keep the letter and number string that geoserver gives as its internal id in the datastore id and workspace id fields. (Try to rename this as something human-readable, and Geoserver will not be able to open when restarting.)
- As you remove the type=PostGIS and enabled=true from the datastore.xml, the layer no longer shows as available, and now points to the app-schema. When the type field under dataStore is left as "PostGIS", the store will try to look for kyanite, no matter what else is in the file. Remove type field completely and indicate under connectionParameters dbtype=app-schema and store will look at the app-schema.
- You can EITHER have the parameters point only at the mapping file (with no mention of database) OR have both the url=file path to mapping file as well as kyanite conneciton parameters, and store will still try to look at the mapping file.
The attached Heat Flow Mapping File and datastore file are my attempts thus far at these files. More to come as we work out this issue!
- Login to post comments
Comments
Previous schema resolution
Previous schema resolution had been achieved running GeoServer on a Linux platform, using Oracle VM VirtualBox. The previously attached files were specifically for app-schema set-up running on this platform. Though not a terribly different process, we also wish to create app-schema documentation for running GeoServer on a Windows platform. A popular software package for Windows is OpenGeo Suite (http://opengeo.org/) which includes Postgres and PostGIS (database management systems) and OpenLayers. Using the PostGIS interface, a shapefile can be imported then converted to a table within the PostGIS database. PgAdminIII is also included for managing PostGIS databases and data tables.
Working from either patform, each workspace created in the GeoServer GUI will have its own file in the workspaces folder. Within that file is the store name file, and within that file is the layer name file. The mapping and datastore.xml files must be in the store name folder, as shown in the graphic below.
where:
hf is the name of the Workspace (created in the GeoServer interface)
azheatflow is the name of the Store (created in the GeoServer interface)
HeatFlow is the name of the Layer (SQL view was created in the GeoServer interface)
Do not edit the namespace.xml or workspace.xml files, nor the files that are in the HeatFlow folder (which are the featuretype.xml and layer.xml files). When working on Windows platform, additional files will be added automatically to the workspace folder, called wfs.xml, wms.xml, and settings.xml. These also must remain unedited.
For a complete, working Windows platform directory, see the "HeatFlow" file (named after the original HeatFlow workspace) at the following GitHub directory: https://github.com/ccaudill/GeoserverWorkspace.
The GetFeature request example URL:
http://localhost:8080/geoserver/wfs?service=wfs&version=1.1.0&request=GetFeature&typeName=HeatFlow:HeatFlow
returned the following with schema resolution (as indicated by the schema location):
Additional Geoserver Documentation
The document linked to below is meant to assist in simple GeoServer set-up using OpenGeo Suite as well as deployment of NGDS data services using Geoserver, with some additional configuration considerations for use of application schemas in Geoserver. It is assumed that schema-valid data tables already exists in an object-relational database management system (Postgres) and/or an open-source software program adding support to that system (PostGIS). (For more information on schemas, visit http://schemas.usgin.org/models/.)
http://repository.stategeothermaldata.org/resources/documents/NGDS/GeoserverDocumentation.docx
I hope this will provide some background for using GeoServer!
Schema resolution is a thing
Schema resolution is a thing of beauty.
It must be noted that one important caveat to creating tables in any database structure using ArcMap is that a primary key is not indicated. We found this when the return of a GetFeature request in Geoserver using app-schemas showed that the data was not correctly mapped to the fields (or columns from the PostGIS data table). It appeared that the first column, the OBJECTID, was acutally being mapped as data and therefore put into the first column from the mapping file, which was ObservationURI. This caused more choas; as the schema was being read by Geoserver with a one-field offset, it misinterpreted some fields as not required and completely left them out of the GetFeature request. To fix this, we used PGAdmin to specify the primary key as the objectid in the table. Then, Geoserver was able to acknowledge this and leave that field's integer out of the data when the GetFeature was returned and thus giving a correct return of the data from that PostGIS table.
See the image below for the final GetFeature request example. For those using PGAdmin, specify a primary key by right-clicking the PostGIS database table and selecting "Properties". Under the "Constraints" tab, click the Add button beside "Primary Key". Give a name, such as "bht_p_key" then click the "Columns" tab to choose the appropriate field (here, the objectid). Click OK.
One thing yet to be resolved is our want for the objectid to appear in the gml:id. Doing this in the mapping file requires writing a short CQL statement (see examples in the previous comment), which did not even give us the results we wanted. It turns out that, according to Geoserver developers, "For performance reasons, we strongly recommend avoiding using CQL to generate gml:id in the mapping file as this prevents the use of database indices and the generation of efficient SQL queries."
So for now, we will continue to use the "getId()" expression recommended until a solution to using our original objectid can be found.
Also note that the name of your layer must match exactly the name of the targetElement in your mapping file, which is the element indicated in the schema. Thus, the layer name in Geoserver used for app-schema configuration must match the name specified in your schema document.
App-Schema Resolution!
It seems that the 'New Mapping File' (above) will allow for schema resolution, which indicates the correct namespace and online schema location in the GetFeature request, along with the features of course. We're not finished yet, though; the idExpression in the mapping file now indicates for Geoserver to give each feature a gml:id, but what we really want is this ID to be indicated by the objectid that already exists in our PostGIS datatables.
The 'working' solution using getId() :
Some of our ideas to use the object id field:
</idExpression>
And we also tried:
<idExpression>
<OCQL>strConcat('BT', strTrim(objectid))</OCQL>
</idExpression>