← Back to index

rest - Wildbook Framework for Mark-Recapture Studies

Created: 2015-12-15 16:10  |  Source: http://www.wildme.org/wildbook/doku.php?id=rest

RESTful data access

Wildbook allows other computer programs to push and pull data to and from it through a REspresentational State Transfer (REST) data interface with JavaScript Object Notation (JSON) data. Why is this important? This interface allows your team to build your own, separate applications that push data to Wildbook or pull data from it. For example, you might give a biostatistician access to your Wildbook's REST interface so that they can do custom population modeling in R using your data, pulling the latest data with every run. In another example, you might provide access to your Wildbook's REST interface to a computer vision specialist developing a new, automated individual identification system for your species, ensuring that the specialist always has predictable access to the latest data from your study.

Having a RESTful data interface means that Wildbook can be one of many programs working together to manage and analyze data in your study.

Principles of REST and JSON

The principles of REST are well documented on the web, such as in this article: http://en.wikipedia.org/wiki/Representational_state_transfer

The programming-language independent method of displaying data called JSON is also well documented elsewhere, such as in this article: http://en.wikipedia.org/wiki/JSON

Default Access URL

Wildbook's REST interface by default uses the base URL of:

http:<your_project_URL>/<your_wildbook_version>/rest

For example, if properly configured, your Wildbook will list all of the Encounters submitted within it as a list of JSON objects displayed at the URL:

http:
<your_project_URL>/<your_wildbook_version>/rest/org.ecocean.Encounter

Here is a real world example from the Wildbook demo site: http://www.splashcatalog.org/mmuwildbook/rest/demo/org.ecocean.Encounter

Configuring REST in Wildbook

The data source for Wildbook's REST data interface(s) must be configured independently. If you are only running one project in Wildbook, then this is just a matter of making the database configuration values in your project's jdoconfig.properties file match the vales in jdoconfig.xml. jdoconfig.xml is located in this folder in Wildbook:

<your_wildbook_name>/WEB-INF/classes/META-INF/jdoconfig.xml

For example, if you are using a MySQL database with Wildbook and have the following configuration in jdoconfig.properties:

# The following three lines create a connection to a MySQL database.
# You will also need to add the JDBC JAR file for your MySQL version to /WEB-INF/lib.
datanucleus.ConnectionDriverName=com.mysql.jdbc.Driver
datanucleus.ConnectionURL=jdbc:mysql://localhost:3306/demo
datanucleus.ConnectionUserName=mysqlUsername
datanucleus.ConnectionPassword=mysqlpassword
javax.jdo.PersistenceManagerFactoryClass=org.datanucleus.api.jdo.JDOPersistenceManagerFactory
datanucleus.autoCreateSchema=true
datanucleus.NontransactionalRead=true
datanucleus.Multithreaded=true
datanucleus.RestoreValues=true
datanucleus.storeManagerType=rdbms

Then your jdoconfig.xml would include:

<jdoconfig xmlns="http://java.sun.com/xml/ns/jdo/jdoconfig"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:noNamespaceSchemaLocation="http://java.sun.com/xml/ns/jdo/jdoconfig">

    <!-- Demo PMF -->
    <persistence-manager-factory name="DemoPMF">
		<property name="javax.jdo.PersistenceManagerFactoryClass" value="org.datanucleus.api.jdo.JDOPersistenceManagerFactory"/>
		<property name="datanucleus.ConnectionDriverName" value="com.mysql.jdbc.Driver"/>
		<property name="datanucleus.ConnectionURL" value="jdbc:mysql://localhost:3306/demo"/>
        <property name="datanucleus.ConnectionUserName" value="mysqlUsername"/>       
		<property name="datanucleus.ConnectionPassword" value="mysqlPassword"/>
		<property name="datanucleus.autoCreateSchema" value="true"/>
        <property name="datanucleus.NontransactionalRead" value="true"/>
        <property name="datanucleus.Multithreaded" value="true"/>
        <property name="datanucleus.RestoreValues" value="true"/>
		<property name="datanucleus.storeManagerType" value="rdbms"/>
	</persistence-manager-factory>
</jdoconfig>

Note that if you are running multiple projects out of Wildbook, you can have multiple <persistence-manager-factory> elements defined here, as long as each “name” attribute is unique, allowing each project within Wildbook to have its own REST interface.

The last step is to configure web.xml in Wildbook to expose your data source at a URL:

...
  <servlet>
    <servlet-name>DemoREST</servlet-name>
    <servlet-class>org.datanucleus.api.rest.RestServlet</servlet-class>
    <init-param>
      <param-name>persistence-context</param-name>
      <param-value>DemoPMF</param-value>
    </init-param>
  </servlet>
...

  <servlet-mapping>
    <servlet-name>DemoREST</servlet-name>
    <url-pattern>/rest/demo/*</url-pattern>
  </servlet-mapping>
...
  

The lines above map your data source to the URL pattern /rest/demo/* from which you can access your project's data.

So the basic steps of configuring the REST interface are:

  1. Copy the values for your data source from jdoconfig.properties to their corresponding values in jdoconfig.xml. Remember that you may make multiple data source definitions in jdoconfig.xml if you are running multiple projects out of a single Wildbook. You will need to give each configured data source a name that will be referenced in the next step in web.xml.
  2. Add servlet definitions(s) and mapping(s) for your REST interface(s) to web.xml, using the corresponding names from jdoconfig.xml for your persistence manager factories.
  3. (Re)Start Tomcat.

Examples of REST in Wildbook

Here are examples of accessing RESTful data in Wildbook.

Access the data for a specific Marked Individual

Access the data for a specific Encounter

Here is how you could get the data for Encounter 462 from Wildbook's REST interface in the JSON data format:

http://www.splashcatalog.org/mmuwildbook/rest/demo/org.ecocean.Encounter/562

Access the photos for a specific Encounter

Here is how you could get the metadata for the SinglePhotoVideo objects (its uploaded media) for Encounter 462 from Wildbook's REST interface in the JSON data format:

http://www.splashcatalog.org/mmuwildbook/rest/demo/org.ecocean.SinglePhotoVideo?correspondingEncounterNumber=="562"

Access the media file for a specific SinglePhotoVideo

Here is how you could build a URL to get the actual media file for a SinglePhotoVideo object (its uploaded media) for Encounter 462 from Wildbook's REST interface in the JSON data format.

First get the media SinglePhotoVideo objects:

http://www.splashcatalog.org/mmuwildbook/rest/demo/org.ecocean.SinglePhotoVideo?correspondingEncounterNumber=="562"

Next, parse out its path on the server. For this attribute:

fullFileSystemPath":"/opt/tomcat6/webapps/shepherd_data_dir/encounters/562/KR011404-008.jpg

you can obtain its storage location and its path relative to Tomcat and Tomcat's URLs:

shepherd_data_dir/encounters/562/KR011404-008.jpg

Now you can build a URL to get that image:

http://<your_wildbook_url>/shepherd_data_dir/encounters/562/KR011404-008.jpg

Note the shepherd_data_dir path. This is the directory under Tomcat's webapps directory that is defined in commonConfiguration.properties with the dataDirectoryName property.

For example:

dataDirectoryName = shepherd_data_dir

Example constructed media URL:

http://www.splashcatalog.org/shepherd_data_dir/encounters/562/KR011404-008.jpg

Securing the REST data access point in Wildbook

While REST support allows your wildbook to work with other research applications in your project, it also means that your data is potentially exposed for others to easily take. By default, Wildbook protects the REST URL with this configuration in web.xml:

/rest/** = authcBasicWildbook,roles[rest]

This configuration means two things:

  1. A user attempting to access the REST URL must be authenticated using HTTP BASIC if not already authenticated.
  2. The user that authenticates to the URL must have the “rest” role.

This means that by default your data is protected. To provide another application with access, two things must happen.

  1. You must create a user account in Wildbook and assign it only the “rest” role. This means that the application can access the REST interface but that its credentials cannot be used to log into the user interface of Wildbook.
  2. The application must authenticate to the REST interface with HTTP BASIC credentials in the URL. For example:
http://<username>:<password>@<your_wildbook_url>/rest/org.ecocean.Encounter

More Information

Wildbook uses the Apache Shiro open source project for security. To learn more about Apache Shiro and how you might further modify web.xml for your own customer REST interface, check out this resource:

http://shiro.apache.org/web.html

Background Information about REST Support in Wildbook from DataNucleus

Within Wildbook, data storage is managed by the open source DataNucleus project, which provides the functionality that allows WIldbook to store data in multiple database types. DataNucleus provides the REST data interface used by Wildbook, and more information about it can be found here:

http://www.datanucleus.org/products/accessplatform/rest/api.html