oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ross Laidlaw" <rlaidlaw.o...@gmail.com>
Subject Review Request 10096: OODT-611: Implement a JAX-RS service for delivery of file manager products in raw data formats (file, zip)
Date Mon, 08 Jul 2013 05:41:07 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10096/
-----------------------------------------------------------

Review request for oodt and Rishi Verma.


Summary (updated)
-----------------

OODT-611: Implement a JAX-RS service for delivery of file manager products in raw data formats
(file, zip)


Bugs: OODT-611
    https://issues.apache.org/jira/browse/OODT-611


Repository: oodt


Description (updated)
-------

Summary
=======

This patch implements a new JAX-RS service for the CAS-Product web application.  It ports
over functionality from the DataDeliveryServlet and DatasetDeliveryServlet classes (and associated
utility classes) from the oodt.apache.org.cas.product.data package.


URLs
====

Example URLs are:

/reference?productID=123abc&refIndex=0&format=file
/product?productID=123abc&format=zip
/dataset?typeID=urn:oodt:GenericFile&format=zip


For backwards compatibility, I've also included the 'data' URL as an equivalent to the 'product'
URL, for example:

/data?productID=123abc&format=zip


While this project is in progress, I've mapped the JAX-RS servlet to the '/service/*' URL
so the new URLs don't clash with URLs from the original HTTP servlets, which are still active.
 For example, if the web application is deployed to http://localhost:8080/fmprod the full
URLs would be as follows:

http://localhost:8080/fmprod/service/reference?productID=123abc&refIndex=0&format=file
http://localhost:8080/fmprod/service/product?productID=123abc&format=zip
http://localhost:8080/fmprod/service/dataset?typeID=urn:oodt:GenericFile&format=zip


Design
======

I've added a new package: oodt.apache.org.cas.product.service.

Within this package, I've created four sub-packages:

* configurations: (currently empty) will contain classes for reading and processing XML configuration
files (e.g. for RDF, RSS)
* exceptions: custom exception classes that extend javax.ws.rs.WebApplicationException to
return specific HTTP response status codes plus messages
* responders: classes that provide HTTP responses in different formats
* resources: JAX-RS resources representing file manager entities

The above design will hopefully make it fairly straightforward to add new output formats by
adding new responders and configurations.  For example, the aim is to add RdfResponder and
RssResponder classes to the responders package, and RdfConfiguration and RssConfiguration
classes to the configurations package for OODT-612 and OODT-613.


Notes
=====

(Please ignore the first diff (version 1) attached to this review request - it's from an old
issue OODT-470.)

I've moved the original test classes from src/test to src/test/java, and the test resources
(test.logging.properties) from src/testdata to src/test/resources.  This matches the recommended
Maven project structure.  But these details have been omitted from the patch, as Review Board
did not cope well with the details of the moved files - instead it shows the files as having
been deleted.

I added a test 'base' class 'ResourceTestBase' but found that the maven-surefire-plugin tried
to run this as a JUnit test class and complained that there were no tests to run.  As a workaround,
I changed the filename recognition pattern to '/*Test.class' so that only classes ending in
'Test' (rather than having 'Test' anywhere in the class name) would be recognised as JUnit
test classes by the surefire plugin.

For zipping products, I've assumed that if the first reference is a directory, then it's a
hierarchical product and all contents of that directory are included in the zip.  We might
want to do something more sophisticated, such as iterating over the list of references, finding
the root directory and building a temp structure to zip (or process in another format).

Currently there's quite a lot of repetition in the ZipResponder class and some of the test
classes.  I'd like to refactor these after I've implemented the other main tasks for the project:
OODT-612 and OODT-613.

The attached patch also contains a few minor updates to the CAS-Product POM and original servlets
that I spotted while carrying out this work.


Diffs (updated)
-----

  /trunk/webapp/fmprod/pom.xml 1498211 
  /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/data/DataUtils.java 1498211

  /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/exceptions/BadRequestException.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/exceptions/InternalServerErrorException.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/exceptions/NotFoundException.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/resources/DatasetResource.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/resources/ProductResource.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/resources/ReferenceResource.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/resources/TransferResource.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/FileResponder.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/NullResponder.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/Responder.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/ResponderFactory.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/UnrecognizedFormatResponder.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/ZipResponder.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/main/webapp/WEB-INF/web.xml 1498211 
  /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/resources/DatasetResourceTest.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/resources/ProductResourceTest.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/resources/ReferenceResourceTest.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/resources/ResourceTestBase.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/responders/FileResponderTest.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/responders/NullResponderTest.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/responders/ResponderFactoryTest.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/responders/UnrecognizedFormatResponderTest.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/responders/ZipResponderTest.java
PRE-CREATION 
  /trunk/webapp/fmprod/src/test/org/apache/oodt/cas/product/rdf/TestRDFConfigReader.java 1498211

  /trunk/webapp/fmprod/src/test/org/apache/oodt/cas/product/rss/RSSConfigReaderTest.java 1498211

  /trunk/webapp/fmprod/src/test/resources/filemgr/etc/filemgr.properties PRE-CREATION 
  /trunk/webapp/fmprod/src/test/resources/filemgr/etc/logging.properties PRE-CREATION 
  /trunk/webapp/fmprod/src/test/resources/filemgr/etc/mime-types.xml PRE-CREATION 
  /trunk/webapp/fmprod/src/test/resources/filemgr/ingest/flat/location.txt PRE-CREATION 
  /trunk/webapp/fmprod/src/test/resources/filemgr/ingest/flat/location.txt.met PRE-CREATION

  /trunk/webapp/fmprod/src/test/resources/filemgr/ingest/flat/test PRE-CREATION 
  /trunk/webapp/fmprod/src/test/resources/filemgr/ingest/flat/test.txt PRE-CREATION 
  /trunk/webapp/fmprod/src/test/resources/filemgr/ingest/flat/test.txt.met PRE-CREATION 
  /trunk/webapp/fmprod/src/test/resources/filemgr/ingest/hierarchical/test.met PRE-CREATION

  /trunk/webapp/fmprod/src/test/resources/filemgr/ingest/hierarchical/test/file.txt PRE-CREATION

  /trunk/webapp/fmprod/src/test/resources/filemgr/ingest/hierarchical/test/subdirectory/sub-file.txt
PRE-CREATION 
  /trunk/webapp/fmprod/src/test/resources/filemgr/policy/core/elements.xml PRE-CREATION 
  /trunk/webapp/fmprod/src/test/resources/filemgr/policy/core/product-type-element-map.xml
PRE-CREATION 
  /trunk/webapp/fmprod/src/test/resources/filemgr/policy/core/product-types.xml PRE-CREATION

  /trunk/webapp/fmprod/src/testdata/test.logging.properties 1498211 

Diff: https://reviews.apache.org/r/10096/diff/


Testing (updated)
-------

I've implemented several unit tests to cover various scenarios.  While not comprehensive,
these unit tests give some confidence that the basic functionality is working.  I've also
built the web application and deployed it to a Tomcat 7 web server to test it via the command
line (using curl -X GET ...) and with a web browser.


Thanks,

Ross Laidlaw


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message