incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Craig L Russell <craig.russ...@oracle.com>
Subject Re: [Proposal] HotdoG for the Apache Incubator
Date Tue, 28 May 2013 16:12:11 GMT
Hi Paul,

It looks like you have a well-thought-out proposal with a diverse core of developers and mentors.

+1

Good luck.

Craig

On May 21, 2013, at 10:55 AM, Ramirez, Paul M (398J) wrote:

> Dear ASF members,
> 
> We would like to propose HotdoG for the incubator. Paul Ramirez (me) will be the Champion
for this project. The proposal draft is available at:
> 
> https://wiki.apache.org/incubator/HotdoGProposal
> 
> Looking forward to integration your suggestions and feedback.
> 
> 
> = HotdoG Proposal =
> 
> == Abstract ==
> 
> The HotdoG project is an effort and suite of tools to convert HDF/HDF-EOS format into
GeoTIFF format. This is the first of a potential series of incoming projects originating from
The HDF Group -- the non-profit organization is interested in evaluating the ASF as a potential
home for many of its projects. The HDF Group is an independently funded organization that
started many years ago with major investment from NASA as the Hierarchical Data Format (HDF),
version 4 and now version 5, is the ''de facto'' remote sensing data format for NASA missions,
and an increasing number of other disciplines including bio medicine, radio astronomy, climate
science, and other domains.
> 
> HDF is both a data and metadata format, as well as a model for representing and access
information. There are numerous downstream tools that can read and write HDF data, including
a growing number of Geospatial data tools (ESRI-based, and also OpenGeo and other community
led efforts). In addition, major interoperability efforts are also occurring between the remote
sensing community and the climate modeling community (which has traditionally favored NetCDF
as opposed to HDF) because of the efforts in HDF5 to leverage a common data format and model.
> 
> HotdoG is poised to be a first of its kind in the form of bringing one of the major 2
data formats for science to the ASF (the other being the NetCDF format).
> 
> == Proposal ==
> HotdoG is a software converter that converts Earth Science data in HDF/HDF-EOS format
into GeoTIFF format. Doing so easily enables users of remote sensing data to interoperate
with common GIS tools (like WebGIS, Web Processing, image analysis, and geo computational
tools). We feel that the project is an incremental step, and an appropriate focus with tangible
success possibilities by restraining our focus to HDF/HDF-EOS to GeoTIFF conversion.
> 
> There are numerous interesting paths that we can take the toolkit in -- as conversion
from remotely sensed data to GeoTIFF involves ensuring that the HDF-EOS metadata elements
can be appropriately represented using GeoTIFF headers, and the associated format. Furthermore,
capturing the HDF's appropriate geo datum in GeoTIFF will be another important challenge.
> 
> == Background ==
> 
> GeoTIFF is a data and metadata standard for Earth science applications. It is based on
binary Tagged Image File Format (TIFF).  A GeoTIFF file has geographic (or cartographic) data
embedded as tags within the TIFF file that are used to geo-locate the image. This is required
for correct integration of the image in Geographic Information Systems (GIS) and other popular
tools like Google Earth Pro.
> 
> In the recent years, GeoTIFF has gained popularity as a visualization format among NASA
HDF Earth science user communities according to the NASA data user's survey. However, the
conversion from HDF to GeoTIFF is not straightforward for end users because NASA HDF data
products are diverse and organized in many different ways. For example, go to http://hdfeos.org/zoo
and you'll see many scripting language examples because no single script can correctly visualize
all NASA HDF data.
> 
> == Rationale ==
> 
> The HEG tool is limited to some NASA HDF-EOS2 products (no support for HDF-EOS5 products)
and it is not an open-source tool. The latest GDAL (version 1.9.2 and above) is an open-source
tool but it cannot handle many non-HDF-EOS NASA HDF products such as TRMM (pure HDF4) and
Aquarius (pure HDF5) correctly and automatically.
> 
> == Initial Goals ==
> 
> We'll improve GDAL to support NASA HDF products better by handling geo-location information
and physical meaning of data correctly and automatically.
> 
> We'll handle NASA products intelligently so novice users don't have to supply many options
or figure out the details about the data products. For advanced users, we'll give a full control
of accessing HDF products in many different ways so that the converted GeoTIFF file is scientifically
valid and meaningful.
> 
> We aim to provide command line tools first and evolve them into a GUI tool.
> 
> 
> == Current Status ==
> 
> We're looking for developers and sponsors.
> 
> === Meritocracy ===
> We will discuss the milestone and the future plan in an open forum. We plan to encourage
an environment that supports a meritocracy. The contributors will have different privileges
according to their contributions.
> 
> === Community ===
> 
> GIS and Earth Science
> 
> === Core developers ===
> * H. Joe Lee <hyoklee AT hdfgroup DOT org>
> * Denis Nadeau <denis DOT nadeau AT gmail DOT com>
> * Pedro Vicente <pvicente AT uci DOt edu>
> 
> === Alignment ===
> 
> HotdoG employs H4CF Conversion Toolkit for reading data and GDAL for writing GeoTIFF.
In addition, we plan to integrate HotdoG with other products from new missions such as SMAP
and IceSAT-2.
> 
> If HEG becomes available as open source, HotdoG will employ it as well for HDF-EOS2 products.
> 
> == Known Risks ==
> 
> HDF-EOS2 and HDF-EOS5 libraries are distributed under the [[http://opensource.org/licenses/NASA-1.3|NOSA
license]] that may not be compatible with Apache License. This will be vetted with the Legal
Affairs Committee and if found incompatible we will find a different way to read HDF-EOS products.
> 
> HDF is a flexible format which affords for unusual objects such as Point & Zonal
Averages (HDF-EOS), VData (HDF4), and compound datatype (HDF5) that present challenges in
mapping to GeoTIFF. To mitigate this challenge the initial goals will focus on those objects
which are more common place and easily map to GeoTIFF.
> 
> == Documentation ==
> 
> User's guide documentation will be provided via Doxygen. The guide will contain specific
GeoTIFF conversion examples for each NASA HDF data product.
> 
> == External Dependencies ==
> * H4CF Conversion Toolkit
> * GDAL
> * HDF-EOS2 / HDF-EOS5
> * HDF4 / HDF5
> 
> == Required Resources ==
> === Mailing List ===
> * hotdog-private (with moderated subscription)
> * hotdog-dev
> * hotdog-commits
> 
> === Issue Tracking ===
> JIRA HotdoG (HotdoG)
> 
> === Other Resources ===
> * Wiki
>  * http://wiki.apache.org/HotdoG
> 
> == Initial Committers ==
> * H. Joe Lee (The HDF Group)
> * Paul Ramirez (NASA JPL)
> * Chris Mattmann (NASA JPL)
> * Lewis John McGibbney (Stanford University)
> * Denis Nadeau (NASA NCCS)
> * Pedro Vicente (University of California at Irvine)
> * Babak Behzad (University of Illinois at Urbana-Champaign)
> 
> == Affiliations ==
> * H. Joe Lee (The HDF Group)
> * Mike Folk  (The HDF Group)
> * Paul Ramirez (NASA JPL)
> * Chris Mattmann (NASA JPL)
> * Lewis John McGibbney (Stanford University)
> * Denis Nadeau (NASA NCCS)
> * Pedro Vicente (University of California at Irvine)
> * Babak Behzad (University of Illinois at Urbana-Champaign)
> 
> === Champion ===
> * Paul Ramirez <paul DOT m DOT ramirez AT jpl DOT nasa DOT gov>
> 
> === Nominated Mentors ===
> * Chris Mattmann
> * Paul Ramirez
> 
> === Sponsoring Entity ===
> * Apache Incubator
> 
> Sincerely,
> Paul Ramirez
> 

Craig L Russell
Architect, Oracle
http://db.apache.org/jdo
408 276-5638 mailto:Craig.Russell@oracle.com
P.S. A good JDO? O, Gasp!


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message