oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lewis John Mcgibbney <lewis.mcgibb...@gmail.com>
Subject Re: Flexible, use case specific metadata cataloging for CAS
Date Fri, 27 Feb 2015 18:12:22 GMT
BOOM Blast from the past with this thread which is in my inbox.
Hopefully this summer will be the time to pounce on Gora powered OODT
catalogs.
BEWM


On Sun, Mar 24, 2013 at 7:20 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:

> Once I've addressed the non trivial lucene update ill move in to this as I
> am interested in it.
> Thanks for reply.
>
>
> On Sunday, March 24, 2013, Mattmann, Chris A (388J) <
> chris.a.mattmann@jpl.nasa.gov> wrote:
> > Lewis,
> >
> > Since I still have not officially received this email from dev@oodt,
> > (grr) and since I saw it on the mail archives, I'm going to copy your
> > email below, with the same subject, and hope it gets threaded right :)
> >
> > Comments below:
> >
> >
> > On 3/20/13 5:55 PM, Lewis John Mcgibbney (lewi...@gmail.com) wrote:
> >
> >>
> >>All,
> >>
> >>I picked up OODT today and immediately thought about an implementation of
> >>Apache Gora [0] for abstracting persistence within the CAS metadata
> >>catalogue.
> >
> > +1, I've wanted this for a while. A GoraCatalog implementation of the FM
> > Catalog
> > Interface.
> >
> >>Right now, for me, the persistence of my metadata catalogue to Lucene or
> >>MySQL is sufficient and I have no immediate justification for using some
> >>alternative storage mechanism however I noticed that there are a few
> areas
> >>where OODT could generally benefit from the Gora implementation.
> >>It is natural that product discovery via daemon driven CAS crawler (for
> >>example) will fire product streams of varying nature towards the
> catalogue
> >>storage mechanism. Lucene or MySQL my not be best best option to store
> >>such
> >>streams of data and/or the best way to later retrieve that data. Gora
> >>would
> >>enable a much more comprehensive variety of data stores to be available
> >>for
> >>persistence of catalogue metadata and would also provide a much more
> >>flexible model specifically geared towards better solutions for metadata
> >>cataloguing. Currently we support Amazon DynamoDB, Accumulo, Cassandra,
> >>HBase, HDFS, HSQLDB and MySQL. We have patches for Solr, MongoDB and
> >>various file based stores. There is also interest to implement an Oracle
> >>NoSQL DB.... don't ask.
> >
> > Haha!
> >
> >>I notice that the SolrIndexer tool implemented by Paul provides an
> >>expressive number of options for indexing to your Solr HTTP server. The
> >>gora-solr module would provide all these plus more.
> >>I suppose this entirely depends on the requirements for expanding
> metadata
> >>catalogues within the File Manager.
> >>Is it envisaged that such an implementation is required for some use
> cases
> >>or would be required?
> >
> > Yes, please, help! :)
> >
> >>As Gora builds on Hadoop principles, I suppose it would also enable folks
> >>use their metadata catalogues in different, possibly useful, use-case
> >>adaptable ways.
> >>Just an initial thought.
> >
> > A great one at that, I would be super +1 for a GoraCatalog to help in
> these
> > situations and would be keen to work on it with you.
> >
> > Cheers,
> > Chris
> >
> >>Thanks
> >>Lewis
> >>
> >>
> >>[0] http://gora.apache.org <http://gora.apache.org/>
> >>--
> >>*Lewis*
> >>
> >>
> >
> >
> >
> >
>
> --
> *Lewis*
>
>


-- 
*Lewis*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message