oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (388J)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: How can I update a filemanager's catalogue?
Date Thu, 19 Jul 2012 02:51:11 GMT
Hey Ricky,

On Jul 16, 2012, at 1:43 PM, Nguyen, Ricky wrote:

> Awhile back, I wrote a tiny Java main program to edit a Lucene catalog (attempting to
clean our catalog), but discovered that once you write out v3.x, you can't read it back in
with OODT.

Yep that's b/c Apache OODT uses 2.0 as you state below and the *modern* version of Luke uses
3.x. You can find older versions (e.g., 0.6.2 or 0.7) lukeall.jars that are compatible with
older formats, but YMMV.

Thanks!

Cheers,
Chris

P.S. Yes this sucks and we need to upgrade to the newest version of Lucene :) If no one beats
me to this in 0.5 I'll do it myself :)

> 
> On Jul 16, 2012, at 11:32 AM, Thomas Bennett wrote:
> 
>> Hey Sheryl and Cameron,
>> 
>> I have downloaded Luke with the intention of trying it out. Thanks for the warning
Sheryl. I'll give it a try unless Ricky advises otherwise.
>> 
>> Yeah, the files are all okay i.e. have size and content, so I suspect a bug of some
sort.
>> 
>> I'm planing to take a look to see how file transfer status is updated. Since I'm
using the ingest with the InPlaceDataTrasfer (I think that's the option for the crawler) it
may have introduces an interesting aspect since the file is not actually transferred :)
>> 
>> Cheers,
>> Tom
>> 
>> On 16 July 2012 17:40, Sheryl John <sheryljj@gmail.com> wrote:
>> Hi Cameron,
>> 
>> Were you able to edit the filemgr lucene catalog? 
>> I think if you update an index with Luke it would write out a version that can't
be read by an older Solr/Lucene version.
>> I remember Ricky trying to edit the filemgr catalog with Luke and faced the above
issue.
>> 
>> 
>> On Mon, Jul 16, 2012 at 8:09 AM, Cameron Goodale <sigep311@gmail.com> wrote:
>> Tom,
>> 
>> You could use lucene's Luke tool to query and edit the catalog. Paul and I discovered
a scripting plugin within Luke that can execute javascript to manipulate the catalog.
>> 
>> I hope that is a good lead for you.
>> 
>> Cameron
>> 
>> On Jul 16, 2012 8:26 AM, "Thomas Bennett" <lmzxq.tom@gmail.com> wrote:
>> Hi,
>> 
>> I've recently created a Lucene file manager catalog. I cataloged about 2000 files
"in place", but have encountered a problem.
>> 
>> From the fmbrowser some of the files are listed as being:
>> 
>> Product
>>        Transfer Status    Percent Complete
>> Received Time References
>>    Metadata
>> 1287587805.h5
>> RECEIVED     100%
>>         UNKNOWN        View References
>>     View Metadata
>> 1287353769.h5
>> RECEIVED    0%
>>                 UNKNOWN        View References
>>     View Metadata
>> 1287355996.h5
>> RECEIVED    0%
>>                 UNKNOWN        View References
>>     View Metadata
>> 1287425397.h5
>> RECEIVED     100%
>>         UNKNOWN        View References
>>     View Metadata
>> 
>> The problem lies with the two products that show % complete being 0%. 
>> 
>> When I use fmprod to download the product:
>> 
>> wget http://..../fmprod/data?productID=26b9cb0b-c5dd-11e1-bb9f-036596780c6e
>> 
>> I get:
>> HTTP request sent, awaiting response... 500 Internal Server Error
>> 2012-07-14 20:20:38 ERROR 500: Internal Server Error.
>> 
>> So I'd like to "edit" the lucene catalog to update this piece of metadata to reflect
that the % complete is 100. Are there any tools available to do this?
>> 
>> Also at some point I must figure out why its happening :)
>> 
>> Cheers,
>> Tom
>> 
>> 
>> 
>> -- 
>> -Sheryl
>> 
> 
> 
> ---------------------------------------------------------------------
> CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, 
> is for the sole use of the intended recipient(s) and may contain confidential
> or legally privileged information. Any unauthorized review, use, disclosure
> or distribution is prohibited. If you are not the intended recipient, please
> contact the sender by reply e-mail and destroy all copies of this original message. 
> 
> ---------------------------------------------------------------------


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


Mime
View raw message