oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nguyen, Ricky" <rngu...@chla.usc.edu>
Subject Re: How can I update a filemanager's catalogue?
Date Mon, 16 Jul 2012 18:43:23 GMT
Awhile back, I wrote a tiny Java main program to edit a Lucene catalog (attempting to clean
our catalog), but discovered that once you write out v3.x, you can't read it back in with

I believe OODT uses Lucene 2.0 and latest Luke uses 3.x. If Luke has the option to pick your
Lucene format version, you may be good to go.


On Jul 16, 2012, at 11:32 AM, Thomas Bennett wrote:

Hey Sheryl and Cameron,

I have downloaded Luke with the intention of trying it out. Thanks for the warning Sheryl.
I'll give it a try unless Ricky advises otherwise.

Yeah, the files are all okay i.e. have size and content, so I suspect a bug of some sort.

I'm planing to take a look to see how file transfer status is updated. Since I'm using the
ingest with the InPlaceDataTrasfer (I think that's the option for the crawler) it may have
introduces an interesting aspect since the file is not actually transferred :)


On 16 July 2012 17:40, Sheryl John <sheryljj@gmail.com<mailto:sheryljj@gmail.com>>
Hi Cameron,

Were you able to edit the filemgr lucene catalog?
I think if you update an index with Luke it would write out a version that can't be read by
an older Solr/Lucene version.
I remember Ricky trying to edit the filemgr catalog with Luke and faced the above issue.

On Mon, Jul 16, 2012 at 8:09 AM, Cameron Goodale <sigep311@gmail.com<mailto:sigep311@gmail.com>>


You could use lucene's Luke tool to query and edit the catalog. Paul and I discovered a scripting
plugin within Luke that can execute javascript to manipulate the catalog.

I hope that is a good lead for you.


On Jul 16, 2012 8:26 AM, "Thomas Bennett" <lmzxq.tom@gmail.com<mailto:lmzxq.tom@gmail.com>>

I've recently created a Lucene file manager catalog. I cataloged about 2000 files "in place",
but have encountered a problem.

From the fmbrowser some of the files are listed as being:

Product        Transfer Status    Percent Complete Received Time References    Metadata
1287587805.h5 RECEIVED     100%         UNKNOWN        View References     View Metadata
1287353769.h5 RECEIVED    0%                 UNKNOWN        View References     View Metadata
1287355996.h5 RECEIVED    0%                 UNKNOWN        View References     View Metadata
1287425397.h5 RECEIVED     100%         UNKNOWN        View References     View Metadata

The problem lies with the two products that show % complete being 0%.

When I use fmprod to download the product:

wget http://..../fmprod/data?productID=26b9cb0b-c5dd-11e1-bb9f-036596780c6e<http://kat-archive.kat.ac.za:8983/fmprod/data?productID=26b9cb0b-c5dd-11e1-bb9f-036596780c6e>

I get:
HTTP request sent, awaiting response... 500 Internal Server Error
2012-07-14 20:20:38 ERROR 500: Internal Server Error.

So I'd like to "edit" the lucene catalog to update this piece of metadata to reflect that
the % complete is 100. Are there any tools available to do this?

Also at some point I must figure out why its happening :)



CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, 
is for the sole use of the intended recipient(s) and may contain confidential
or legally privileged information. Any unauthorized review, use, disclosure
or distribution is prohibited. If you are not the intended recipient, please
contact the sender by reply e-mail and destroy all copies of this original message.  


View raw message