lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (Commented) (JIRA)" <>
Subject [jira] [Commented] (LUCENE-3728) better handling of files inside/outside CFS by codec
Date Mon, 30 Jan 2012 18:46:10 GMT


Michael McCandless commented on LUCENE-3728:

+1, this is a nice cleanup!  SegmentInfo.files is awesome now :)
> better handling of files inside/outside CFS by codec
> ----------------------------------------------------
>                 Key: LUCENE-3728
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>            Reporter: Robert Muir
>         Attachments: LUCENE-3728.patch
> Since norms and deletes were moved under Codec (LUCENE-3606, LUCENE-3661),
> we never really properly addressed the issue of how Codec.files() should work,
> considering these files are always stored outside of CFS.
> LUCENE-3606 added a hack, LUCENE-3661 cleaned up the hack a little bit more,
> but its still a hack.
> Currently the logic in SegmentInfo.files() is:
> {code}
> clearCache()
> if (compoundFile) {
>   // don't call Codec.files(), hardcoded CFS extensions, etc
> } else {
>   Codec.files()
> }
> // always add files stored outside CFS regardless of CFS setting
> Codec.separateFiles()
> if (sharedDocStores) {
>   // hardcoded shared doc store extensions, etc
> }
> {code}
> Also various codec methods take a Directory parameter, but its inconsistent
> what this Directory is in the case of CFS: for some parts of the index its
> the CFS directory, for others (deletes, separate norms) its not.
> I wonder if instead we could restructure this so that SegmentInfo.files() logic is:
> {code}
> clearCache()
> Codec.files()
> {code}
> and so that Codec is instead responsible.
> instead Codec.files logic by default would do the if (compoundFile) thing, and
> Lucene3x codec itself would only have the if (sharedDocStores) thing, and any
> part of the codec that wants to put stuff always outside of CFS (e.g. Lucene3x separate
norms, deletes) 
> could just use SegmentInfo.dir. Directory parameters in the case of CFS would always
> consistently be the CFSDirectory.
> I haven't totally tested if this will work but there is definitely some cleanups 
> we can do either way, and I think it would be a good step to try to clean this up
> and simplify it.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message