lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-6813) OfflineSorter.sort isn't thread-safe
Date Thu, 24 Sep 2015 08:46:04 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Michael McCandless updated LUCENE-6813:
---------------------------------------
    Attachment: LUCENE-6813.patch

Patch, I think it's ready.

I added a thread safety test, but it passes even before my fix.  I'll
update the issue title ... sorry for the false blame!  Now I don't
know why BKD was angry before ... but it seems happy now.

I also fixed test infra to set the default temp dir used by
OfflineSorter, so we get MockFS checking, and (YAY!) it caught the
offending place in RangeTreeWriter where it was leaking a file handle!
This should fix the windows failures (finally!).


> OfflineSorter.sort isn't thread-safe
> ------------------------------------
>
>                 Key: LUCENE-6813
>                 URL: https://issues.apache.org/jira/browse/LUCENE-6813
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>             Fix For: Trunk, 5.4
>
>         Attachments: LUCENE-6813.patch, LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe.  It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message