lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Muir <rcm...@gmail.com>
Subject Re: RefSorter files in /tmp?
Date Sun, 08 Jul 2012 16:33:33 GMT
I think there are two problems:

1. Sort allows you to specify a temp dir, but it seems BytesRefSorter
does not respect this?
2. each jvm when running tests should get its own sandboxed tempdir (i
think it should be in the build/ directory with other temp test
stuff).


On Sun, Jul 8, 2012 at 12:29 PM, Uwe Schindler <uwe@thetaphi.de> wrote:
> Yes, ist the offline sorting on disk of Suggester. But:
>
> The code uses File.createTempFile() without any chance to configure it! So we should
make this configureable when initializing the sorter and configure this to local test directory
on tests.
>
> Especially the always failing SolrSuggesterTest fills my RAM-resident /tmp.
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>> -----Original Message-----
>> From: Robert Muir [mailto:rcmuir@gmail.com]
>> Sent: Sunday, July 08, 2012 2:07 PM
>> To: dev@lucene.apache.org
>> Subject: Re: RefSorter files in /tmp?
>>
>> when suggester tests fail maybe this happens?
>>
>>
>> On Sun, Jul 8, 2012 at 5:07 AM, Uwe Schindler <uwe@thetaphi.de> wrote:
>> > When reviewing my Jenkins installation (because slave was taken
>> > offline, which was config bug), I found out that /tmp is filled by
>> > Jenkins with the following files:
>> >
>> > -rw-r--r-- 1 jenkins  nogroup     12433 Jul  5 21:14
>> > RefSorter-1839005885812820606.sorted
>> > -rw-r--r-- 1 jenkins  nogroup     13574 Jul  5 19:26
>> > RefSorter-2799526995307200478.sorted
>> > -rw-r--r-- 1 jenkins  nogroup     12600 Jul  5 17:14
>> > RefSorter-2841491891429925756.sorted
>> > -rw-r--r-- 1 jenkins  nogroup     11697 Jul  5 19:57
>> > RefSorter-3302954184439492426.sorted
>> > -rw-r--r-- 1 jenkins  nogroup     13496 Jul  5 16:30
>> > RefSorter-3738422482066276549.sorted
>> > -rw-r--r-- 1 jenkins  nogroup     13781 Jul  5 15:36
>> > RefSorter-4235756851148318773.sorted
>> > -rw-r--r-- 1 jenkins  nogroup     12719 Jul  5 18:54
>> > RefSorter-4530019493984469514.sorted
>> > -rw-r--r-- 1 jenkins  nogroup     12696 Jul  5 16:04
>> > RefSorter-5245195867837976219.sorted
>> > -rw-r--r-- 1 jenkins  nogroup     13879 Jul  5 18:27
>> > RefSorter-5977302780601133089.sorted
>> > -rw-r--r-- 1 jenkins  nogroup     12712 Jul  5 21:39
>> > RefSorter-6336186633027300753.sorted
>> > -rw-r--r-- 1 jenkins  nogroup     12820 Jul  5 20:30
>> > RefSorter-6447286760971372233.sorted
>> > -rw-r--r-- 1 jenkins  nogroup     12105 Jul  5 17:48
>> > RefSorter-6532780916605441895.sorted
>> > -rw-r--r-- 1 jenkins  nogroup     13505 Jul  5 20:53
>> > RefSorter-7247901917320979657.sorted
>> > -rw-r--r-- 1 jenkins  nogroup     12688 Jul  5 22:10
>> > RefSorter-7796370222379929612.sorted
>> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 18:54
>> > sort1277839437346448611partition
>> > -rw-r--r-- 1 jenkins  nogroup  21299752 Jul  5 15:35
>> > sort1362726822297484023intermediate
>> > -rw-r--r-- 1 jenkins  nogroup  21300496 Jul  5 17:48
>> > sort1435680293746542872intermediate
>> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 16:30
>> > sort1498884601796138622partition
>> > -rw-r--r-- 1 jenkins  nogroup  21300869 Jul  5 20:30
>> > sort1634015425760928615intermediate
>> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 20:30
>> > sort1954741677243403383partition
>> > -rw-r--r-- 1 jenkins  nogroup  21300802 Jul  5 20:53
>> > sort2203784121687916561intermediate
>> > -rw-r--r-- 1 jenkins  nogroup  21300493 Jul  5 22:10
>> > sort24154414907891444intermediate
>> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 22:10
>> > sort2816986454022083882partition
>> > -rw-r--r-- 1 jenkins  nogroup  21300111 Jul  5 18:27
>> > sort285022281545547041intermediate
>> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 18:28
>> > sort295507558144077223partition
>> > -rw-r--r-- 1 jenkins  nogroup  21300569 Jul  5 16:30
>> > sort3013772538520090513intermediate
>> > -rw-r--r-- 1 jenkins  nogroup  21300574 Jul  5 17:14
>> > sort3297463807520676013intermediate
>> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 21:14
>> > sort3364874175018276528partition
>> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 17:14
>> > sort3846182021346233750partition
>> > -rw-r--r-- 1 jenkins  nogroup  21300204 Jul  5 19:26
>> > sort4397860673342757974intermediate
>> > -rw-r--r-- 1 jenkins  nogroup  21300050 Jul  5 16:04
>> > sort4474792189525490476intermediate
>> > -rw-r--r-- 1 jenkins  nogroup  21300825 Jul  5 18:54
>> > sort4518448528614283778intermediate
>> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 21:39
>> > sort4756172476965226743partition
>> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 20:53
>> > sort5416699953867843402partition
>> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 19:26
>> > sort5558474409634346477partition
>> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 17:48
>> > sort6281513108922200314partition
>> > -rw-r--r-- 1 jenkins  nogroup  21300155 Jul  5 21:39
>> > sort6639309492804635005intermediate
>> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 19:57
>> > sort6777765458777941142partition
>> > -rw-r--r-- 1 jenkins  nogroup  21301369 Jul  5 19:57
>> > sort6973021800616034113intermediate
>> > -rw-r--r-- 1 jenkins  nogroup  21300341 Jul  5 21:14
>> > sort7260811068342958052intermediate
>> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 16:04
>> > sort852078170643422390partition
>> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 15:35
>> > sort8857722113319559286partition
>> >
>> > The pattern "RefSorter-" I found in Lucene's source code, so it must
>> > come from tests. Interstingly, all files are from July 5, so not
>> > appearing on every run. Why are they not cleaned up and why do we need
>> > those files? Would a RamDirectory not be enough for this?
>> >
>> > On the Jenkins machine, /tmp is a maximum size 2 GB tmpfs (like on
>> > Solaris), so it's limited (previously it was maximum 1 GB and Jenkins
>> > took slaves offline when size < 1 GB -> fixed).
>> >
>> > # /tmp as 2048 MB tmpfs like on Slowlaris /tmp  /tmp  tmpfs
>> > defaults,size=2048m,mode=1777  0  0
>> >
>> > Uwe
>> >
>> > -----
>> > Uwe Schindler
>> > H.-H.-Meier-Allee 63, D-28213 Bremen
>> > http://www.thetaphi.de
>> > eMail: uwe@thetaphi.de
>> >
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>> > additional commands, e-mail: dev-help@lucene.apache.org
>> >
>>
>>
>>
>> --
>> lucidimagination.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
>> commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>



-- 
lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message