lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: RefSorter files in /tmp?
Date Sun, 08 Jul 2012 16:29:00 GMT
Yes, ist the offline sorting on disk of Suggester. But:

The code uses File.createTempFile() without any chance to configure it! So we should make
this configureable when initializing the sorter and configure this to local test directory
on tests.

Especially the always failing SolrSuggesterTest fills my RAM-resident /tmp.

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Robert Muir [mailto:rcmuir@gmail.com]
> Sent: Sunday, July 08, 2012 2:07 PM
> To: dev@lucene.apache.org
> Subject: Re: RefSorter files in /tmp?
> 
> when suggester tests fail maybe this happens?
> 
> 
> On Sun, Jul 8, 2012 at 5:07 AM, Uwe Schindler <uwe@thetaphi.de> wrote:
> > When reviewing my Jenkins installation (because slave was taken
> > offline, which was config bug), I found out that /tmp is filled by
> > Jenkins with the following files:
> >
> > -rw-r--r-- 1 jenkins  nogroup     12433 Jul  5 21:14
> > RefSorter-1839005885812820606.sorted
> > -rw-r--r-- 1 jenkins  nogroup     13574 Jul  5 19:26
> > RefSorter-2799526995307200478.sorted
> > -rw-r--r-- 1 jenkins  nogroup     12600 Jul  5 17:14
> > RefSorter-2841491891429925756.sorted
> > -rw-r--r-- 1 jenkins  nogroup     11697 Jul  5 19:57
> > RefSorter-3302954184439492426.sorted
> > -rw-r--r-- 1 jenkins  nogroup     13496 Jul  5 16:30
> > RefSorter-3738422482066276549.sorted
> > -rw-r--r-- 1 jenkins  nogroup     13781 Jul  5 15:36
> > RefSorter-4235756851148318773.sorted
> > -rw-r--r-- 1 jenkins  nogroup     12719 Jul  5 18:54
> > RefSorter-4530019493984469514.sorted
> > -rw-r--r-- 1 jenkins  nogroup     12696 Jul  5 16:04
> > RefSorter-5245195867837976219.sorted
> > -rw-r--r-- 1 jenkins  nogroup     13879 Jul  5 18:27
> > RefSorter-5977302780601133089.sorted
> > -rw-r--r-- 1 jenkins  nogroup     12712 Jul  5 21:39
> > RefSorter-6336186633027300753.sorted
> > -rw-r--r-- 1 jenkins  nogroup     12820 Jul  5 20:30
> > RefSorter-6447286760971372233.sorted
> > -rw-r--r-- 1 jenkins  nogroup     12105 Jul  5 17:48
> > RefSorter-6532780916605441895.sorted
> > -rw-r--r-- 1 jenkins  nogroup     13505 Jul  5 20:53
> > RefSorter-7247901917320979657.sorted
> > -rw-r--r-- 1 jenkins  nogroup     12688 Jul  5 22:10
> > RefSorter-7796370222379929612.sorted
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 18:54
> > sort1277839437346448611partition
> > -rw-r--r-- 1 jenkins  nogroup  21299752 Jul  5 15:35
> > sort1362726822297484023intermediate
> > -rw-r--r-- 1 jenkins  nogroup  21300496 Jul  5 17:48
> > sort1435680293746542872intermediate
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 16:30
> > sort1498884601796138622partition
> > -rw-r--r-- 1 jenkins  nogroup  21300869 Jul  5 20:30
> > sort1634015425760928615intermediate
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 20:30
> > sort1954741677243403383partition
> > -rw-r--r-- 1 jenkins  nogroup  21300802 Jul  5 20:53
> > sort2203784121687916561intermediate
> > -rw-r--r-- 1 jenkins  nogroup  21300493 Jul  5 22:10
> > sort24154414907891444intermediate
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 22:10
> > sort2816986454022083882partition
> > -rw-r--r-- 1 jenkins  nogroup  21300111 Jul  5 18:27
> > sort285022281545547041intermediate
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 18:28
> > sort295507558144077223partition
> > -rw-r--r-- 1 jenkins  nogroup  21300569 Jul  5 16:30
> > sort3013772538520090513intermediate
> > -rw-r--r-- 1 jenkins  nogroup  21300574 Jul  5 17:14
> > sort3297463807520676013intermediate
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 21:14
> > sort3364874175018276528partition
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 17:14
> > sort3846182021346233750partition
> > -rw-r--r-- 1 jenkins  nogroup  21300204 Jul  5 19:26
> > sort4397860673342757974intermediate
> > -rw-r--r-- 1 jenkins  nogroup  21300050 Jul  5 16:04
> > sort4474792189525490476intermediate
> > -rw-r--r-- 1 jenkins  nogroup  21300825 Jul  5 18:54
> > sort4518448528614283778intermediate
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 21:39
> > sort4756172476965226743partition
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 20:53
> > sort5416699953867843402partition
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 19:26
> > sort5558474409634346477partition
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 17:48
> > sort6281513108922200314partition
> > -rw-r--r-- 1 jenkins  nogroup  21300155 Jul  5 21:39
> > sort6639309492804635005intermediate
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 19:57
> > sort6777765458777941142partition
> > -rw-r--r-- 1 jenkins  nogroup  21301369 Jul  5 19:57
> > sort6973021800616034113intermediate
> > -rw-r--r-- 1 jenkins  nogroup  21300341 Jul  5 21:14
> > sort7260811068342958052intermediate
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 16:04
> > sort852078170643422390partition
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 15:35
> > sort8857722113319559286partition
> >
> > The pattern "RefSorter-" I found in Lucene's source code, so it must
> > come from tests. Interstingly, all files are from July 5, so not
> > appearing on every run. Why are they not cleaned up and why do we need
> > those files? Would a RamDirectory not be enough for this?
> >
> > On the Jenkins machine, /tmp is a maximum size 2 GB tmpfs (like on
> > Solaris), so it's limited (previously it was maximum 1 GB and Jenkins
> > took slaves offline when size < 1 GB -> fixed).
> >
> > # /tmp as 2048 MB tmpfs like on Slowlaris /tmp  /tmp  tmpfs
> > defaults,size=2048m,mode=1777  0  0
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: uwe@thetaphi.de
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> > additional commands, e-mail: dev-help@lucene.apache.org
> >
> 
> 
> 
> --
> lucidimagination.com
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message