lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: RefSorter files in /tmp?
Date Tue, 10 Jul 2012 18:08:33 GMT
I checked again: On the windows node C:\Users\JenkinsSlave\AppData\Temp
contained 65000 files with 12 Gigabytes! And I reviewed, they are never
cleaned up, every test run creates new ones!

I'll open issue.

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Uwe Schindler [mailto:uwe@thetaphi.de]
> Sent: Tuesday, July 10, 2012 8:03 PM
> To: dev@lucene.apache.org
> Subject: RE: RefSorter files in /tmp?
> 
> Hi,
> 
> This is now a serious problem! On the Linux/Windows Jenkins server this
filled
> the 2 GB temp tmpfs in less than 24hrs without cleaning up. We must at
least in
> the test that uses this sorter stuff clean up the temp files.
> Also RefSporter must mark those tempf files as "delete on JVM exit".
> 
> Uwe
> 
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
> 
> > -----Original Message-----
> > From: Uwe Schindler [mailto:uwe@thetaphi.de]
> > Sent: Sunday, July 08, 2012 11:08 AM
> > To: dev@lucene.apache.org
> > Subject: RefSorter files in /tmp?
> >
> > When reviewing my Jenkins installation (because slave was taken
> > offline, which was config bug), I found out that /tmp is filled by
> > Jenkins with the following files:
> >
> > -rw-r--r-- 1 jenkins  nogroup     12433 Jul  5 21:14
> > RefSorter-1839005885812820606.sorted
> > -rw-r--r-- 1 jenkins  nogroup     13574 Jul  5 19:26
> > RefSorter-2799526995307200478.sorted
> > -rw-r--r-- 1 jenkins  nogroup     12600 Jul  5 17:14
> > RefSorter-2841491891429925756.sorted
> > -rw-r--r-- 1 jenkins  nogroup     11697 Jul  5 19:57
> > RefSorter-3302954184439492426.sorted
> > -rw-r--r-- 1 jenkins  nogroup     13496 Jul  5 16:30
> > RefSorter-3738422482066276549.sorted
> > -rw-r--r-- 1 jenkins  nogroup     13781 Jul  5 15:36
> > RefSorter-4235756851148318773.sorted
> > -rw-r--r-- 1 jenkins  nogroup     12719 Jul  5 18:54
> > RefSorter-4530019493984469514.sorted
> > -rw-r--r-- 1 jenkins  nogroup     12696 Jul  5 16:04
> > RefSorter-5245195867837976219.sorted
> > -rw-r--r-- 1 jenkins  nogroup     13879 Jul  5 18:27
> > RefSorter-5977302780601133089.sorted
> > -rw-r--r-- 1 jenkins  nogroup     12712 Jul  5 21:39
> > RefSorter-6336186633027300753.sorted
> > -rw-r--r-- 1 jenkins  nogroup     12820 Jul  5 20:30
> > RefSorter-6447286760971372233.sorted
> > -rw-r--r-- 1 jenkins  nogroup     12105 Jul  5 17:48
> > RefSorter-6532780916605441895.sorted
> > -rw-r--r-- 1 jenkins  nogroup     13505 Jul  5 20:53
> > RefSorter-7247901917320979657.sorted
> > -rw-r--r-- 1 jenkins  nogroup     12688 Jul  5 22:10
> > RefSorter-7796370222379929612.sorted
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 18:54
> > sort1277839437346448611partition
> > -rw-r--r-- 1 jenkins  nogroup  21299752 Jul  5 15:35
> > sort1362726822297484023intermediate
> > -rw-r--r-- 1 jenkins  nogroup  21300496 Jul  5 17:48
> > sort1435680293746542872intermediate
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 16:30
> > sort1498884601796138622partition
> > -rw-r--r-- 1 jenkins  nogroup  21300869 Jul  5 20:30
> > sort1634015425760928615intermediate
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 20:30
> > sort1954741677243403383partition
> > -rw-r--r-- 1 jenkins  nogroup  21300802 Jul  5 20:53
> > sort2203784121687916561intermediate
> > -rw-r--r-- 1 jenkins  nogroup  21300493 Jul  5 22:10
> > sort24154414907891444intermediate
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 22:10
> > sort2816986454022083882partition
> > -rw-r--r-- 1 jenkins  nogroup  21300111 Jul  5 18:27
> > sort285022281545547041intermediate
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 18:28
> > sort295507558144077223partition
> > -rw-r--r-- 1 jenkins  nogroup  21300569 Jul  5 16:30
> > sort3013772538520090513intermediate
> > -rw-r--r-- 1 jenkins  nogroup  21300574 Jul  5 17:14
> > sort3297463807520676013intermediate
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 21:14
> > sort3364874175018276528partition
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 17:14
> > sort3846182021346233750partition
> > -rw-r--r-- 1 jenkins  nogroup  21300204 Jul  5 19:26
> > sort4397860673342757974intermediate
> > -rw-r--r-- 1 jenkins  nogroup  21300050 Jul  5 16:04
> > sort4474792189525490476intermediate
> > -rw-r--r-- 1 jenkins  nogroup  21300825 Jul  5 18:54
> > sort4518448528614283778intermediate
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 21:39
> > sort4756172476965226743partition
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 20:53
> > sort5416699953867843402partition
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 19:26
> > sort5558474409634346477partition
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 17:48
> > sort6281513108922200314partition
> > -rw-r--r-- 1 jenkins  nogroup  21300155 Jul  5 21:39
> > sort6639309492804635005intermediate
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 19:57
> > sort6777765458777941142partition
> > -rw-r--r-- 1 jenkins  nogroup  21301369 Jul  5 19:57
> > sort6973021800616034113intermediate
> > -rw-r--r-- 1 jenkins  nogroup  21300341 Jul  5 21:14
> > sort7260811068342958052intermediate
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 16:04
> > sort852078170643422390partition
> > -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 15:35
> > sort8857722113319559286partition
> >
> > The pattern "RefSorter-" I found in Lucene's source code, so it must
> > come from tests. Interstingly, all files are from July 5, so not
> > appearing on every run. Why are they not cleaned up and why do we need
> those files?
> Would
> > a RamDirectory not be enough for this?
> >
> > On the Jenkins machine, /tmp is a maximum size 2 GB tmpfs (like on
> Solaris),
> > so it's limited (previously it was maximum 1 GB and Jenkins took
> > slaves offline when size < 1 GB -> fixed).
> >
> > # /tmp as 2048 MB tmpfs like on Slowlaris /tmp  /tmp  tmpfs
> > defaults,size=2048m,mode=1777  0  0
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: uwe@thetaphi.de
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> > additional commands, e-mail: dev-help@lucene.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message