lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: RefSorter files in /tmp?
Date Tue, 10 Jul 2012 18:03:22 GMT
Hi,

This is now a serious problem! On the Linux/Windows Jenkins server this
filled the 2 GB temp tmpfs in less than 24hrs without cleaning up. We must
at least in the test that uses this sorter stuff clean up the temp files.
Also RefSporter must mark those tempf files as "delete on JVM exit".

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Uwe Schindler [mailto:uwe@thetaphi.de]
> Sent: Sunday, July 08, 2012 11:08 AM
> To: dev@lucene.apache.org
> Subject: RefSorter files in /tmp?
> 
> When reviewing my Jenkins installation (because slave was taken offline,
> which was config bug), I found out that /tmp is filled by Jenkins with the
> following files:
> 
> -rw-r--r-- 1 jenkins  nogroup     12433 Jul  5 21:14
> RefSorter-1839005885812820606.sorted
> -rw-r--r-- 1 jenkins  nogroup     13574 Jul  5 19:26
> RefSorter-2799526995307200478.sorted
> -rw-r--r-- 1 jenkins  nogroup     12600 Jul  5 17:14
> RefSorter-2841491891429925756.sorted
> -rw-r--r-- 1 jenkins  nogroup     11697 Jul  5 19:57
> RefSorter-3302954184439492426.sorted
> -rw-r--r-- 1 jenkins  nogroup     13496 Jul  5 16:30
> RefSorter-3738422482066276549.sorted
> -rw-r--r-- 1 jenkins  nogroup     13781 Jul  5 15:36
> RefSorter-4235756851148318773.sorted
> -rw-r--r-- 1 jenkins  nogroup     12719 Jul  5 18:54
> RefSorter-4530019493984469514.sorted
> -rw-r--r-- 1 jenkins  nogroup     12696 Jul  5 16:04
> RefSorter-5245195867837976219.sorted
> -rw-r--r-- 1 jenkins  nogroup     13879 Jul  5 18:27
> RefSorter-5977302780601133089.sorted
> -rw-r--r-- 1 jenkins  nogroup     12712 Jul  5 21:39
> RefSorter-6336186633027300753.sorted
> -rw-r--r-- 1 jenkins  nogroup     12820 Jul  5 20:30
> RefSorter-6447286760971372233.sorted
> -rw-r--r-- 1 jenkins  nogroup     12105 Jul  5 17:48
> RefSorter-6532780916605441895.sorted
> -rw-r--r-- 1 jenkins  nogroup     13505 Jul  5 20:53
> RefSorter-7247901917320979657.sorted
> -rw-r--r-- 1 jenkins  nogroup     12688 Jul  5 22:10
> RefSorter-7796370222379929612.sorted
> -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 18:54
> sort1277839437346448611partition
> -rw-r--r-- 1 jenkins  nogroup  21299752 Jul  5 15:35
> sort1362726822297484023intermediate
> -rw-r--r-- 1 jenkins  nogroup  21300496 Jul  5 17:48
> sort1435680293746542872intermediate
> -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 16:30
> sort1498884601796138622partition
> -rw-r--r-- 1 jenkins  nogroup  21300869 Jul  5 20:30
> sort1634015425760928615intermediate
> -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 20:30
> sort1954741677243403383partition
> -rw-r--r-- 1 jenkins  nogroup  21300802 Jul  5 20:53
> sort2203784121687916561intermediate
> -rw-r--r-- 1 jenkins  nogroup  21300493 Jul  5 22:10
> sort24154414907891444intermediate
> -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 22:10
> sort2816986454022083882partition
> -rw-r--r-- 1 jenkins  nogroup  21300111 Jul  5 18:27
> sort285022281545547041intermediate
> -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 18:28
> sort295507558144077223partition
> -rw-r--r-- 1 jenkins  nogroup  21300569 Jul  5 16:30
> sort3013772538520090513intermediate
> -rw-r--r-- 1 jenkins  nogroup  21300574 Jul  5 17:14
> sort3297463807520676013intermediate
> -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 21:14
> sort3364874175018276528partition
> -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 17:14
> sort3846182021346233750partition
> -rw-r--r-- 1 jenkins  nogroup  21300204 Jul  5 19:26
> sort4397860673342757974intermediate
> -rw-r--r-- 1 jenkins  nogroup  21300050 Jul  5 16:04
> sort4474792189525490476intermediate
> -rw-r--r-- 1 jenkins  nogroup  21300825 Jul  5 18:54
> sort4518448528614283778intermediate
> -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 21:39
> sort4756172476965226743partition
> -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 20:53
> sort5416699953867843402partition
> -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 19:26
> sort5558474409634346477partition
> -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 17:48
> sort6281513108922200314partition
> -rw-r--r-- 1 jenkins  nogroup  21300155 Jul  5 21:39
> sort6639309492804635005intermediate
> -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 19:57
> sort6777765458777941142partition
> -rw-r--r-- 1 jenkins  nogroup  21301369 Jul  5 19:57
> sort6973021800616034113intermediate
> -rw-r--r-- 1 jenkins  nogroup  21300341 Jul  5 21:14
> sort7260811068342958052intermediate
> -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 16:04
> sort852078170643422390partition
> -rw-r--r-- 1 jenkins  nogroup        19 Jul  5 15:35
> sort8857722113319559286partition
> 
> The pattern "RefSorter-" I found in Lucene's source code, so it must come
> from tests. Interstingly, all files are from July 5, so not appearing on
> every run. Why are they not cleaned up and why do we need those files?
Would
> a RamDirectory not be enough for this?
> 
> On the Jenkins machine, /tmp is a maximum size 2 GB tmpfs (like on
Solaris),
> so it's limited (previously it was maximum 1 GB and Jenkins took slaves
> offline when size < 1 GB -> fixed).
> 
> # /tmp as 2048 MB tmpfs like on Slowlaris
> /tmp  /tmp  tmpfs  defaults,size=2048m,mode=1777  0  0
> 
> Uwe
> 
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message