flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dominique Rondé <dominique.ro...@allsecur.de>
Subject Re: Lot of RocksDB files in tmp Directory
Date Tue, 15 Nov 2016 14:00:22 GMT
Hi Aljoscha,

i try to reproduce this. The job is started only once and fill my /tmp
Directory within minutes.

Greets

Dominique


Am 14.11.2016 um 16:31 schrieb Aljoscha Krettek:
> Hi,
> could it be that the job has restarted due to failures a large number
> of times?
>
> Cheers,
> Aljoscha
>
> On Mon, 7 Nov 2016 at 09:21 Dominique Rondé
> <dominique.ronde@allsecur.de <mailto:dominique.ronde@allsecur.de>> wrote:
>
>     First of all, thanks for the explanation. That sounds reasonable.
>
>     But I started the flink routes 3 days ago and went out for the
>     weekend. Since we are two people with access to flink i guess
>     there is something strange happen...
>
>     Greets
>     Dominique 
>
>
>
>     Von meinem Samsung Gerät gesendet.
>
>
>     -------- Ursprüngliche Nachricht --------
>     Von: "xiaogang.sxg" <xiaogang.sxg@alibaba-inc.com
>     <mailto:xiaogang.sxg@alibaba-inc.com>>
>     Datum: 07.11.16 04:00 (GMT+01:00)
>     An: user@flink.apache.org <mailto:user@flink.apache.org>
>     Betreff: Re: Lot of RocksDB files in tmp Directory
>
>     RocksDB is orginally  implemented in C. To use it in Java, Flink
>     uses a RocksDB jar with JNI techniques. 
>
>     Whenever a Flink job starts, the RocksDB libraries will be loaded.
>     The file named librocksdb*.so will be extracted from the RocksDB
>     jar and will be put in the tmp directory. So I think you have run
>     Flink jobs many many times in the three days to get those 451 files.
>
>     I don’t think the numerous files are a big problem because they
>     are located in the tmp directory and  they can be automatically
>     cleaned by OS.
>
>>     在 2016年11月7日,上午8:24,Dominique Rondé
>>     <dominique.ronde@allsecur.de
>>     <mailto:dominique.ronde@allsecur.de>> 写道:
>>
>>     Hi @ll,
>>
>>     we just change the backend from filesystem to RocksDB. Since that
>>     time (3 Days) we got 451 files with 1.8 GB stored in the
>>     tmp-Directory. All files are named librocksdb*.so
>>
>>     Did we something wrong or is it a Bug?
>>
>>     Greets
>>     Dominique 
>>
>>
>>
>>     Von meinem Samsung Gerät gesendet.
>


Mime
View raw message