flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Lot of RocksDB files in tmp Directory
Date Tue, 15 Nov 2016 17:02:04 GMT
Ok, please let me know when you manage to reproduce.

Cheers,
Aljoscha

On Tue, 15 Nov 2016 at 15:00 Dominique Rondé <dominique.ronde@allsecur.de>
wrote:

> Hi Aljoscha,
>
> i try to reproduce this. The job is started only once and fill my /tmp
> Directory within minutes.
>
> Greets
>
> Dominique
>
> Am 14.11.2016 um 16:31 schrieb Aljoscha Krettek:
>
> Hi,
> could it be that the job has restarted due to failures a large number of
> times?
>
> Cheers,
> Aljoscha
>
> On Mon, 7 Nov 2016 at 09:21 Dominique Rondé <dominique.ronde@allsecur.de>
> wrote:
>
> First of all, thanks for the explanation. That sounds reasonable.
>
> But I started the flink routes 3 days ago and went out for the weekend.
> Since we are two people with access to flink i guess there is something
> strange happen...
>
> Greets
> Dominique
>
>
>
> Von meinem Samsung Gerät gesendet.
>
>
> -------- Ursprüngliche Nachricht --------
> Von: "xiaogang.sxg" <xiaogang.sxg@alibaba-inc.com>
> Datum: 07.11.16 04:00 (GMT+01:00)
> An: user@flink.apache.org
> Betreff: Re: Lot of RocksDB files in tmp Directory
>
> RocksDB is orginally  implemented in C. To use it in Java, Flink uses a
> RocksDB jar with JNI techniques.
>
> Whenever a Flink job starts, the RocksDB libraries will be loaded. The
> file named librocksdb*.so will be extracted from the RocksDB jar and will
> be put in the tmp directory. So I think you have run Flink jobs many many
> times in the three days to get those 451 files.
>
> I don’t think the numerous files are a big problem because they are
> located in the tmp directory and  they can be automatically cleaned by OS.
>
> 在 2016年11月7日,上午8:24,Dominique Rondé <dominique.ronde@allsecur.de>
写道:
>
> Hi @ll,
>
> we just change the backend from filesystem to RocksDB. Since that time (3
> Days) we got 451 files with 1.8 GB stored in the tmp-Directory. All files
> are named librocksdb*.so
>
> Did we something wrong or is it a Bug?
>
> Greets
> Dominique
>
>
>
> Von meinem Samsung Gerät gesendet.
>
>
>
>

Mime
View raw message