Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id A4923200BC0 for ; Tue, 15 Nov 2016 18:02:49 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id A340D160B03; Tue, 15 Nov 2016 17:02:49 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 9C9FE160AF2 for ; Tue, 15 Nov 2016 18:02:48 +0100 (CET) Received: (qmail 77614 invoked by uid 500); 15 Nov 2016 17:02:47 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 77605 invoked by uid 99); 15 Nov 2016 17:02:47 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 15 Nov 2016 17:02:47 +0000 Received: from mail-it0-f47.google.com (mail-it0-f47.google.com [209.85.214.47]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 63BAC1A03EB for ; Tue, 15 Nov 2016 17:02:47 +0000 (UTC) Received: by mail-it0-f47.google.com with SMTP id o1so9977195ito.1 for ; Tue, 15 Nov 2016 09:02:47 -0800 (PST) X-Gm-Message-State: ABUngvfTWRGZK76ADrbkGy//6HBBnCeYAGxkpVITvAWQ3sF4Ubro5K1O7U6xrn3P2bt2M2HE13xgSYdLlUrB1A== X-Received: by 10.107.11.161 with SMTP id 33mr31229109iol.210.1479229335191; Tue, 15 Nov 2016 09:02:15 -0800 (PST) MIME-Version: 1.0 References: <06b3494a-8deb-c07c-153b-0897abf0a9fa@allsecur.de> In-Reply-To: <06b3494a-8deb-c07c-153b-0897abf0a9fa@allsecur.de> From: Aljoscha Krettek Date: Tue, 15 Nov 2016 17:02:04 +0000 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Lot of RocksDB files in tmp Directory To: user@flink.apache.org Content-Type: multipart/alternative; boundary=001a113e9d6c10eb3a054159eb09 archived-at: Tue, 15 Nov 2016 17:02:49 -0000 --001a113e9d6c10eb3a054159eb09 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Ok, please let me know when you manage to reproduce. Cheers, Aljoscha On Tue, 15 Nov 2016 at 15:00 Dominique Rond=C3=A9 wrote: > Hi Aljoscha, > > i try to reproduce this. The job is started only once and fill my /tmp > Directory within minutes. > > Greets > > Dominique > > Am 14.11.2016 um 16:31 schrieb Aljoscha Krettek: > > Hi, > could it be that the job has restarted due to failures a large number of > times? > > Cheers, > Aljoscha > > On Mon, 7 Nov 2016 at 09:21 Dominique Rond=C3=A9 > wrote: > > First of all, thanks for the explanation. That sounds reasonable. > > But I started the flink routes 3 days ago and went out for the weekend. > Since we are two people with access to flink i guess there is something > strange happen... > > Greets > Dominique > > > > Von meinem Samsung Ger=C3=A4t gesendet. > > > -------- Urspr=C3=BCngliche Nachricht -------- > Von: "xiaogang.sxg" > Datum: 07.11.16 04:00 (GMT+01:00) > An: user@flink.apache.org > Betreff: Re: Lot of RocksDB files in tmp Directory > > RocksDB is orginally implemented in C. To use it in Java, Flink uses a > RocksDB jar with JNI techniques. > > Whenever a Flink job starts, the RocksDB libraries will be loaded. The > file named librocksdb*.so will be extracted from the RocksDB jar and will > be put in the tmp directory. So I think you have run Flink jobs many many > times in the three days to get those 451 files. > > I don=E2=80=99t think the numerous files are a big problem because they a= re > located in the tmp directory and they can be automatically cleaned by OS= . > > =E5=9C=A8 2016=E5=B9=B411=E6=9C=887=E6=97=A5=EF=BC=8C=E4=B8=8A=E5=8D=888:= 24=EF=BC=8CDominique Rond=C3=A9 =E5=86=99=E9= =81=93=EF=BC=9A > > Hi @ll, > > we just change the backend from filesystem to RocksDB. Since that time (3 > Days) we got 451 files with 1.8 GB stored in the tmp-Directory. All files > are named librocksdb*.so > > Did we something wrong or is it a Bug? > > Greets > Dominique > > > > Von meinem Samsung Ger=C3=A4t gesendet. > > > > --001a113e9d6c10eb3a054159eb09 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Ok, please let me know when you manage to reproduce.
<= br>
Cheers,
Aljoscha

On Tue, 15 Nov 2016 at 15:00 Dominique Rond=C3=A9 = <dominique.ronde@allsecur= .de> wrote:
=20 =20 =20

Hi Aljoscha,

i try to reproduce this. The job is started only= once and fill my /tmp Directory within minutes.

Greets

Dominique


Am 14.11= .2016 um 16:31 schrieb Aljoscha Krettek:
Hi,
could it be that the job has restarted due= to failures a large number of times?

Cheers,
Aljoscha

On Mon, 7 Nov 2016 at 09:21 Do= minique Rond=C3=A9 <dominique.ronde@allsecur.de> wrote:
First of all, thanks for the explanation. That sounds reasonable.

But I started the flink routes 3 days ago and went out for the weekend. Since we are two people with access to flink i guess there is something strange happen...

Greets
Dominique=C2=A0



Von meinem Samsung Ger=C3=A4t gesendet.


-------- Urspr=C3=BCngliche Nachricht --------
Von: "xiaogang.sxg" <xiaogang.sxg@aliba= ba-inc.com>
Datum: 07.11.16 04:00 (GMT+01:00)
An: user@flink.apache.org
Betreff: Re: Lot of RocksDB files in tmp Directory

RocksDB is orginally =C2=A0implemented= in C. To use it in Java, Flink uses a RocksDB jar with JNI techniques.=C2=A0

Whenever a Flink job starts, the RocksDB libraries will be loaded. The file named librocksdb*.so will be extracted from the RocksDB jar and will be put in the tmp directory. So I think you have run Flink jobs many many times in the three days to get those 451 files.

I don=E2=80=99t think the numerous fil= es are a big problem because they are located in the tmp directory and =C2=A0they can be automatically cleaned by OS.<= /div>

=E5=9C=A8 2016=E5=B9=B411=E6=9C=88= 7=E6=97=A5=EF=BC=8C=E4=B8=8A=E5=8D=888:24=EF=BC=8CDominique Rond=C3=A9 <dominique.ronde@allsecur.de&= gt; =E5=86=99=E9=81=93=EF=BC=9A

Hi @ll,

we just change the backend from filesystem to RocksDB. Since that time (3 Days) we got 451 files with 1.8 GB stored in the tmp-Directory. All files are named librocksdb*.so

Did we something wrong or is it a Bug?

Greets
Dominique=C2=A0



Von meinem Samsung Ger=C3=A4t gesendet.


--001a113e9d6c10eb3a054159eb09--