flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <trohrm...@apache.org>
Subject Re: about blob.storage.dir and .buffer files
Date Thu, 28 Jan 2016 09:18:03 GMT
Hi Gwenhael,

in theory the blob storage files can be any binary data. At the moment,
this is however only used to distribute the user code jars. The jars are
kept around as long as the job is running. Every
library-cache-manager.cleanup.interval interval the files are checked and
those which are no longer referenced are deleted. In the case of a
termination of Flink all files should be purged. If this is not the case,
then we have to check what the problem is.

Could you check the size of your user jars?

Cheers,
Till
​

On Wed, Jan 27, 2016 at 4:38 PM, Gwenhael Pasquiers <
gwenhael.pasquiers@ericsson.com> wrote:

> Hello,
>
>
>
> We got a question about blob.storage.dir and it’s .buffer files :
>
>
>
> What are they ? And are they cleaned or is there a way to limit their size
> and to evaluate the necessary space ?
>
> We got a node root volume disk filled by those files (~20GB) and it
> crashed.
>
>
>
> Well, the root was filled because we changed the path to /tmp_flink and it
> was on the root filesystem, our bad. We changed it in an urge because the
> default path (/tmp) was being periodically cleaned by the OS and that made
> flink crash.
>
>
>
> B.R.
>
>
>

Mime
View raw message