apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Weise <...@apache.org>
Subject Re: Need suggestion about temporary file usage on container node
Date Fri, 17 Mar 2017 14:36:11 GMT
If you use LIB_JARS or FILES, then those files will be localized by YARN on
the container node, you don't need to manually copy them from HDFS or write
cleanup code for it.

Thomas

On Fri, Mar 17, 2017 at 12:49 AM, vikram patil <patilvikram@gmail.com>
wrote:

> Hello All,
>
> I am working on operator which would download the file from HDFS and store
> in on the local machine during run-time.  For storing files to HDFS from
> client machine I will be using LIB_JARS or FILES configuration. Once
> operator fails/shuts down, I would like to clean up these files
> automatically if possible.
>
> Thanks & Regards,
> Vikram
>

Mime
View raw message