reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Weimer (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (REEF-1080) Make local temp file configurable in FileSystemInputPartition
Date Tue, 15 Dec 2015 16:03:46 GMT

     [ https://issues.apache.org/jira/browse/REEF-1080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Markus Weimer updated REEF-1080:
--------------------------------
    Issue Type: Improvement  (was: Bug)

> Make local temp file configurable in FileSystemInputPartition
> -------------------------------------------------------------
>
>                 Key: REEF-1080
>                 URL: https://issues.apache.org/jira/browse/REEF-1080
>             Project: REEF
>          Issue Type: Improvement
>          Components: REEF.NET
>            Reporter: Julia
>            Assignee: Julia
>
> Currently in FileSystemInputPartition, when download remote file to a local file before
processing the data. The local file is based on Path.GetTempPath() which usually is a folder
under C driver. There are multiple issues:
> When file size is big, c: driver may not have enough space 
> We would like the temp file to be deleted in the end, although we already implemented
disposable to delete the temp file, and client should make sure it is called, there is still
possibility the file is not cleared if the system suddenly crash. 
> To resolve above issues, we would like to make the temp file folder for FileSystemInputPartition
configurable, and default to the current working folder so that in case the system is crashed,
the working folder will still be cleared. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message