reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julia (JIRA)" <>
Subject [jira] [Created] (REEF-1080) Make local temp file configurable in FileSystemInputPartition
Date Tue, 15 Dec 2015 06:40:46 GMT
Julia created REEF-1080:

             Summary: Make local temp file configurable in FileSystemInputPartition
                 Key: REEF-1080
             Project: REEF
          Issue Type: Bug
            Reporter: Julia

Currently in FileSystemInputPartition, when download remote file to a local file before processing
the data. The local file is based on Path.GetTempPath() which usually is a folder under C
driver. There are multiple issues:
When file size is big, c: driver may not have enough space 
We would like the temp file to be deleted in the end, although we already implemented disposable
to delete the temp file, and client should make sure it is called, there is still possibility
the file is not cleared if the system suddenly crash. 
To resolve above issues, we would like to make the temp file folder for FileSystemInputPartition
configurable, and default to the current working folder so that in case the system is crashed,
the working folder will still be cleared. 

This message was sent by Atlassian JIRA

View raw message