reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dhruv Mahajan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (REEF-1361) IFileDeSerializer does not have a Dispose function
Date Thu, 28 Apr 2016 20:00:15 GMT

    [ https://issues.apache.org/jira/browse/REEF-1361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15262860#comment-15262860
] 

Dhruv Mahajan commented on REEF-1361:
-------------------------------------

So these files are the raw files that were downloaded from remote. Now my upstream has a more
efficient format and if the file is in that format the subsequent iterations over data from
disk wud be efficient. So here is what I want to do: In DeSerializer I read these raw files
and convert them in to another temp file in the format needed by upstream. Now when we are
exiting the IInputPartition this temp file needs to be disposed also along with raw files.
Does that make sense?

> IFileDeSerializer does not have a Dispose function
> --------------------------------------------------
>
>                 Key: REEF-1361
>                 URL: https://issues.apache.org/jira/browse/REEF-1361
>             Project: REEF
>          Issue Type: Improvement
>          Components: REEF.NET IO
>         Environment: C#
>            Reporter: Dhruv Mahajan
>
> Currently {{IFileDeserializer}} does not inherit {{IDisposable}} class to dispose unmanaged
resources like files. This becomes necessary in case it decides to add some disk based cache
and writes a cache to the disk at some temporary location. This cache then needs to be deleted
once {{IInputPartition}} is being disposed by calling the {{IFileDeserializer.Dispose()}}
in {{IInputPartition.Dispose()}}.
> The aim of this JIRA is to introduce the dispose functionality in  {{IFileDeserializer}}
and modify {{FileSystemInputPartition}} to use it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message