hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-611) SequenceFile.Sorter should have a merge method that returns an iterator
Date Wed, 25 Oct 2006 23:07:18 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-611?page=comments#action_12444751 ] 
            
Devaraj Das commented on HADOOP-611:
------------------------------------

A few changes:

The API (1) should be  
RawKeyValueIterator merge (Path[] input)  
//this is to avoid duplicate method names since API (2) also uses the same 
//signature

The other change in API (1) is that it would take one more boolean argument "deleteInputs"
(true means input paths will be deleted when merge has consumed them).

Missed this earlier: In the SegmentDescriptor, a method is provided called "cleanup()". During
the merge process, this method is invoked whenever a segment is done with. One can override
the SegmentDescriptor and define a cleanup that deletes the segment (for e.g., if the segment
in question is the only segment in the segment's pathname).

> SequenceFile.Sorter should have a merge method that returns an iterator
> -----------------------------------------------------------------------
>
>                 Key: HADOOP-611
>                 URL: http://issues.apache.org/jira/browse/HADOOP-611
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: io
>            Reporter: Owen O'Malley
>         Assigned To: Devaraj Das
>             Fix For: 0.8.0
>
>
> SequenceFile.Sorter should get a new merge method that returns an iterator over the keys/values.
> The current merge method should become a simple method that gets the iterator and writes
the records out to a file.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message