hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5266) Values Iterator should support "mark" and "reset"
Date Mon, 27 Apr 2009 11:18:30 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703069#action_12703069

Devaraj Das commented on HADOOP-5266:

1) Minor : IFile.Writer has a new constructor. This can directly invoke an existing constructor
with nulls for some arguments.
2) Use a flag to signify EOF has been reached instead of handling EOFException explicitly
in the BackUpStore.hasNext method
3) The casting in Reducer.run is not required for the context argument.
4) Remove the checkBaseIterator calls in MarkableIterator.java
5) MarkableInterface can be package private
6) ReduceContext.nextKeyValue should reuse nextKey/nextVal instead of allocating new DIB objects
7) ReduceContext.resetBackupStore can go away. The caller of that can directly call the resetBackupStore
on the iterator object.
8) ReduceContext.createBackupStore can be moved to the iterator as a method
9) Move the call to Segment.init from BackupStore.FileCache.createInDiskSegment to BackupStore.hasNext
10) Is the inReset check in BackupStore.MemoryCache.reserveSpace(int) ?
11) Improve the testcase by storing the values in an array during the first iteration, and
verifying against the values obtained during the iteration after reset.

> Values Iterator should support "mark" and "reset"
> -------------------------------------------------
>                 Key: HADOOP-5266
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5266
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Jothi Padmanabhan
>            Assignee: Jothi Padmanabhan
>             Fix For: 0.21.0
>         Attachments: hadoop-5266-v1.patch, hadoop-5266-v2.patch
> Some users have expressed interest in having a mark-reset functionality on values iterator.
Users can call mark() at any point during the iteration process and a subsequent reset() should
move the iterator to the last value emitted when mark() was called. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message