accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Scott Kuehn (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-488) InputFormats' RecordReaders should call Context.progress
Date Fri, 11 May 2012 08:07:55 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273109#comment-13273109
] 

Scott Kuehn commented on ACCUMULO-488:
--------------------------------------

John, I took another look and confirmed the implicit progress reporting by the task/tasktracker.
I don't think this is an issue for Accumulo. Given your description above ( a single invocation
of the user's map() takes longer than the timeout), best practice is for the user to update
custom counters or invoke context.progress(), so the tasktracker knows their job is progressing.
 

The sample sleepiterator code and MR job in the attached file can demonstrate the timeout
and reset.  You can adjust the params in the job to force a timeout (or lack thereof).
                
> InputFormats' RecordReaders should call Context.progress
> --------------------------------------------------------
>
>                 Key: ACCUMULO-488
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-488
>             Project: Accumulo
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 1.4.0, 1.3.5
>            Reporter: John Vines
>              Labels: newbie
>             Fix For: 1.4.1
>
>         Attachments: Sample-driver-ACCUMULO-488.patch
>
>
> The RecordReader for both input formats never call Context.progress(). This can leave
long-running tasks timing out while still making progress.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message