hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4189) HADOOP-3245 is incomplete
Date Fri, 19 Sep 2008 04:28:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632506#action_12632506

Amar Kamat commented on HADOOP-4189:

bq. Null config values are not the right way to go. At best it should be "-1" 
I am not sure if -1 internally selects the system defaults. If nothing is passed, by default
the history code selects filesystem's default values for block size and buffer.

bq. In these specific cases I'm not sure we even need to make them configurable.
With the system defaults, the % recovery is very less as history info is not frequently flushed
to the disk. So controlling these parameters, one can control the extent of recovery. Again
there is a tradeoff between how much to recover versus the dfs performance w.r.t low block-size/buffer-size.

bq. Do we need to bump up TaskUmbilicalProtocol's version?
I have added the comment in HADOOP-3245 but forgot to bump up the version number. So fixing
it in this patch.

> HADOOP-3245 is incomplete
> -------------------------
>                 Key: HADOOP-4189
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4189
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Amar Kamat
>            Assignee: Amar Kamat
>            Priority: Blocker
>             Fix For: 0.19.0
>         Attachments: HADOOP-4189-v1.patch, HADOOP-4189-v2.patch, HADOOP-4189-v3.1.patch,
HADOOP-4189-v3.patch, HADOOP-4189.patch
> There are 2 issues with HADOOP-3245
> - The default block size for the history files in hadoop-default.conf is set to 0. This
file will be empty throughout. It should be made null.
> - Same goes for the buffer size. 
> - InterTrackerProtocol version needs to be bumped.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message