hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-4063) Separate spill thresholds for serialization/accounting in MapTask
Date Fri, 05 Sep 2008 21:34:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Chris Douglas updated HADOOP-4063:

    Resolution: Won't Fix
        Status: Resolved  (was: Patch Available)

In explaining how to configure MapTask to a few users, many asked why the spill thresholds
weren't separately configurable. It's easy enough to accommodate the request, but there aren't
many good use cases for it.

Auto-tuning these parameters is awkward, but we'll probably get there eventually.

> Separate spill thresholds for serialization/accounting in MapTask
> -----------------------------------------------------------------
>                 Key: HADOOP-4063
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4063
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>            Priority: Minor
>             Fix For: 0.19.0
>         Attachments: 4063-0.patch, 4063-1.patch
> In MapTask, there is a single parameter controlling the threshold for starting a spill
thread concurrently with collection. However, some users may want to set different thresholds
for the serialization buffer (holding record bytes) and the accounting buffer (holding record

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message