hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kevin Wilfong (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-3707) Round map/reduce progress down when it is in the range [99.5, 100)
Date Tue, 13 Nov 2012 20:02:15 GMT
Kevin Wilfong created HIVE-3707:
-----------------------------------

             Summary: Round map/reduce progress down when it is in the range [99.5, 100)
                 Key: HIVE-3707
                 URL: https://issues.apache.org/jira/browse/HIVE-3707
             Project: Hive
          Issue Type: Improvement
          Components: Logging, Query Processor
    Affects Versions: 0.10.0
            Reporter: Kevin Wilfong
            Assignee: Kevin Wilfong
            Priority: Minor


In HadoopJobExecHelper the mapProgress and reduceProgress are the value of these counters
taken from the running job rounded to an integer percentage.  This means that e.g. if the
mappers are 99.5% done this is stored as 100%.

One of the most common questions I see from new users is, the map and reduce both report being
100% done, why is the query still running?

By rounding down the value in this interval so it's only 100% when it's really 100% we could
avoid that confusion.

Also, the way it appears the QueryPlan and MapRedTask determine if the map/reduce phases are
done is by checking if this value == 100.  I couldn't find anywhere where they're used for
anything significant, but they're reporting early completion.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message