hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sahil Takiar (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HIVE-16509) Improve HoS user-level explain plan for Local Work and Map Reduce Local
Date Sun, 07 May 2017 20:31:04 GMT

     [ https://issues.apache.org/jira/browse/HIVE-16509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sahil Takiar resolved HIVE-16509.
---------------------------------
    Resolution: Not A Problem

> Improve HoS user-level explain plan for Local Work and Map Reduce Local
> -----------------------------------------------------------------------
>
>                 Key: HIVE-16509
>                 URL: https://issues.apache.org/jira/browse/HIVE-16509
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark
>            Reporter: Sahil Takiar
>            Assignee: Sahil Takiar
>
> The current user-level explain plans (this may affect regular explain plans too, haven't
checked) for HoS create empty entries for {{Local Work}} and {{Map Reduce Local}}
> The sections for local work are largely empty. We should add more info this section (or
remove if it isn't needed).
> Example:
> {code}
> Vertex dependency in root stage
> Reducer 2 <- Map 1 (GROUP)
> Reducer 3 <- Reducer 2 (GROUP)
> Stage-0
>   Fetch Operator
>     limit:-1
>     Stage-1
>       Reducer 3
>       File Output Operator [FS_14]
>         Group By Operator [GBY_12] (rows=1 width=16)
>           Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"]
>         <-Reducer 2 [GROUP]
>           GROUP [RS_11]
>             Group By Operator [GBY_10] (rows=1 width=16)
>               Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"]
>             <-Map 1 [GROUP]
>               GROUP [RS_9]
>                 PartitionCols:rand()
>                 Select Operator [SEL_7] (rows=1 width=33)
>                   Output:["_col0","_col1"]
>                   Map Join Operator [MAPJOIN_15] (rows=1 width=33)
>                     Conds:SEL_1.(UDFToDouble(_col0) + 1.0)=SEL_1.UDFToDouble(_col0)(Left
Outer),Output:["_col0","_col2"]
>                   <-Select Operator [SEL_1] (rows=1 width=30)
>                       Output:["_col0"]
>                       TableScan [TS_0] (rows=1 width=30)
>                         default@t1,k,Tbl:COMPLETE,Col:NONE,Output:["key"]
>               Map Reduce Local Work
>         Stage-2
>           Map 4
>           keys: [HASHTABLESINK_17]
>             0(UDFToDouble(_col0) + 1.0),1UDFToDouble(_col0)
>             Select Operator [SEL_3] (rows=1 width=30)
>               Output:["_col0","_col1"]
>               TableScan [TS_2] (rows=1 width=30)
>                 default@t1,v,Tbl:COMPLETE,Col:NONE,Output:["key","val"]
>           Map Reduce Local Work
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message