hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2282) Local mode needs to work well with block sampling
Date Fri, 15 Jul 2011 17:15:01 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066083#comment-13066083
] 

jiraposter@reviews.apache.org commented on HIVE-2282:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1132/#review1080
-----------------------------------------------------------



ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java
<https://reviews.apache.org/r/1132/#comment2209>

    This function name seems to be confusing. Looks like the input size is set even if there
is no sampling, right? Also, can you add comments to this function?
    
    Other than that, the patch looks OK.


- Siying


On 2011-07-15 02:16:34, Kevin Wilfong wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/1132/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-07-15 02:16:34)
bq.  
bq.  
bq.  Review request for hive and Siying Dong.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  A query should run in local mode when block sampling is used and the sample is small
enough.  The size of the sample is currently being estimated, as it is done to estimate the
number of reducers.
bq.  
bq.  
bq.  This addresses bug HIVE-2282.
bq.      https://issues.apache.org/jira/browse/HIVE-2282
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    ql/src/test/queries/clientpositive/sample_islocalmode_hook.q PRE-CREATION 
bq.    ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java 53769a0 
bq.    ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java cd3de76 
bq.    ql/src/test/org/apache/hadoop/hive/ql/hooks/VerifyIsLocalModeHook.java PRE-CREATION

bq.  
bq.  Diff: https://reviews.apache.org/r/1132/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  TestCliDriver TestNegativeCliDriver, manually tested
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Kevin
bq.  
bq.



> Local mode needs to work well with block sampling
> -------------------------------------------------
>
>                 Key: HIVE-2282
>                 URL: https://issues.apache.org/jira/browse/HIVE-2282
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Siying Dong
>            Assignee: Kevin Wilfong
>         Attachments: HIVE-2282.1.patch.txt
>
>
> Currently, if block sampling is enabled and large set of data are sampled to a small
set, local mode needs to be kicked in. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message