hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-1511) Hive plan serialization is slow
Date Wed, 04 Sep 2013 18:32:52 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13758131#comment-13758131
] 

Ashutosh Chauhan commented on HIVE-1511:
----------------------------------------

[~brocknoland] I think rewriting setColumnNameList so that it doesn't require RowSchema is
way to go forward. Because, RowSchema is designed to be used at compile time. For execution,
we rely on OIs to carry all the type information. Not needing RowSchema to be serializable
will make compile time and execution time data structures neatly decoupled.
                
> Hive plan serialization is slow
> -------------------------------
>
>                 Key: HIVE-1511
>                 URL: https://issues.apache.org/jira/browse/HIVE-1511
>             Project: Hive
>          Issue Type: Improvement
>    Affects Versions: 0.7.0, 0.11.0
>            Reporter: Ning Zhang
>            Assignee: Mohammad Kamrul Islam
>         Attachments: failedPlan.xml, generated_plan.xml, HIVE-1511.10.patch, HIVE-1511.11.patch,
HIVE-1511.4.patch, HIVE-1511.5.patch, HIVE-1511.6.patch, HIVE-1511.7.patch, HIVE-1511.8.patch,
HIVE-1511.9.patch, HIVE-1511.patch, HIVE-1511-wip2.patch, HIVE-1511-wip3.patch, HIVE-1511-wip4.patch,
HIVE-1511.wip.9.patch, HIVE-1511-wip.patch, KryoHiveTest.java, run.sh
>
>
> As reported by Edward Capriolo:
> For reference I did this as a test case....
> SELECT * FROM src where
> key=0 OR key=0 OR key=0 OR  key=0 OR key=0 OR key=0 OR key=0 OR key=0
> OR key=0 OR key=0 OR key=0 OR
> key=0 OR key=0 OR key=0 OR  key=0 OR key=0 OR key=0 OR key=0 OR key=0
> OR key=0 OR key=0 OR key=0 OR
> ...(100 more of these)
> No OOM but I gave up after the test case did not go anywhere for about
> 2 minutes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message