hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-5263) Query Plan cloning time could be improved by using Kryo
Date Wed, 11 Sep 2013 21:38:51 GMT

     [ https://issues.apache.org/jira/browse/HIVE-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Brock Noland updated HIVE-5263:
-------------------------------

    Attachment: HIVE-5263.patch

Great to hear and thank you very much for doing the testing!

My latest patch only copies MapWork.{opParseCtxMap,joinTree}, Operator.colExprMap, and ColumnInfo.objectInspector
when we are cloning the plan (Thx for the tip Mohammad). Therefore in the runtime serialization
case they will not be copied. I think we should commit this since it gives a substantial performance
boost for complex queries and works on java 7. We can do further optimizations in a followup
jira.

https://reviews.apache.org/r/14095/
                
> Query Plan  cloning time could be improved by using Kryo
> --------------------------------------------------------
>
>                 Key: HIVE-5263
>                 URL: https://issues.apache.org/jira/browse/HIVE-5263
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Brock Noland
>            Assignee: Brock Noland
>         Attachments: HIVE-5263.patch, HIVE-5263.patch, HIVE-5263.patch
>
>
> In HIVE-1511 we implemented Kryo for serialization but did not use for query plan cloning.
As was discussed there it's a possible speed improvement. In addition the current XML serialization
method does not work with Java 7 so using Kryo is a workaround for this issue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message