hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carl Steinbach (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2084) Upgrade datanucleus from 2.0.3 to 2.2.3
Date Thu, 31 Mar 2011 23:42:05 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13014294#comment-13014294
] 

Carl Steinbach commented on HIVE-2084:
--------------------------------------

bq. One change (at line 49) in package.jdo is to fix a bug that was not exposed by the old
datanucleus version. Without the change datanucleus will throw an exception in runtime (FCOMMENT
is not a column of COLUMNS table). I guess the old version of datanucleus didn't check MFieldSchema
mapping in package.jdo, by only retrieving the columns mentioned in the <embedded> elements.

Yup, looks like that's the case. It also looks like Datanucleus was ignoring the size of the
FCOMMENTS field, so the older versions of TYPE_FIELDS.COMMENT and COLUMNS.COMMENT have size
256, which must be the default value. In the new schema these fields both get bumped to 4000
bytes, which is the correct size. Can you please include upgrade scripts that update the size
of these columns accordingly?

Also, as far as I can tell the change to the MOrder mapping has no effect since it is only
referenced by the SORT_COLS table, which overrides the name to COLUMN_NAME instead.

> Upgrade datanucleus from 2.0.3 to 2.2.3
> ---------------------------------------
>
>                 Key: HIVE-2084
>                 URL: https://issues.apache.org/jira/browse/HIVE-2084
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Ning Zhang
>            Assignee: Ning Zhang
>         Attachments: HIVE-2084.patch
>
>
> It seems the datanucleus 2.2.3 does a better join in caching. The time it takes to get
the same set of partition objects takes about 1/4 of the time it took for the first time.
While with 2.0.3, it took almost the same amount of time in the second execution. We should
retest the test case mentioned in HIVE-1853, HIVE-1862.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message