hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zheng Shao (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-460) Improve ColumnPruner to prune more aggressively and keep column information for input tables
Date Sun, 31 May 2009 02:36:07 GMT

    [ https://issues.apache.org/jira/browse/HIVE-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714807#action_12714807

Zheng Shao commented on HIVE-460:


* HiveStringUtils: I think it's better to use http://java.sun.com/j2se/1.4.2/docs/api/java/lang/Character.html#isLetterOrDigit(char)
* patch line 103: I think we can revert this change 
* patch line 124 & 132: what is the special case for join?
* patch line 334: The replaceAllIllegalCharWithUnderline does not return a unique string:
"a+b" and "a-b" will both become "a_b".
* patch line 362: fields -> fieldNames, also, we need to find a way to get unique names
if there are duplicates.
* patch line 405: we are regenerating the field names here. We should serialize the field
names in the operator desc - this will make sure the way to get unique names in the compilation
time and runtime will be the same.

I think the basic idea is good - we are now using the real expression (or modified real expression
due to illegal chars and duplicates), instead of "0", "1", "2", as the internal names.
In case of "SELECT a + b as c", I think we should use "c" as internal name. In case of "SELECT
a + b", we just use "a_b" as internal name. If it's "SELECT a + b, a - b", then we can use
"a_b", "a_b2". What do you think?

BTW, I think "SELECT kv1.key as key_alias, sum(substr(kv1.value,5)) GROUP BY key_alias LIMIT
5" never worked, so it's not an issue here.

But we definitely need to make sure:
"SELECT field1, field2 FROM (SELECT key + value as field1, key - value as field2 FROM src)

"SELECT * FROM (SELECT key + value, key - value FROM src) t" should also work.

> Improve ColumnPruner to prune more aggressively and keep column information for input
> --------------------------------------------------------------------------------------------
>                 Key: HIVE-460
>                 URL: https://issues.apache.org/jira/browse/HIVE-460
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Query Processor
>    Affects Versions: 0.4.0
>            Reporter: Zheng Shao
>            Assignee: He Yongqiang
>         Attachments: hive-460-2009-05-29-noaliasallowed.patch, HIVE-460.1.patch
> This is required for column-based table format.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message