hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zheng Shao (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-352) Make Hive support column based storage
Date Thu, 30 Apr 2009 21:51:30 GMT

    [ https://issues.apache.org/jira/browse/HIVE-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12704807#action_12704807
] 

Zheng Shao commented on HIVE-352:
---------------------------------

hive-352-2009-5-1-3.patch

Can you remove the extra message "FileSplit's start is 0, its length is 299"?
Or use LOG.info/LOG.debug.
{code}
hive> select * from zshao_rc;
OK
FileSplit's start is 0, its length is 299
123     456     NULL
Time taken: 0.09 seconds
{code}

Can you find the error messsage in the code, and fix it?
You probably just need to add your ColumnarSerDe to the internal SerDe list.
{code}
hive> alter table zshao_rc replace columns(a int);
Replace columns is not supported for this table. SerDe may be incompatible.
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
{code}

Can you allow extra columns in the metadata? Just assign NULLs to the columns in the metadata
but NOT in the data.
{code}
hive> alter table zshao_rc add columns(a int);
Column 'a' exists
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive> alter table zshao_rc add columns(d int);
hive> select * from zshao_rc;
FileSplit's start is 0, its length is 299
Failed with exception This BytesRefArrayWritable only has 3 valid values.
{code}


> Make Hive support column based storage
> --------------------------------------
>
>                 Key: HIVE-352
>                 URL: https://issues.apache.org/jira/browse/HIVE-352
>             Project: Hadoop Hive
>          Issue Type: New Feature
>            Reporter: He Yongqiang
>            Assignee: He Yongqiang
>         Attachments: 4-22 performace2.txt, 4-22 performance.txt, 4-22 progress.txt, hive-352-2009-4-15.patch,
hive-352-2009-4-16.patch, hive-352-2009-4-17.patch, hive-352-2009-4-19.patch, hive-352-2009-4-22-2.patch,
hive-352-2009-4-22.patch, hive-352-2009-4-23.patch, hive-352-2009-4-27.patch, hive-352-2009-4-30-2.patch,
hive-352-2009-4-30-3.patch, hive-352-2009-4-30-4.patch, hive-352-2009-5-1-3.patch, hive-352-2009-5-1.patch,
HIve-352-draft-2009-03-28.patch, Hive-352-draft-2009-03-30.patch
>
>
> column based storage has been proven a better storage layout for OLAP. 
> Hive does a great job on raw row oriented storage. In this issue, we will enhance hive
to support column based storage. 
> Acctually we have done some work on column based storage on top of hdfs, i think it will
need some review and refactoring to port it to Hive.
> Any thoughts?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message