hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gabriel balan <gabriel.ba...@oracle.com>
Subject Re: parque table
Date Fri, 01 May 2015 19:08:01 GMT
Hi

I can't reproduce your problem. Here's what happens on hive-1.1.0-cdh5.4.0:

    hive> LOAD DATA LOCAL INPATH 'access.log.gz' into table rawpartition (fiscal_year=2015,
fiscal_period=1);
    Loading data to table db_apw.raw partition (fiscal_year=2015, fiscal_period=1)
    Partition db_apw.raw{fiscal_year=2015, fiscal_period=1} stats: [numFiles=1, numRows=0,
totalSize=74, rawDataSize=0]
    OK

    hive> select * from raw;
    OK
    106,"2003-02-03",20,2,"A","2","2","037" 2015    1
    106,"2003-02-03",20,3,"A","2","2","037" 2015    1
    106,"2003-02-03",8,2,"A","2","2","037"  2015    1


As already mentioned in this thread, you need to specify the partition clause (in red above),
or you get an error:

    hive> LOAD DATA LOCAL INPATH 'access.log.gz' into table raw;
    FAILED: SemanticException [Error 10062]: Need to specify partition columns because the
destination table is partitioned


hth
Gabriel Balan

On 5/1/2015 11:21 AM, Hadoop User wrote:
> Here is the content of the file once it's unzip
>
> 106,"2003-02-03",20,2,"A","2","2","037"
> 106,"2003-02-03",20,3,"A","2","2","037"
> 106,"2003-02-03",8,2,"A","2","2","037"
>
>
>
>
>
> On May 1, 2015, at 7:32 AM, Asit Parija <asit@sigmoidanalytics.com <mailto:asit@sigmoidanalytics.com>>
wrote:
>
>> Hi Kumar ,
>>   You can remove the stored as text file part and then try that out by default it
should be able to read the .gz files ( if they are comma delimited csv files ) .
>>
>>
>> Thanks
>> Asit
>>
>> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kjayapal17@gmail.com <mailto:kjayapal17@gmail.com>>
wrote:
>>
>>     Hello Nitin,
>>
>>     Dint understand what you mean. Are you telling me to  set COMPRESSION_CODEC=gzip
?
>>
>>     thanks
>>     Jay
>>
>>     On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <nitinpawar432@gmail.com <mailto:nitinpawar432@gmail.com>>
wrote:
>>
>>         You loaded a gz file in a table stored as text file
>>         either define compression format or uncompress the file and load it
>>
>>         On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kjayapal17@gmail.com <mailto:kjayapal17@gmail.com>>
wrote:
>>
>>             Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR
 smallint, FISCAL_PERIOD smallint)
>>             STORED AS TEXTFILE;
>>
>>             and loaded it with data.
>>
>>             |LOAD DATA LOCAL INPATH ||'/tmp/weblogs/20090603-access.log.gz'||INTO
TABLE raw;|
>>             |
>>             |
>>             |I have to load it to parque table|
>>             |
>>             |
>>             |when I say select * from raw it shows all null values.|
>>             |
>>             |
>>             |
>>
>>             	NULL 	NULL 	NULL 	NULL 	NULL 	NULL 	NULL 	NULL
>>
>>             	NULL 	NULL 	NULL 	NULL 	NULL 	NULL 	NULL 	NULL
>>
>>             	NULL 	NULL 	NULL 	NULL 	NULL 	NULL 	NULL 	NULL
>>
>>             	NULL 	NULL 	NULL 	NULL 	NULL 	NULL 	NULL 	NULL
>>
>>             |
>>             Why is not show showing the actual data in file. will it show once I
load it to parque table?
>>
>>             Please let me know if I am doing anything wrong.
>>
>>
>>             Thanks
>>             jay
>>             |
>>             |
>>
>>
>>
>>
>>         -- 
>>         Nitin Pawar
>>
>>
>>

-- 
The statements and opinions expressed here are my own and do not necessarily represent those
of Oracle Corporation.


Mime
View raw message