impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sailesh Mukil <sail...@cloudera.com>
Subject Re: Impala Failed to read file from HDFS
Date Fri, 10 Mar 2017 07:13:09 GMT
Hi,

Can you do a 'show create table parquet_data.test;'  and paste the output?

On Thu, Mar 9, 2017 at 11:09 PM, 俊杰陈 <cjjnjust@gmail.com> wrote:

> Plus:
>
> In my root directory I found user/hive/warehouse/parquet_data.db/test/2.parquet.
> So it seems impalad is manipulating on local file system.  How do I
> configure this?
>
> 2017-03-10 15:03 GMT+08:00 俊杰陈 <cjjnjust@gmail.com>:
>
>> Thanks from quick reply:)
>>
>> 1.parquet is always in the hdfs. I also did following command for you
>> reference, please note the URI which is start with file:. It looks weird.
>>
>> [bdpe30-cjj:21000] > use parquet_data;
>> Query: use parquet_data
>> [bdpe30-cjj:21000] > load data inpath "hdfs:///data/2.parquet" into table
>> test;
>> Query: load data inpath "hdfs:///data/2.parquet" into table test
>> +----------------------------------------------------------+
>> | summary                                                  |
>> +----------------------------------------------------------+
>> | Loaded 1 file(s). Total files in destination location: 2 |
>> +----------------------------------------------------------+
>> Fetched 1 row(s) in 0.50s
>> [bdpe30-cjj:21000] > select count(*) from test;
>> Query: select count(*) from test
>> Query submitted at: 2017-03-10 07:14:45 (Coordinator:
>> http://bdpe30-cjj:25000)
>> Query progress can be monitored at: http://bdpe30-cjj:25000/query_
>> plan?query_id=5d4ecce7d21182cc:e2dd7f5700000000
>> WARNINGS:
>> Failed to open HDFS file *file:*/user/hive/warehouse/parq
>> uet_data.db/test/1.parquet
>> Error(2): No such file or directory
>>
>>
>> It seems like the load operation read data from hdfs, but not put into
>> right place for query. Also the impalad seems access the file in local file
>> system.
>>
>>
>> 2017-03-10 14:48 GMT+08:00 Jeszy <jeszyb@gmail.com>:
>>
>>> Hello,
>>>
>>> Sounds like Impala expected 1.parquet to be in the folder, but it wasn't.
>>> You probably forgot to do 'refresh <table>' after altering data from
>>> the outside.
>>>
>>> HTH
>>>
>>> On Fri, Mar 10, 2017 at 7:30 AM, 俊杰陈 <cjjnjust@gmail.com> wrote:
>>> > Hi,
>>> > I'm using latest impala built from github,  and setup impala cluster
>>> with
>>> > 2-nodes like below:
>>> > node-1: statestored, catalogd, namenode,datanode.
>>> > node-2: impalad, datanode.
>>> >
>>> > Then I created database and table, loaded data from external parquet
>>> file
>>> > into table. Everything was OK, but when I executed a query it failed
>>> with
>>> > following message:
>>> >
>>> > Failed to open HDFS file
>>> > file:/user/hive/warehouse/parquet_data.db/test/1.parquet
>>> > Error(2): No such file or directory
>>> >
>>> > But I can still ‘desc test’. Anyone met with this? Thanks in advanced.
>>> >
>>> >
>>> >
>>> > --
>>> > Thanks & Best Regards
>>>
>>
>>
>>
>> --
>> Thanks & Best Regards
>>
>
>
>
> --
> Thanks & Best Regards
>

Mime
View raw message