impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 俊杰陈 <cjjnj...@gmail.com>
Subject Re: Impala Failed to read file from HDFS
Date Fri, 10 Mar 2017 07:25:18 GMT
Hi
Please see the following output. In node bdpe822n2, it worked well. I don't
know why it looks weird today.

[bdpe822n2:21000] > create table test like parquet 'hdfs:///data/1.parquet'
stored as parquet;
Query: create table test like parquet 'hdfs:///data/1.parquet' stored as
parquet

Fetched 0 row(s) in 0.14s
[bdpe822n2:21000] > load data inpath "hdfs:///data/1.parquet" into table
test;
Query: load data inpath "hdfs:///data/1.parquet" into table test
+----------------------------------------------------------+
| summary                                                  |
+----------------------------------------------------------+
| Loaded 1 file(s). Total files in destination location: 2 |
+----------------------------------------------------------+
Fetched 1 row(s) in 3.39s
[bdpe822n2:21000] > refresh test;
Query: refresh test
Query submitted at: 2017-03-10 14:46:54 (Coordinator: http://bdpe822n2:25000
)
Query progress can be monitored at:
http://bdpe822n2:25000/query_plan?query_id=4d4ad8038a0362d3:8c7b326a00000000

Fetched 0 row(s) in 0.09s
[bdpe822n2:21000] > show create table parquet_data.test;
Query: show create table parquet_data.test
ERROR: AnalysisException: Table does not exist: parquet_data.test

[bdpe822n2:21000] > use parquet_data;
Query: use parquet_data
[bdpe822n2:21000] > show tables;
Query: show tables

Fetched 0 row(s) in 0.02s




2017-03-10 15:13 GMT+08:00 Sailesh Mukil <sailesh@cloudera.com>:

> Hi,
>
> Can you do a 'show create table parquet_data.test;'  and paste the output?
>
> On Thu, Mar 9, 2017 at 11:09 PM, 俊杰陈 <cjjnjust@gmail.com> wrote:
>
>> Plus:
>>
>> In my root directory I found user/hive/warehouse/parquet_data.db/test/2.parquet.
>> So it seems impalad is manipulating on local file system.  How do I
>> configure this?
>>
>> 2017-03-10 15:03 GMT+08:00 俊杰陈 <cjjnjust@gmail.com>:
>>
>>> Thanks from quick reply:)
>>>
>>> 1.parquet is always in the hdfs. I also did following command for you
>>> reference, please note the URI which is start with file:. It looks weird.
>>>
>>> [bdpe30-cjj:21000] > use parquet_data;
>>> Query: use parquet_data
>>> [bdpe30-cjj:21000] > load data inpath "hdfs:///data/2.parquet" into
>>> table test;
>>> Query: load data inpath "hdfs:///data/2.parquet" into table test
>>> +----------------------------------------------------------+
>>> | summary                                                  |
>>> +----------------------------------------------------------+
>>> | Loaded 1 file(s). Total files in destination location: 2 |
>>> +----------------------------------------------------------+
>>> Fetched 1 row(s) in 0.50s
>>> [bdpe30-cjj:21000] > select count(*) from test;
>>> Query: select count(*) from test
>>> Query submitted at: 2017-03-10 07:14:45 (Coordinator:
>>> http://bdpe30-cjj:25000)
>>> Query progress can be monitored at: http://bdpe30-cjj:25000/query_
>>> plan?query_id=5d4ecce7d21182cc:e2dd7f5700000000
>>> WARNINGS:
>>> Failed to open HDFS file *file:*/user/hive/warehouse/parq
>>> uet_data.db/test/1.parquet
>>> Error(2): No such file or directory
>>>
>>>
>>> It seems like the load operation read data from hdfs, but not put into
>>> right place for query. Also the impalad seems access the file in local file
>>> system.
>>>
>>>
>>> 2017-03-10 14:48 GMT+08:00 Jeszy <jeszyb@gmail.com>:
>>>
>>>> Hello,
>>>>
>>>> Sounds like Impala expected 1.parquet to be in the folder, but it
>>>> wasn't.
>>>> You probably forgot to do 'refresh <table>' after altering data from
>>>> the outside.
>>>>
>>>> HTH
>>>>
>>>> On Fri, Mar 10, 2017 at 7:30 AM, 俊杰陈 <cjjnjust@gmail.com> wrote:
>>>> > Hi,
>>>> > I'm using latest impala built from github,  and setup impala cluster
>>>> with
>>>> > 2-nodes like below:
>>>> > node-1: statestored, catalogd, namenode,datanode.
>>>> > node-2: impalad, datanode.
>>>> >
>>>> > Then I created database and table, loaded data from external parquet
>>>> file
>>>> > into table. Everything was OK, but when I executed a query it failed
>>>> with
>>>> > following message:
>>>> >
>>>> > Failed to open HDFS file
>>>> > file:/user/hive/warehouse/parquet_data.db/test/1.parquet
>>>> > Error(2): No such file or directory
>>>> >
>>>> > But I can still ‘desc test’. Anyone met with this? Thanks in advanced.
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Thanks & Best Regards
>>>>
>>>
>>>
>>>
>>> --
>>> Thanks & Best Regards
>>>
>>
>>
>>
>> --
>> Thanks & Best Regards
>>
>
>


-- 
Thanks & Best Regards

Mime
View raw message