impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 俊杰陈 <cjjnj...@gmail.com>
Subject Re: Impala Failed to read file from HDFS
Date Fri, 10 Mar 2017 07:09:21 GMT
Plus:

In my root directory I found
user/hive/warehouse/parquet_data.db/test/2.parquet.
So it seems impalad is manipulating on local file system.  How do I
configure this?

2017-03-10 15:03 GMT+08:00 俊杰陈 <cjjnjust@gmail.com>:

> Thanks from quick reply:)
>
> 1.parquet is always in the hdfs. I also did following command for you
> reference, please note the URI which is start with file:. It looks weird.
>
> [bdpe30-cjj:21000] > use parquet_data;
> Query: use parquet_data
> [bdpe30-cjj:21000] > load data inpath "hdfs:///data/2.parquet" into table
> test;
> Query: load data inpath "hdfs:///data/2.parquet" into table test
> +----------------------------------------------------------+
> | summary                                                  |
> +----------------------------------------------------------+
> | Loaded 1 file(s). Total files in destination location: 2 |
> +----------------------------------------------------------+
> Fetched 1 row(s) in 0.50s
> [bdpe30-cjj:21000] > select count(*) from test;
> Query: select count(*) from test
> Query submitted at: 2017-03-10 07:14:45 (Coordinator:
> http://bdpe30-cjj:25000)
> Query progress can be monitored at: http://bdpe30-cjj:25000/query_
> plan?query_id=5d4ecce7d21182cc:e2dd7f5700000000
> WARNINGS:
> Failed to open HDFS file *file:*/user/hive/warehouse/
> parquet_data.db/test/1.parquet
> Error(2): No such file or directory
>
>
> It seems like the load operation read data from hdfs, but not put into
> right place for query. Also the impalad seems access the file in local file
> system.
>
>
> 2017-03-10 14:48 GMT+08:00 Jeszy <jeszyb@gmail.com>:
>
>> Hello,
>>
>> Sounds like Impala expected 1.parquet to be in the folder, but it wasn't.
>> You probably forgot to do 'refresh <table>' after altering data from
>> the outside.
>>
>> HTH
>>
>> On Fri, Mar 10, 2017 at 7:30 AM, 俊杰陈 <cjjnjust@gmail.com> wrote:
>> > Hi,
>> > I'm using latest impala built from github,  and setup impala cluster
>> with
>> > 2-nodes like below:
>> > node-1: statestored, catalogd, namenode,datanode.
>> > node-2: impalad, datanode.
>> >
>> > Then I created database and table, loaded data from external parquet
>> file
>> > into table. Everything was OK, but when I executed a query it failed
>> with
>> > following message:
>> >
>> > Failed to open HDFS file
>> > file:/user/hive/warehouse/parquet_data.db/test/1.parquet
>> > Error(2): No such file or directory
>> >
>> > But I can still ‘desc test’. Anyone met with this? Thanks in advanced.
>> >
>> >
>> >
>> > --
>> > Thanks & Best Regards
>>
>
>
>
> --
> Thanks & Best Regards
>



-- 
Thanks & Best Regards

Mime
View raw message