Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 178F0200C44 for ; Mon, 13 Mar 2017 02:49:06 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 161AB160B87; Mon, 13 Mar 2017 01:49:06 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id B856A160B77 for ; Mon, 13 Mar 2017 02:49:04 +0100 (CET) Received: (qmail 54835 invoked by uid 500); 13 Mar 2017 01:49:03 -0000 Mailing-List: contact user-help@impala.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@impala.incubator.apache.org Delivered-To: mailing list user@impala.incubator.apache.org Received: (qmail 54825 invoked by uid 99); 13 Mar 2017 01:49:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 13 Mar 2017 01:49:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 9C78BC0115 for ; Mon, 13 Mar 2017 01:49:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 4.005 X-Spam-Level: **** X-Spam-Status: No, score=4.005 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HK_RANDOM_ENVFROM=0.626, HK_RANDOM_FROM=0.999, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id Xb6BzTFt7QkE for ; Mon, 13 Mar 2017 01:48:58 +0000 (UTC) Received: from mail-oi0-f54.google.com (mail-oi0-f54.google.com [209.85.218.54]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id D5E4F5F39D for ; Mon, 13 Mar 2017 01:48:57 +0000 (UTC) Received: by mail-oi0-f54.google.com with SMTP id 62so72113136oih.2 for ; Sun, 12 Mar 2017 18:48:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=r3HTT2RbVOSq+CgWuHj4XtT78KUryD6BoSXU3iayUIE=; b=kyI4QFdWzBlU6qYnqavkKvjMb1BleQpdrLLvtrKvUq1Ia+RJBaEI+rdMRzmAD1bzDy 61uAJsDaj9pqlCgVnExLOrJJPZuLEQHlvUhQr7Fbg10iM0AErvhovS+2WX7d4SnwJKY5 KWlT2V5lBgSnD6Sp4OzmKpB5p30iVHs2oYYQ96a36IEdjLLE5Kk2Q3wdT3VIBujlwNUA 2C00WluViTL6139NQXVJXQZnXWimpnqvLFngVHEcubuze3phdwhYfRKiXjdD5Ddjn8SS jJPFmIhw9LvZ4tWYv4SmWfrzZT4/+btHIyFWEcBO56Y82Ebd2R/E7e4urUKccVcn93dq HiYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=r3HTT2RbVOSq+CgWuHj4XtT78KUryD6BoSXU3iayUIE=; b=YmLakJNgrkt8W+IGiiR7QJ44FQSVS26ERiE08n/ayFv/DElpUgzascwxZ9yz8VlDN/ FnUlXgOHcgZVZcH6j1wB0bUt5hiZoKN/HYNqM26wKGddlNS/QjEfxqa7HGyx4MPSqXF2 DTEzwQpAGH2wAP7rxZ4QpO4cRD7J7ZzScgfrAFfJEvSNZlWdDpyxh6vIPEDomYLS6YKL q1V5bMZhMN8nuozw5Z19OgEBk2mfEA/aYdsM8qVma/j8ZykNB938eCDgYfoIm6c8XHtV i7aK/T2SlAP6AtKOnQSQSBUdeo6mY8zJP42lBN68vU91Uzz662UHSJL7cP9Vs6OtRcJt K6sQ== X-Gm-Message-State: AMke39kYIcMvEP21ab2MSB8Pvx86qDWv9T7A8GTgujDajTYAeTCaJfUmWZ1XHM2F7JtgZfzPfFiK2oF6KWcUQg== X-Received: by 10.202.5.5 with SMTP id 5mr16668654oif.141.1489369327759; Sun, 12 Mar 2017 18:42:07 -0700 (PDT) MIME-Version: 1.0 Received: by 10.74.14.73 with HTTP; Sun, 12 Mar 2017 18:42:07 -0700 (PDT) In-Reply-To: References: From: =?UTF-8?B?5L+K5p2w6ZmI?= Date: Mon, 13 Mar 2017 09:42:07 +0800 Message-ID: Subject: Re: Impala Failed to read file from HDFS To: user@impala.incubator.apache.org Content-Type: multipart/alternative; boundary=94eb2c18d080b875df054a92d108 archived-at: Mon, 13 Mar 2017 01:49:06 -0000 --94eb2c18d080b875df054a92d108 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Please see following: [bdpe30-cjj:21000] > create table test2 like parquet 'hdfs:///data/2.parquet' stored as parquet; Query: create table test2 like parquet 'hdfs:///data/2.parquet' stored as parquet Fetched 0 row(s) in 0.21s [bdpe30-cjj:21000] > show create table test2; Query: show create table test2 +------------------------------------------------------------+ | result | +------------------------------------------------------------+ | CREATE TABLE parquet_data.test2 ( | | a STRING COMMENT 'Inferred from Parquet file.', | | b STRING COMMENT 'Inferred from Parquet file.', | | c BIGINT COMMENT 'Inferred from Parquet file.', | | d INT COMMENT 'Inferred from Parquet file.', | | e INT COMMENT 'Inferred from Parquet file.', | | f BIGINT COMMENT 'Inferred from Parquet file.', | | g INT COMMENT 'Inferred from Parquet file.', | | aa STRING COMMENT 'Inferred from Parquet file.', | | bb BIGINT COMMENT 'Inferred from Parquet file.', | | cc INT COMMENT 'Inferred from Parquet file.', | | dd STRING COMMENT 'Inferred from Parquet file.', | | ee BIGINT COMMENT 'Inferred from Parquet file.', | | ff BIGINT COMMENT 'Inferred from Parquet file.', | | gg BIGINT COMMENT 'Inferred from Parquet file.', | | h BIGINT COMMENT 'Inferred from Parquet file.', | | i STRING COMMENT 'Inferred from Parquet file.', | | j STRING COMMENT 'Inferred from Parquet file.', | | k INT COMMENT 'Inferred from Parquet file.', | | l STRING COMMENT 'Inferred from Parquet file.', | | m STRING COMMENT 'Inferred from Parquet file.', | | n STRING COMMENT 'Inferred from Parquet file.', | | hh STRING COMMENT 'Inferred from Parquet file.', | | ii STRING COMMENT 'Inferred from Parquet file.', | | jj INT COMMENT 'Inferred from Parquet file.', | | kk INT COMMENT 'Inferred from Parquet file.', | | ll BIGINT COMMENT 'Inferred from Parquet file.', | | mm INT COMMENT 'Inferred from Parquet file.', | | nn BIGINT COMMENT 'Inferred from Parquet file.', | | o STRING COMMENT 'Inferred from Parquet file.', | | p BIGINT COMMENT 'Inferred from Parquet file.', | | q BIGINT COMMENT 'Inferred from Parquet file.', | | r BIGINT COMMENT 'Inferred from Parquet file.', | | s INT COMMENT 'Inferred from Parquet file.', | | t INT COMMENT 'Inferred from Parquet file.', | | u INT COMMENT 'Inferred from Parquet file.', | | v INT COMMENT 'Inferred from Parquet file.', | | w INT COMMENT 'Inferred from Parquet file.', | | oo INT COMMENT 'Inferred from Parquet file.', | | pp BIGINT COMMENT 'Inferred from Parquet file.', | | qq INT COMMENT 'Inferred from Parquet file.', | | rr BIGINT COMMENT 'Inferred from Parquet file.', | | ss INT COMMENT 'Inferred from Parquet file.', | | tt BIGINT COMMENT 'Inferred from Parquet file.', | | u1 INT COMMENT 'Inferred from Parquet file.', | | v1 INT COMMENT 'Inferred from Parquet file.', | | w1 INT COMMENT 'Inferred from Parquet file.', | | x STRING COMMENT 'Inferred from Parquet file.', | | y INT COMMENT 'Inferred from Parquet file.', | | z INT COMMENT 'Inferred from Parquet file.', | | uu STRING COMMENT 'Inferred from Parquet file.', | | vv STRING COMMENT 'Inferred from Parquet file.', | | ww INT COMMENT 'Inferred from Parquet file.', | | xx INT COMMENT 'Inferred from Parquet file.', | | yy BIGINT COMMENT 'Inferred from Parquet file.', | | zz BIGINT COMMENT 'Inferred from Parquet file.', | | aaa STRING COMMENT 'Inferred from Parquet file.', | | bbb STRING COMMENT 'Inferred from Parquet file.', | | ccc STRING COMMENT 'Inferred from Parquet file.' | | ) | | STORED AS PARQUET | | LOCATION 'file:/user/hive/warehouse/parquet_data.db/test2' | | | +------------------------------------------------------------+ Fetched 1 row(s) in 3.22s [bdpe30-cjj:21000] > select * from test2 limit 5; Query: select * from test2 limit 5 Query submitted at: 2017-03-13 02:05:40 (Coordinator: http://bdpe30-cjj:25000) Query progress can be monitored at: http://bdpe30-cjj:25000/query_plan?query_id=3De444cb64be71c69:96546e2600000= 000 Fetched 0 row(s) in 0.28s 2017-03-10 17:12 GMT+08:00 Jeszy : > The above looks like you accidentally created the table in a different > database - can you repro the 'file:/' error and paste 'show create > table' of that table? > > On Fri, Mar 10, 2017 at 8:25 AM, =E4=BF=8A=E6=9D=B0=E9=99=88 wrote: > > Hi > > Please see the following output. In node bdpe822n2, it worked well. I > don't > > know why it looks weird today. > > > > [bdpe822n2:21000] > create table test like parquet > 'hdfs:///data/1.parquet' > > stored as parquet; > > Query: create table test like parquet 'hdfs:///data/1.parquet' stored a= s > > parquet > > > > Fetched 0 row(s) in 0.14s > > [bdpe822n2:21000] > load data inpath "hdfs:///data/1.parquet" into tabl= e > > test; > > Query: load data inpath "hdfs:///data/1.parquet" into table test > > +----------------------------------------------------------+ > > | summary | > > +----------------------------------------------------------+ > > | Loaded 1 file(s). Total files in destination location: 2 | > > +----------------------------------------------------------+ > > Fetched 1 row(s) in 3.39s > > [bdpe822n2:21000] > refresh test; > > Query: refresh test > > Query submitted at: 2017-03-10 14:46:54 (Coordinator: > > http://bdpe822n2:25000) > > Query progress can be monitored at: > > http://bdpe822n2:25000/query_plan?query_id=3D4d4ad8038a0362d3: > 8c7b326a00000000 > > > > Fetched 0 row(s) in 0.09s > > [bdpe822n2:21000] > show create table parquet_data.test; > > Query: show create table parquet_data.test > > ERROR: AnalysisException: Table does not exist: parquet_data.test > > > > [bdpe822n2:21000] > use parquet_data; > > Query: use parquet_data > > [bdpe822n2:21000] > show tables; > > Query: show tables > > > > Fetched 0 row(s) in 0.02s > > > > > > > > > > 2017-03-10 15:13 GMT+08:00 Sailesh Mukil : > >> > >> Hi, > >> > >> Can you do a 'show create table parquet_data.test;' and paste the > output? > >> > >> On Thu, Mar 9, 2017 at 11:09 PM, =E4=BF=8A=E6=9D=B0=E9=99=88 wrote: > >>> > >>> Plus: > >>> > >>> In my root directory I found > >>> user/hive/warehouse/parquet_data.db/test/2.parquet. So it seems > impalad is > >>> manipulating on local file system. How do I configure this? > >>> > >>> 2017-03-10 15:03 GMT+08:00 =E4=BF=8A=E6=9D=B0=E9=99=88 : > >>>> > >>>> Thanks from quick reply:) > >>>> > >>>> 1.parquet is always in the hdfs. I also did following command for yo= u > >>>> reference, please note the URI which is start with file:. It looks > weird. > >>>> > >>>> [bdpe30-cjj:21000] > use parquet_data; > >>>> Query: use parquet_data > >>>> [bdpe30-cjj:21000] > load data inpath "hdfs:///data/2.parquet" into > >>>> table test; > >>>> Query: load data inpath "hdfs:///data/2.parquet" into table test > >>>> +----------------------------------------------------------+ > >>>> | summary | > >>>> +----------------------------------------------------------+ > >>>> | Loaded 1 file(s). Total files in destination location: 2 | > >>>> +----------------------------------------------------------+ > >>>> Fetched 1 row(s) in 0.50s > >>>> [bdpe30-cjj:21000] > select count(*) from test; > >>>> Query: select count(*) from test > >>>> Query submitted at: 2017-03-10 07:14:45 (Coordinator: > >>>> http://bdpe30-cjj:25000) > >>>> Query progress can be monitored at: > >>>> http://bdpe30-cjj:25000/query_plan?query_id=3D5d4ecce7d21182cc: > e2dd7f5700000000 > >>>> WARNINGS: > >>>> Failed to open HDFS file > >>>> file:/user/hive/warehouse/parquet_data.db/test/1.parquet > >>>> Error(2): No such file or directory > >>>> > >>>> > >>>> It seems like the load operation read data from hdfs, but not put in= to > >>>> right place for query. Also the impalad seems access the file in > local file > >>>> system. > >>>> > >>>> > >>>> 2017-03-10 14:48 GMT+08:00 Jeszy : > >>>>> > >>>>> Hello, > >>>>> > >>>>> Sounds like Impala expected 1.parquet to be in the folder, but it > >>>>> wasn't. > >>>>> You probably forgot to do 'refresh ' after altering data fro= m > >>>>> the outside. > >>>>> > >>>>> HTH > >>>>> > >>>>> On Fri, Mar 10, 2017 at 7:30 AM, =E4=BF=8A=E6=9D=B0=E9=99=88 wrote: > >>>>> > Hi, > >>>>> > I'm using latest impala built from github, and setup impala > cluster > >>>>> > with > >>>>> > 2-nodes like below: > >>>>> > node-1: statestored, catalogd, namenode,datanode. > >>>>> > node-2: impalad, datanode. > >>>>> > > >>>>> > Then I created database and table, loaded data from external > parquet > >>>>> > file > >>>>> > into table. Everything was OK, but when I executed a query it > failed > >>>>> > with > >>>>> > following message: > >>>>> > > >>>>> > Failed to open HDFS file > >>>>> > file:/user/hive/warehouse/parquet_data.db/test/1.parquet > >>>>> > Error(2): No such file or directory > >>>>> > > >>>>> > But I can still =E2=80=98desc test=E2=80=99. Anyone met with this= ? Thanks in > >>>>> > advanced. > >>>>> > > >>>>> > > >>>>> > > >>>>> > -- > >>>>> > Thanks & Best Regards > >>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Thanks & Best Regards > >>> > >>> > >>> > >>> > >>> -- > >>> Thanks & Best Regards > >> > >> > > > > > > > > -- > > Thanks & Best Regards > --=20 Thanks & Best Regards --94eb2c18d080b875df054a92d108 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi=C2=A0
Please see following:=C2=A0
[b= dpe30-cjj:21000] > create table test2 like parquet 'hdfs:///data/2.p= arquet' stored as parquet;
Query: create table test2 like par= quet 'hdfs:///data/2.parquet' stored as parquet

Fetched 0 row(s) in 0.21s
[bdpe30-cjj:21000] > show crea= te table test2;
Query: show create table test2
+-------= -----------------------------------------------------+
| result = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
+---------------------------= ---------------------------------+
| CREATE TABLE parquet_data.te= st2 ( =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 a STRING COMMENT 'Inferred fr= om Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2= =A0 b STRING COMMENT 'Inferred from Parquet file.', =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 c BIGINT COMMENT 'Inferred fro= m Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2= =A0 d INT COMMENT 'Inferred from Parquet file.', =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2=A0 e INT COMMENT 'Inferred f= rom Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2=A0 f BIGINT COMMENT 'Inferred from Parquet file.', =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 g INT COMMENT 'Inferred= from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
=
| =C2=A0 aa STRING COMMENT 'Inferred from Parquet file.', =C2= =A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2=A0 bb BIGINT COMMENT 'Inferr= ed from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2= =A0 cc INT COMMENT 'Inferred from Parquet file.', =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 dd STRING COMMENT 'Inferre= d from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2=A0= ee BIGINT COMMENT 'Inferred from Parquet file.', =C2=A0 =C2=A0 =C2= =A0 =C2=A0 |
| =C2=A0 ff BIGINT COMMENT 'Inferred from Parque= t file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2=A0 gg BIGINT CO= MMENT 'Inferred from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 |<= /div>
| =C2=A0 h BIGINT COMMENT 'Inferred from Parquet file.', = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 i STRING COMMENT = 9;Inferred from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 j STRING COMMENT 'Inferred from Parquet file.', =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 k INT COMMENT 'Infe= rred from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2=A0 l STRING COMMENT 'Inferred from Parquet file.', = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 m STRING COMMENT = 9;Inferred from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 n STRING COMMENT 'Inferred from Parquet file.', =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 hh STRING COMMENT '= Inferred from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
|= =C2=A0 ii STRING COMMENT 'Inferred from Parquet file.', =C2=A0 =C2= =A0 =C2=A0 =C2=A0 |
| =C2=A0 jj INT COMMENT 'Inferred from Pa= rquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| = =C2=A0 kk INT COMMENT 'Inferred from Parquet file.', =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 ll BIGINT COMMENT 'Infe= rred from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2= =A0 mm INT COMMENT 'Inferred from Parquet file.', =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 nn BIGINT COMMENT 'Inferre= d from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2=A0= o STRING COMMENT 'Inferred from Parquet file.', =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0|
| =C2=A0 p BIGINT COMMENT 'Inferred from P= arquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 q= BIGINT COMMENT 'Inferred from Parquet file.', =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0|
| =C2=A0 r BIGINT COMMENT 'Inferred from Parq= uet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 s IN= T COMMENT 'Inferred from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 |
| =C2=A0 t INT COMMENT 'Inferred from Par= quet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| = =C2=A0 u INT COMMENT 'Inferred from Parquet file.', =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2=A0 v INT COMMENT 'Inferre= d from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2=A0 w INT COMMENT 'Inferred from Parquet file.', =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2=A0 oo INT COMMENT '= ;Inferred from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0|
| =C2=A0 pp BIGINT COMMENT 'Inferred from Parquet file.&= #39;, =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2=A0 qq INT COMMENT '= Inferred from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= |
| =C2=A0 rr BIGINT COMMENT 'Inferred from Parquet file.'= ;, =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2=A0 ss INT COMMENT 'Inf= erred from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 tt BIGINT COMMENT 'Inferred from Parquet file.', = =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2=A0 u1 INT COMMENT 'Inferr= ed from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 v1 INT COMMENT 'Inferred from Parquet file.', =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 w1 INT COMMENT '= ;Inferred from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0|
| =C2=A0 x STRING COMMENT 'Inferred from Parquet file.&#= 39;, =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 y INT COMMENT &= #39;Inferred from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 |
| =C2=A0 z INT COMMENT 'Inferred from Parquet file.&= #39;, =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2=A0 uu STR= ING COMMENT 'Inferred from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2= =A0 |
| =C2=A0 vv STRING COMMENT 'Inferred from Parquet file.= ', =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2=A0 ww INT COMMENT '= ;Inferred from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0|
| =C2=A0 xx INT COMMENT 'Inferred from Parquet file.'= ;, =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 yy BIGINT = COMMENT 'Inferred from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0 = |
| =C2=A0 zz BIGINT COMMENT 'Inferred from Parquet file.'= ;, =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| =C2=A0 aaa STRING COMMENT '= ;Inferred from Parquet file.', =C2=A0 =C2=A0 =C2=A0 =C2=A0|
|= =C2=A0 bbb STRING COMMENT 'Inferred from Parquet file.', =C2=A0 = =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 ccc STRING COMMENT 'Inferred f= rom Parquet file.' =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
| ) =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| STORED AS PAR= QUET =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0|
| LOCATION 'file:/user/hive/warehouse/parquet_data.db/te= st2' |
| =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0|
+----------------------------------------------------= --------+
Fetched 1 row(s) in 3.22s
[bdpe30-cjj:21000] = > select * from test2 limit 5;
Query: select * from test2 limi= t 5
Query submitted at: 2017-03-13 02:05:40 (Coordinator: http://bdpe30-cjj:25000)

Fetched 0 row(s) in 0.28s


2017-03-10 17:12 GMT+08:00 Jesz= y <jeszyb@gmail.com>:
The a= bove looks like you accidentally created the table in a different
database - can you repro the 'file:/' error and paste 'show cre= ate
table' of that table?

On Fri, Mar 10, 2017 at 8:25 AM, =E4=BF=8A=E6=9D=B0=E9=99=88 <cjjnjust@gmail.com> wrote:
> Hi
> Please see the following output. In node bdpe822n2, it worked well. I = don't
> know why it looks weird today.
>
> [bdpe822n2:21000] > create table test like parquet 'hdfs:///dat= a/1.parquet'
> stored as parquet;
> Query: create table test like parquet 'hdfs:///data/1.parquet'= stored as
> parquet
>
> Fetched 0 row(s) in 0.14s
> [bdpe822n2:21000] > load data inpath "hdfs:///data/1.parquet&q= uot; into table
> test;
> Query: load data inpath "hdfs:///data/1.parquet" into table = test
> +----------------------------------------------------------+
> | summary=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
> +----------------------------------------------------------+
> | Loaded 1 file(s). Total files in destination location: 2 |
> +----------------------------------------------------------+
> Fetched 1 row(s) in 3.39s
> [bdpe822n2:21000] > refresh test;
> Query: refresh test
> Query submitted at: 2017-03-10 14:46:54 (Coordinator:
> http://bdpe822n2:25000)
> Query progress can be monitored at:
> http://bdpe822n2:= 25000/query_plan?query_id=3D4d4ad8038a0362d3:8c7b326a0000000= 0
>
> Fetched 0 row(s) in 0.09s
> [bdpe822n2:21000] > show create table parquet_data.test;
> Query: show create table parquet_data.test
> ERROR: AnalysisException: Table does not exist: parquet_data.test
>
> [bdpe822n2:21000] > use parquet_data;
> Query: use parquet_data
> [bdpe822n2:21000] > show tables;
> Query: show tables
>
> Fetched 0 row(s) in 0.02s
>
>
>
>
> 2017-03-10 15:13 GMT+08:00 Sailesh Mukil <sailesh@cloudera.com>:
>>
>> Hi,
>>
>> Can you do a 'show create table parquet_data.test;'=C2=A0 = and paste the output?
>>
>> On Thu, Mar 9, 2017 at 11:09 PM, =E4=BF=8A=E6=9D=B0=E9=99=88 <<= a href=3D"mailto:cjjnjust@gmail.com">cjjnjust@gmail.com> wrote:
>>>
>>> Plus:
>>>
>>> In my root directory I found
>>> user/hive/warehouse/parquet_data.db/test/2.parquet. So it= seems impalad is
>>> manipulating on local file system.=C2=A0 How do I configure th= is?
>>>
>>> 2017-03-10 15:03 GMT+08:00 =E4=BF=8A=E6=9D=B0=E9=99=88 <cjjnjust@gmail.com>:
>>>>
>>>> Thanks from quick reply:)
>>>>
>>>> 1.parquet is always in the hdfs. I also did following comm= and for you
>>>> reference, please note the URI which is start with file:. = It looks weird.
>>>>
>>>> [bdpe30-cjj:21000] > use parquet_data;
>>>> Query: use parquet_data
>>>> [bdpe30-cjj:21000] > load data inpath "hdfs:///dat= a/2.parquet" into
>>>> table test;
>>>> Query: load data inpath "hdfs:///data/2.parquet"= into table test
>>>> +----------------------------------------------------= ------+
>>>> | summary=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |
>>>> +----------------------------------------------------= ------+
>>>> | Loaded 1 file(s). Total files in destination location: 2= |
>>>> +----------------------------------------------------= ------+
>>>> Fetched 1 row(s) in 0.50s
>>>> [bdpe30-cjj:21000] > select count(*) from test;
>>>> Query: select count(*) from test
>>>> Query submitted at: 2017-03-10 07:14:45 (Coordinator:
>>>> http://bdpe30-cjj:25000)
>>>> Query progress can be monitored at:
>>>> http= ://bdpe30-cjj:25000/query_plan?query_id=3D5d4ecce7d21182cc:e= 2dd7f5700000000
>>>> WARNINGS:
>>>> Failed to open HDFS file
>>>> file:/user/hive/warehouse/parquet_data.db/test/1.parq= uet
>>>> Error(2): No such file or directory
>>>>
>>>>
>>>> It seems like the load operation read data from hdfs, but = not put into
>>>> right place for query. Also the impalad seems access the f= ile in local file
>>>> system.
>>>>
>>>>
>>>> 2017-03-10 14:48 GMT+08:00 Jeszy <jeszyb@gmail.com>:
>>>>>
>>>>> Hello,
>>>>>
>>>>> Sounds like Impala expected 1.parquet to be in the fol= der, but it
>>>>> wasn't.
>>>>> You probably forgot to do 'refresh <table>&#= 39; after altering data from
>>>>> the outside.
>>>>>
>>>>> HTH
>>>>>
>>>>> On Fri, Mar 10, 2017 at 7:30 AM, =E4=BF=8A=E6=9D=B0=E9= =99=88 <cjjnjust@gmail.com>= wrote:
>>>>> > Hi,
>>>>> > I'm using latest impala built from github,=C2= =A0 and setup impala cluster
>>>>> > with
>>>>> > 2-nodes like below:
>>>>> > node-1: statestored, catalogd, namenode,datanode.=
>>>>> > node-2: impalad, datanode.
>>>>> >
>>>>> > Then I created database and table, loaded data fr= om external parquet
>>>>> > file
>>>>> > into table. Everything was OK, but when I execute= d a query it failed
>>>>> > with
>>>>> > following message:
>>>>> >
>>>>> > Failed to open HDFS file
>>>>> > file:/user/hive/warehouse/parquet_data.db/te= st/1.parquet
>>>>> > Error(2): No such file or directory
>>>>> >
>>>>> > But I can still =E2=80=98desc test=E2=80=99. Anyo= ne met with this? Thanks in
>>>>> > advanced.
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Thanks & Best Regards
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Thanks & Best Regards
>>>
>>>
>>>
>>>
>>> --
>>> Thanks & Best Regards
>>
>>
>
>
>
> --
> Thanks & Best Regards



--
=
Thanks &a= mp; Best Regards
--94eb2c18d080b875df054a92d108--