Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id B8FA0200BB1 for ; Thu, 3 Nov 2016 10:00:05 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id B78B4160B0B; Thu, 3 Nov 2016 09:00:05 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id D5723160AFE for ; Thu, 3 Nov 2016 10:00:04 +0100 (CET) Received: (qmail 9290 invoked by uid 500); 3 Nov 2016 09:00:04 -0000 Mailing-List: contact dev-help@hawq.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hawq.incubator.apache.org Delivered-To: mailing list dev@hawq.incubator.apache.org Received: (qmail 9278 invoked by uid 99); 3 Nov 2016 09:00:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Nov 2016 09:00:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 30605C6F6F for ; Thu, 3 Nov 2016 09:00:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -7.018 X-Spam-Level: X-Spam-Status: No, score=-7.018 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1, LOTS_OF_MONEY=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id MHheEFHwzAzU for ; Thu, 3 Nov 2016 09:00:01 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id A94605FB7F for ; Thu, 3 Nov 2016 09:00:00 +0000 (UTC) Received: (qmail 8853 invoked by uid 99); 3 Nov 2016 08:59:59 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Nov 2016 08:59:59 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 94B112C2A66 for ; Thu, 3 Nov 2016 08:59:59 +0000 (UTC) Date: Thu, 3 Nov 2016 08:59:59 +0000 (UTC) From: "Lili Ma (JIRA)" To: dev@hawq.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (HAWQ-1144) Register into a 2-level partition table, hawq register didn't throw error, and indicates that hawq register succeed, but no data can be selected out. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 03 Nov 2016 09:00:05 -0000 Lili Ma created HAWQ-1144: ----------------------------- Summary: Register into a 2-level partition table, hawq registe= r didn't throw error, and indicates that hawq register succeed, but no data= can be selected out. Key: HAWQ-1144 URL: https://issues.apache.org/jira/browse/HAWQ-1144 Project: Apache HAWQ Issue Type: Bug Components: Command Line Tools Reporter: Lili Ma Assignee: Lei Chang Fix For: 2.0.1.0-incubating Register into a 2-level partition table, hawq register didn't throw error, = and indicates that hawq register succeed, but no data can be selected out. Reproduce Steps: 1. Create a one-level partition table ``` create table parquet_wt (id SERIAL,a1 int,a2 char(5),a3 numeric,a4 boolean= DEFAULT false ,a5 char DEFAULT 'd',a6 text,a7 timestamp,a8 character varyi= ng(705),a9 bigint,a10 date,a11 varchar(600),a12 text,a13 decimal,a14 real,a= 15 bigint,a16 int4 ,a17 bytea,a18 timestamp with time zone,a19 timetz,a20 p= ath,a21 box,a22 macaddr,a23 interval,a24 character varying(800),a25 lseg,a2= 6 point,a27 double precision,a28 circle,a29 int4,a30 numeric(8),a31 polygon= ,a32 date,a33 real,a34 money,a35 cidr,a36 inet,a37 time,a38 text,a39 bit,a4= 0 bit varying(5),a41 smallint,a42 int ) WITH (appendonly=3Dtrue, orientat= ion=3Dparquet) distributed randomly Partition by range(a1) (start(1) end(= 5000) every(1000) ); ``` 2. insert some data into this table ``` insert into parquet_wt (a1,a2,a3,a4,a5,a6,a7,a8,a9,a10,a11,a12,a13,a14,a15,= a16,a17,a18,a19,a20,a21,a22,a23,a24,a25,a26,a27,a28,a29,a30,a31,a32,a33,a34= ,a35,a36,a37,a38,a39,a40,a41,a42) values(generate_series(1,20),'M',2011,'t'= ,'a','This is news of today: Deadlock between Republicans and Democrats ove= r how best to reduce the U.S. deficit, and over what period, has blocked an= agreement to allow the raising of the $14.3 trillion debt ceiling','2001-1= 2-24 02:26:11','U.S. House of Representatives Speaker John Boehner, the top= Republican in Congress who has put forward a deficit reduction plan to be = voted on later on Thursday said he had no control over whether his bill wou= ld avert a credit downgrade.',generate_series(2490,2505),'2011-10-11','The = Republican-controlled House is tentatively scheduled to vote on Boehner pro= posal this afternoon at around 6 p.m. EDT (2200 GMT). The main Republican v= ote counter in the House, Kevin McCarthy, would not say if there were enoug= h votes to pass the bill.','WASHINGTON:House Speaker John Boehner says his = plan mixing spending cuts in exchange for raising the nations $14.3 trillio= n debt limit is not perfect but is as large a step that a divided governmen= t can take that is doable and signable by President Barack Obama.The Ohio R= epublican says the measure is an honest and sincere attempt at compromise a= nd was negotiated with Democrats last weekend and that passing it would end= the ongoing debt crisis. The plan blends $900 billion-plus in spending cut= s with a companion increase in the nations borrowing cap.','1234.56',323453= ,generate_series(3452,3462),7845,'0011','2005-07-16 01:51:15+1359','2001-12= -13 01:51:15','((1,2),(0,3),(2,1))','((2,3)(4,5))','08:00:2b:01:02:03','1-2= ','Republicans had been working throughout the day Thursday to lock down su= pport for their plan to raise the nations debt ceiling, even as Senate Demo= crats vowed to swiftly kill it if passed.','((2,3)(4,5))','(6,7)',11.222,'(= (4,5),7)',32,3214,'(1,0,2,3)','2010-02-21',43564,'$1,000.00','192.168.1','1= 26.1.3.4','12:30:45','Johnson & Johnsons McNeil Consumer Healthcare announc= ed the voluntary dosage reduction today. Labels will carry new dosing instr= uctions this fall.The company says it will cut the maximum dosage of Regula= r Strength Tylenol and other acetaminophen-containing products in 2012.Acet= aminophen is safe when used as directed, says Edwin Kuffner, MD, McNeil vic= e president of over-the-counter medical affairs. But, when too much is take= n, it can cause liver damage.The action is intended to cut the risk of such= accidental overdoses, the company says in a news release.','1','0',12,23); ``` 3. extract the metadata out for the table ``` hawq extract -d postgres -o ~/parquet.yaml parquet_wt ``` 4. create a two-level partition table ``` CREATE TABLE parquet_wt_subpartgzip2 = (id SERIAL,a= 1 int,a2 char(5),a3 numeric,a4 boolean DEFAULT false ,a5 char DEFAULT 'd',a= 6 text,a7 timestamp,a8 character varying(705),a9 bigint,a10 date,a11 varcha= r(600),a12 text,a13 decimal,a14 real,a15 bigint,a16 int4 ,a17 bytea,a18 tim= estamp with time zone,a19 timetz,a20 path,a21 box,a22 macaddr,a23 interval,= a24 character varying(800),a25 lseg,a26 point,a27 double precision,a28 circ= le,a29 int4,a30 numeric(8),a31 polygon,a32 date,a33 real,a34 money,a35 cidr= ,a36 inet,a37 time,a38 text,a39 bit,a40 bit varying(5),a41 smallint,a42 int= ) WITH (appendonly=3Dtrue, orientation=3Dp= arquet) distributed randomly Partition by range(a1) Subpartition by list(a= 2) subpartition template ( default subpartition df_sp, subpartition sp1 val= ues('M') , subpartition sp2 values('F') = WITH (appendonly=3Dtrue, orientation=3Dparquet,c= ompresstype=3Dgzip,compresslevel=3D2)) (start(1) end(5000) every(1000) ); ``` 5. call register ``` hawq register -d postgres -c ~/parquet.yaml parquet_wt_subpartgzip2 ``` 6. It reflects register succeed. ``` malilis-MacBook-Pro:tpch malili$ hawq register -d postgres -c ~/parquet.yam= l parquet_wt_subpartgzip2 20161103:15:58:10:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-tr= y to connect database localhost:5432 postgres 20161103:15:58:10:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Fi= les check... 20161103:15:58:11:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Fi= les check done... 20161103:15:58:11:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Fi= les check... 20161103:15:58:13:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Fi= les check done... 20161103:15:58:13:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Fi= les check... 20161103:15:58:14:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Fi= les check done... 20161103:15:58:14:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Fi= les check... 20161103:15:58:16:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Fi= les check done... 20161103:15:58:16:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Fi= les check... 20161103:15:58:17:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Fi= les check done... 20161103:15:58:20:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Ne= w file(s) to be registered: ['hdfs://localhost:8020/hawq_default/16385/1638= 7/17065/1'] 20161103:15:58:29:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Ne= w file(s) to be registered: ['hdfs://localhost:8020/hawq_default/16385/1638= 7/17074/1'] 20161103:15:58:35:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Ne= w file(s) to be registered: ['hdfs://localhost:8020/hawq_default/16385/1638= 7/17083/1'] 20161103:15:58:41:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Ne= w file(s) to be registered: ['hdfs://localhost:8020/hawq_default/16385/1638= 7/17092/1'] 20161103:15:58:47:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Ne= w file(s) to be registered: ['hdfs://localhost:8020/hawq_default/16385/1638= 7/17101/1'] hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/1706= 5/1 hdfs://localhost:8020/hawq_default/16385/16387/16784/1" hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/1707= 4/1 hdfs://localhost:8020/hawq_default/16385/16387/16822/1" hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/1708= 3/1 hdfs://localhost:8020/hawq_default/16385/16387/16860/1" hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/1709= 2/1 hdfs://localhost:8020/hawq_default/16385/16387/16898/1" hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/1710= 1/1 hdfs://localhost:8020/hawq_default/16385/16387/16936/1" 20161103:15:58:58:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Ha= wq Register Succeed. ``` 7. But when we select the table, no data can be selected out. =20 ``` postgres=3D# select count(*) from parquet_wt_subpartgzip2; count ------- 0 (1 row) ``` Actually we should throw error if hawq register wants to register into a 2 = or more -level partition table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)