hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lili Ma (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HAWQ-1144) Register into a 2-level partition table, hawq register didn't throw error, and indicates that hawq register succeed, but no data can be selected out.
Date Thu, 03 Nov 2016 08:59:59 GMT
Lili Ma created HAWQ-1144:
-----------------------------

             Summary: Register into a 2-level partition table, hawq register didn't throw
error, and indicates that hawq register succeed, but no data can be selected out.
                 Key: HAWQ-1144
                 URL: https://issues.apache.org/jira/browse/HAWQ-1144
             Project: Apache HAWQ
          Issue Type: Bug
          Components: Command Line Tools
            Reporter: Lili Ma
            Assignee: Lei Chang
             Fix For: 2.0.1.0-incubating


Register into a 2-level partition table, hawq register didn't throw error, and indicates that
hawq register succeed, but no data can be selected out.

Reproduce Steps:
1. Create a one-level partition table
```
 create table parquet_wt (id SERIAL,a1 int,a2 char(5),a3 numeric,a4 boolean DEFAULT false
,a5 char DEFAULT 'd',a6 text,a7 timestamp,a8 character varying(705),a9 bigint,a10 date,a11
varchar(600),a12 text,a13 decimal,a14 real,a15 bigint,a16 int4 ,a17 bytea,a18 timestamp with
time zone,a19 timetz,a20 path,a21 box,a22 macaddr,a23 interval,a24 character varying(800),a25
lseg,a26 point,a27 double precision,a28 circle,a29 int4,a30 numeric(8),a31 polygon,a32 date,a33
real,a34 money,a35 cidr,a36 inet,a37 time,a38 text,a39 bit,a40 bit varying(5),a41 smallint,a42
int )   WITH (appendonly=true, orientation=parquet) distributed randomly  Partition by range(a1)
(start(1)  end(5000) every(1000) );
```
2. insert some data into this table
```
insert into parquet_wt (a1,a2,a3,a4,a5,a6,a7,a8,a9,a10,a11,a12,a13,a14,a15,a16,a17,a18,a19,a20,a21,a22,a23,a24,a25,a26,a27,a28,a29,a30,a31,a32,a33,a34,a35,a36,a37,a38,a39,a40,a41,a42)
values(generate_series(1,20),'M',2011,'t','a','This is news of today: Deadlock between Republicans
and Democrats over how best to reduce the U.S. deficit, and over what period, has blocked
an agreement to allow the raising of the $14.3 trillion debt ceiling','2001-12-24 02:26:11','U.S.
House of Representatives Speaker John Boehner, the top Republican in Congress who has put
forward a deficit reduction plan to be voted on later on Thursday said he had no control over
whether his bill would avert a credit downgrade.',generate_series(2490,2505),'2011-10-11','The
Republican-controlled House is tentatively scheduled to vote on Boehner proposal this afternoon
at around 6 p.m. EDT (2200 GMT). The main Republican vote counter in the House, Kevin McCarthy,
would not say if there were enough votes to pass the bill.','WASHINGTON:House Speaker John
Boehner says his plan mixing spending cuts in exchange for raising the nations $14.3 trillion
debt limit is not perfect but is as large a step that a divided government can take that is
doable and signable by President Barack Obama.The Ohio Republican says the measure is an honest
and sincere attempt at compromise and was negotiated with Democrats last weekend and that
passing it would end the ongoing debt crisis. The plan blends $900 billion-plus in spending
cuts with a companion increase in the nations borrowing cap.','1234.56',323453,generate_series(3452,3462),7845,'0011','2005-07-16
01:51:15+1359','2001-12-13 01:51:15','((1,2),(0,3),(2,1))','((2,3)(4,5))','08:00:2b:01:02:03','1-2','Republicans
had been working throughout the day Thursday to lock down support for their plan to raise
the nations debt ceiling, even as Senate Democrats vowed to swiftly kill it if passed.','((2,3)(4,5))','(6,7)',11.222,'((4,5),7)',32,3214,'(1,0,2,3)','2010-02-21',43564,'$1,000.00','192.168.1','126.1.3.4','12:30:45','Johnson
& Johnsons McNeil Consumer Healthcare announced the voluntary dosage reduction today.
Labels will carry new dosing instructions this fall.The company says it will cut the maximum
dosage of Regular Strength Tylenol and other acetaminophen-containing products in 2012.Acetaminophen
is safe when used as directed, says Edwin Kuffner, MD, McNeil vice president of over-the-counter
medical affairs. But, when too much is taken, it can cause liver damage.The action is intended
to cut the risk of such accidental overdoses, the company says in a news release.','1','0',12,23);
```
3. extract the metadata out for the table
```
hawq extract -d postgres -o ~/parquet.yaml parquet_wt
```
4. create a two-level partition table
```
CREATE TABLE parquet_wt_subpartgzip2                                                     
                                                (id SERIAL,a1 int,a2 char(5),a3 numeric,a4
boolean DEFAULT false ,a5 char DEFAULT 'd',a6 text,a7 timestamp,a8 character varying(705),a9
bigint,a10 date,a11 varchar(600),a12 text,a13 decimal,a14 real,a15 bigint,a16 int4 ,a17 bytea,a18
timestamp with time zone,a19 timetz,a20 path,a21 box,a22 macaddr,a23 interval,a24 character
varying(800),a25 lseg,a26 point,a27 double precision,a28 circle,a29 int4,a30 numeric(8),a31
polygon,a32 date,a33 real,a34 money,a35 cidr,a36 inet,a37 time,a38 text,a39 bit,a40 bit varying(5),a41
smallint,a42 int )                                 WITH (appendonly=true, orientation=parquet)
distributed randomly  Partition by range(a1) Subpartition by list(a2) subpartition template
( default subpartition df_sp, subpartition sp1 values('M') , subpartition sp2 values('F')
                                                              WITH (appendonly=true, orientation=parquet,compresstype=gzip,compresslevel=2))
(start(1)  end(5000) every(1000) );
```
5. call register
```
 hawq register -d postgres -c ~/parquet.yaml parquet_wt_subpartgzip2
```
6. It reflects register succeed.
```
malilis-MacBook-Pro:tpch malili$ hawq register -d postgres -c ~/parquet.yaml parquet_wt_subpartgzip2
20161103:15:58:10:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-try to connect database
localhost:5432 postgres
20161103:15:58:10:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check...
20161103:15:58:11:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check done...
20161103:15:58:11:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check...
20161103:15:58:13:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check done...
20161103:15:58:13:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check...
20161103:15:58:14:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check done...
20161103:15:58:14:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check...
20161103:15:58:16:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check done...
20161103:15:58:16:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check...
20161103:15:58:17:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check done...
20161103:15:58:20:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-New file(s) to be
registered: ['hdfs://localhost:8020/hawq_default/16385/16387/17065/1']
20161103:15:58:29:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-New file(s) to be
registered: ['hdfs://localhost:8020/hawq_default/16385/16387/17074/1']
20161103:15:58:35:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-New file(s) to be
registered: ['hdfs://localhost:8020/hawq_default/16385/16387/17083/1']
20161103:15:58:41:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-New file(s) to be
registered: ['hdfs://localhost:8020/hawq_default/16385/16387/17092/1']
20161103:15:58:47:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-New file(s) to be
registered: ['hdfs://localhost:8020/hawq_default/16385/16387/17101/1']
hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/17065/1 hdfs://localhost:8020/hawq_default/16385/16387/16784/1"
hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/17074/1 hdfs://localhost:8020/hawq_default/16385/16387/16822/1"
hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/17083/1 hdfs://localhost:8020/hawq_default/16385/16387/16860/1"
hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/17092/1 hdfs://localhost:8020/hawq_default/16385/16387/16898/1"
hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/17101/1 hdfs://localhost:8020/hawq_default/16385/16387/16936/1"
20161103:15:58:58:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Hawq Register Succeed.
```
7. But when we select the table, no data can be selected out.  
```
postgres=# select count(*) from parquet_wt_subpartgzip2;
 count
-------
     0
(1 row)
```
Actually we should throw error if hawq register wants to register into a 2 or more -level
partition table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message