tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Min Zhou (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (TAJO-475) Table partition catalog recap
Date Wed, 08 Jan 2014 05:54:50 GMT

    [ https://issues.apache.org/jira/browse/TAJO-475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13865111#comment-13865111
] 

Min Zhou edited comment on TAJO-475 at 1/8/14 5:52 AM:
-------------------------------------------------------

Here is the proto after discuss with Hyunsik offline

{noformat}
message PartitionMethodProto {
  required string tableId = 1;
  required PartitionType partitionType = 2;
  required string expression = 3;
  required SchemaProto expressionSchema = 4;
}

message PartitionDescProto {
  required string tableId = 1;
  required int32 partitionMethodId = 2;
  optional string partitionName = 3;
  required int32  ordinalPosition = 4;
  optional string paritionValue = 5;
  optional string path = 6;
  optional string cacheNodes = 7;
}
{noformat}

At most one partiton method for one table.  PARTITION_METHODS metadata records the partition
type, partition expression(or columns) and the expression for that expression. 

PARTITIONS metadata records the detail of each partition, such as partition name if it has,
partition value is the parition method is range/column partitioning, and hdfs path , local
cache nodes path.

If we create a tajo table like this way
{noformat}
 CREATE TABLE IF NOT EXISTS `user` (  
 ->   `id` int(11) NOT NULL AUTO_INCREMENT COMMENT 'user ID',  
 ->   `name` varchar(50) NOT NULL DEFAULT '' COMMENT user name',  
 ->   `sex` int(1) NOT NULL DEFAULT '0' COMMENT '0 for male,1 for female',  
 ->   PRIMARY KEY (`id`)  
 -> ) ENGINE=MyISAM  DEFAULT CHARSET=utf8 AUTO_INCREMENT=1  
 -> PARTITION BY RANGE (id) (  
 ->     PARTITION p0 VALUES LESS THAN (3),  
 ->     PARTITION p1 VALUES LESS THAN (6),  
 ->     PARTITION p2 VALUES LESS THAN (9),  
 ->     PARTITION p3 VALUES LESS THAN (12),  
 ->     PARTITION p4 VALUES LESS THAN MAXVALUE  
 -> );  
{noformat}

The PARTITION_METHODS table in metadata will add one row, and PARTITIONS table will add 5
rows for 5 partitions.







was (Author: coderplay):
Here is the proto after discuss with Hyunsik offline

{noformat}
message PartitionMethodProto {
  required string tableId = 1;
  required PartitionType partitionType = 2;
  required string expression = 3;
  required SchemaProto expressionSchema = 4;
}

message PartitionDescProto {
  required string tableId = 1;
  required int32 partitionMethodId = 2;
  optional string partitionName = 3;
  required int32  ordinalPosition = 4;
  optional string paritionValue = 5;
  optional string path = 6;
  optional string cacheNodes = 7;
}
{noformat}

At most one partiton method for one table.  PARTITION_METHODS metadata records the partition
type, partition expression(or columns) and the expression for that expression. 

PARTITIONS metadata records the detail of each partition, such as partition name if it has,
partition value is the parition method is range/column partitioning, and hdfs path , local
cache nodes path.

If we create a tajo table like this way
{noformat}
 CREATE TABLE IF NOT EXISTS `user` (  
 ->   `id` int(11) NOT NULL AUTO_INCREMENT COMMENT 'user ID',  
 ->   `name` varchar(50) NOT NULL DEFAULT '' COMMENT user name',  
 ->   `sex` int(1) NOT NULL DEFAULT '0' COMMENT '0 for male,1 for female',  
 ->   PRIMARY KEY (`id`)  
 -> ) ENGINE=MyISAM  DEFAULT CHARSET=utf8 AUTO_INCREMENT=1  
 -> PARTITION BY RANGE (id) (  
 ->     PARTITION p0 VALUES LESS THAN (3),  
 ->     PARTITION p1 VALUES LESS THAN (6),  
 ->     PARTITION p2 VALUES LESS THAN (9),  
 ->     PARTITION p3 VALUES LESS THAN (12),  
 ->     PARTITION p4 VALUES LESS THAN MAXVALUE  
 -> );  
{format}

The PARTITION_METHODS table in mysql will add one row, and PARTITIONS table will add 5 rows
for 5 partitions.






> Table partition catalog recap
> -----------------------------
>
>                 Key: TAJO-475
>                 URL: https://issues.apache.org/jira/browse/TAJO-475
>             Project: Tajo
>          Issue Type: Sub-task
>          Components: catalog
>            Reporter: Min Zhou
>            Assignee: Min Zhou
>
> Query master need to know where partitions of memory cached table locate. 
> At least we need a meta table contain such information
> |partition_id|
> |partition_value|
> |ordinal_position|
> |locations|
> Any suggestion?
>  



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message