Return-Path: X-Original-To: apmail-tajo-dev-archive@minotaur.apache.org Delivered-To: apmail-tajo-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D42B910BE7 for ; Wed, 18 Dec 2013 18:00:38 +0000 (UTC) Received: (qmail 67923 invoked by uid 500); 18 Dec 2013 18:00:38 -0000 Delivered-To: apmail-tajo-dev-archive@tajo.apache.org Received: (qmail 67845 invoked by uid 500); 18 Dec 2013 18:00:37 -0000 Mailing-List: contact dev-help@tajo.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@tajo.incubator.apache.org Delivered-To: mailing list dev@tajo.incubator.apache.org Received: (qmail 67833 invoked by uid 99); 18 Dec 2013 18:00:35 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 Dec 2013 18:00:35 +0000 X-ASF-Spam-Status: No, hits=-2000.5 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Wed, 18 Dec 2013 18:00:34 +0000 Received: (qmail 66810 invoked by uid 99); 18 Dec 2013 18:00:13 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 Dec 2013 18:00:13 +0000 Date: Wed, 18 Dec 2013 18:00:13 +0000 (UTC) From: "Min Zhou (JIRA)" To: dev@tajo.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (TAJO-283) Add Table Partitioning MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/TAJO-283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851968#comment-13851968 ] Min Zhou commented on TAJO-283: ------------------------------- Great! thanks for the information. I was considering about the small hdfs files issue if we won't do a merge through shuffle. The file number should be M * R, where M is the mapper tasks number and R is the reducer tasks number. If data shuffling is added, files numbers would drop into R. > Add Table Partitioning > ---------------------- > > Key: TAJO-283 > URL: https://issues.apache.org/jira/browse/TAJO-283 > Project: Tajo > Issue Type: New Feature > Components: catalog, physical operator, planner/optimizer > Reporter: Hyunsik Choi > Assignee: Hyunsik Choi > Fix For: 0.8-incubating > > > Table partitioning gives many facilities to maintain large tables. First of all, it enables the data management system to prune many input data which are actually not necessary. In addition, it gives the system more optimization opportunities that exploit the physical layouts. > Basically, Tajo should follow the RDBMS-style partitioning system, including range, list, hash, and so on. In order to keep Hive compatibility, we need to add Hive partition type that does not exists in existing DBMS systems. -- This message was sent by Atlassian JIRA (v6.1.4#6159)