Return-Path: X-Original-To: apmail-tajo-dev-archive@minotaur.apache.org Delivered-To: apmail-tajo-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4395210383 for ; Tue, 29 Oct 2013 13:57:08 +0000 (UTC) Received: (qmail 76038 invoked by uid 500); 29 Oct 2013 13:57:05 -0000 Delivered-To: apmail-tajo-dev-archive@tajo.apache.org Received: (qmail 75930 invoked by uid 500); 29 Oct 2013 13:56:58 -0000 Mailing-List: contact dev-help@tajo.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@tajo.incubator.apache.org Delivered-To: mailing list dev@tajo.incubator.apache.org Received: (qmail 75905 invoked by uid 99); 29 Oct 2013 13:56:55 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Oct 2013 13:56:55 +0000 X-ASF-Spam-Status: No, hits=-2000.5 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Tue, 29 Oct 2013 13:56:53 +0000 Received: (qmail 75223 invoked by uid 99); 29 Oct 2013 13:56:30 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Oct 2013 13:56:30 +0000 Date: Tue, 29 Oct 2013 13:56:30 +0000 (UTC) From: "Jihoon Son (JIRA)" To: dev@tajo.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (TAJO-292) Too many intermediate partition files MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/TAJO-292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13807991#comment-13807991 ] Jihoon Son commented on TAJO-292: --------------------------------- +1 for this issue. This will improve the performance significantly. > Too many intermediate partition files > ------------------------------------- > > Key: TAJO-292 > URL: https://issues.apache.org/jira/browse/TAJO-292 > Project: Tajo > Issue Type: Bug > Components: repartitioning > Reporter: Hyunsik Choi > Assignee: Hyunsik Choi > Priority: Critical > Fix For: 0.8-incubating > > > Unlike the before, the number of partitions are being currently determined by the volume size and the number of distinct keys. It can cause unnecessary overheads. We need to improve the partition number determiner to consider the number of cluster nodes. -- This message was sent by Atlassian JIRA (v6.1#6144)