Return-Path: Delivered-To: apmail-incubator-pig-dev-archive@locus.apache.org Received: (qmail 34975 invoked from network); 1 Apr 2008 00:40:50 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 1 Apr 2008 00:40:50 -0000 Received: (qmail 89713 invoked by uid 500); 1 Apr 2008 00:40:50 -0000 Delivered-To: apmail-incubator-pig-dev-archive@incubator.apache.org Received: (qmail 89636 invoked by uid 500); 1 Apr 2008 00:40:50 -0000 Mailing-List: contact pig-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: pig-dev@incubator.apache.org Delivered-To: mailing list pig-dev@incubator.apache.org Received: (qmail 89603 invoked by uid 99); 1 Apr 2008 00:40:50 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 31 Mar 2008 17:40:50 -0700 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.140] (HELO brutus.apache.org) (140.211.11.140) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 Apr 2008 00:40:07 +0000 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 3347C234C0B0 for ; Mon, 31 Mar 2008 17:38:24 -0700 (PDT) Message-ID: <2113415185.1207010304202.JavaMail.jira@brutus> Date: Mon, 31 Mar 2008 17:38:24 -0700 (PDT) From: "Olga Natkovich (JIRA)" To: pig-dev@incubator.apache.org Subject: [jira] Created: (PIG-176) pig creates many small files when it spills MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org pig creates many small files when it spills ------------------------------------------- Key: PIG-176 URL: https://issues.apache.org/jira/browse/PIG-176 Project: Pig Issue Type: Bug Reporter: Olga Natkovich Assignee: Alan Gates Currently, on spill pig can generate millions of small (under 128K) files. Partially this is due to PIG-170 but even with that patch, you can still try and spill small bags. The proposal is to not spill small files. Alan told me that the logic is already there but we just need to bump the size limit. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.