pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Olga Natkovich (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-176) pig creates many small files when it spills
Date Tue, 01 Apr 2008 00:38:24 GMT
pig creates many small files when it spills

                 Key: PIG-176
                 URL: https://issues.apache.org/jira/browse/PIG-176
             Project: Pig
          Issue Type: Bug
            Reporter: Olga Natkovich
            Assignee: Alan Gates

Currently, on spill pig can generate millions of small (under 128K) files. Partially this
is due to PIG-170 but even with that patch, you can still try and spill small bags.

The proposal is to not spill small files. Alan told me that the logic is already there but
we just need to bump the size limit.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message