hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Olga Natkovich (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-252) Allow multiple paths in the load statement
Date Wed, 04 Jun 2008 21:19:44 GMT
Allow multiple paths in the load statement
------------------------------------------

                 Key: PIG-252
                 URL: https://issues.apache.org/jira/browse/PIG-252
             Project: Pig
          Issue Type: Improvement
            Reporter: Olga Natkovich


>From Tom White:

I;m having a problem loading data from multiple paths in Pig. What I'm trying to do is to
load data from a range of dates, so I would like to specify an input of two globbed paths:

x = LOAD '2008/05/{26,27,28,29,30,31},2008/06/{1,2}'

Pig doesn't seem to like this though as it's trying to interpret it as a single path. The
best I can do it to use UNION:

x1 = LOAD '2008/05/{26,27,28,29,30,31}'
x2 = LOAD '2008/06/{1,2}'
x = UNION x1, x2

The downside to this is that I want to parameterize my paths, and having separate script for
each number of paths in the input is cumbersome.

Is there a better way of doing this? Are there any plans to support multiple paths, and/or
PathFilters?


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message