pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Johnny Zhang" <xiao...@cloudera.com>
Subject Re: Review Request: PIG-3223 AvroStorage does not handle comma separated input paths
Date Thu, 02 May 2013 20:36:04 GMT


> On April 30, 2013, 8:07 p.m., Rohini Palaniswamy wrote:
> > contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/AvroStorageUtils.java,
line 160
> > <https://reviews.apache.org/r/10351/diff/1/?file=279039#file279039line160>
> >
> >     Pattern should be a private static variable. This pattern only takes into account
globs of the form {x,y}.  Hadoop glob status supports a lot more
> >     
> >     http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/FileSystem.html#globStatus(org.apache.hadoop.fs.Path).

> >     
> >     Found this method in pig which would take care of the logic - LoadFunc.getPathStrings()
. Use this for splitting paths.  This should simplify the whole change
> >     
> >

Thanks a lot for your comments, Rohini. I am going to update the revised patch very soon today.


- Johnny


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10351/#review19974
-----------------------------------------------------------


On April 8, 2013, 10:03 p.m., Johnny Zhang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10351/
> -----------------------------------------------------------
> 
> (Updated April 8, 2013, 10:03 p.m.)
> 
> 
> Review request for pig.
> 
> 
> Description
> -------
> 
> we want to support comma separated input paths in AvroStorage, for example
> "test_dir1/test_glob1.avro,test_dir1/test_glob2.avro,test_dir1/test_glob3.avro"
> "test_dir1/*, test_dir2/test_glob4.avro, test_dir2/test_glob5.avro"
> 
> 
> This addresses bug PIG-3223.
>     https://issues.apache.org/jira/browse/PIG-3223
> 
> 
> Diffs
> -----
> 
>   contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/AvroStorageUtils.java
0ac0225 
>   contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/TestAvroStorage.java
bd7a6d2 
> 
> Diff: https://reviews.apache.org/r/10351/diff/
> 
> 
> Testing
> -------
> 
> added two more test cases in TestAvroStorage.java and they all pass
> 
> 
> Thanks,
> 
> Johnny Zhang
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message