hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prashant Kommireddi <prash1...@gmail.com>
Subject Re: Reading multiple files of a directory using a Single LOAD Command in PIG
Date Tue, 11 Jun 2013 21:32:06 GMT
What is the error?

The LoadFunc should be ignoring any filenames that begin with "_" or a
period "."
If you are trying to skip the _SUCCESS file, the loader you are using
(PigStorage) already handles that.

Also, can you double check your path is not "/Output/part-m* as opposed to
backward slashes?


On Tue, Jun 11, 2013 at 2:26 PM, Mix Nin <pig.mixed@gmail.com> wrote:

> I have a directory "Output2. It has file names as below
>
> -----------------
> _SUCCESS
> part-m-00000
> part-m-00001
> part-m-00002
> part-m-00003
> .
> .
> .
> .
> part-m-00100
> -----------------
>
> The above files are produced by PIG output STORE command .
>
> I want to read the files starting with "part-m-" using PIG command
>
> When I tried using Data= LOAD '\Output2\part-m-*' AS ( );
> It does not work and it throws error.
>
> How do I read these files in a single LOAD statement?
>
> Thanks
>
>

Mime
View raw message