hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bejoy Ks <bejoy.had...@gmail.com>
Subject Re: Pig Output
Date Mon, 05 Dec 2011 20:34:43 GMT
Hi Aaron
         Instead of copyFromLocal use getmerge. It would do your job. The
syntax for CLI is
hadoop fs -getmerge <source dir in hdfs/pig output dir> <lfs destn
dir>/xyz.txt


Hope it helps!...

Regards
Bejoy.K.S

On Tue, Dec 6, 2011 at 1:57 AM, Aaron Griffith
<aaron.c.griffith@gmail.com>wrote:

> Using PigStorage() my pig script output gets put into partial files on the
> hadoop
> file system.
>
> When I use the copyToLocal fuction from Hadoop it creates a local
> directory with
> all the partial files.
>
> Is there a way to copy the partial files from hadoop into a single local
> file?
>
> Thanks
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message