Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: 
 <CAN1suNOk6j44GhPpfvj7WUEz9Yk7JEYqQJDp1_A1tFQApiiacA@mail.gmail.com>
References: 
 <CAN1suNOk6j44GhPpfvj7WUEz9Yk7JEYqQJDp1_A1tFQApiiacA@mail.gmail.com>
Date: Mon, 18 Feb 2013 12:23:28 -0500
Message-ID: 
 <CAPQV63VCDvNUvpf3THPundtM8HB75ATyDPSnJ0Vuik5E35ZTjg@mail.gmail.com>
Subject: Re: Piping output of hadoop command
From: Jean-Marc Spaggiari <jean-marc@spaggiari.org>
To: user@hadoop.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi Julian,

I think it's not outputing on the standard output bu on the error one.

You might want to test that:
hadoop fs -copyToLocal FILE_IN_HDFS 1>&2 | ssh REMOTE_HOST "dd
of=3DFILE_ON REMOTE_HOST"

Which will redirect the stderr to the stdout too.

Not sure, but it might be your issue.

JM

2013/2/18, Julian Wissmann <julian.wissmann@sdace.de>:
> Hi,
>
> we're running a Hadoop cluster with hbase for the purpose of
> evaluating it as database for a research project and we've more or
> less decided to go with it.
> So now I'm exploring backup mechanisms and have decided to experiment
> with hadoops export functionality for that.
>
> What I am trying to achieve is getting data out of hbase and into hdfs
> via hadoop export and then copy it out of hdfs onto a backup system.
> However while copying data out of hdfs to the backup machine I am
> experiencing problems.
>
> What I am trying to do is the following:
>
> hadoop fs -copyToLocal FILE_IN_HDFS | ssh REMOTE_HOST "dd of=3DTARGET_FIL=
E"
>
> It creates a file on the remote host, however this file is 0kb in
> size; instead of copying any data over there, the file just lands in
> my home folder.
>
> The command output looks like this: hadoop fs -copyToLocal
> FILE_IN_HDFS | ssh REMOTE_HOST "dd of=3DFILE_ON REMOTE_HOST"
> 0+0 Datens=C3=A4tze ein
> 0+0 Datens=C3=A4tze aus
> 0 Bytes (0 B) kopiert, 1,10011 s, 0,0 kB/s
>
> I cannot think of any reason, why this command would behave in this
> way. Is this some Java-ism that I'm missing here (like not correctly
> treating stdout), or am I actually doing it wrong?
>
> The Hadoop Version is 2.0.0-cdh4.1.2
>
> Regards
>
> Julian
>