hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joseph Stein <crypt...@gmail.com>
Subject Re: copyToLocal
Date Thu, 10 Jun 2010 23:30:51 GMT
I think 3 weeks of no sleep caused this.  My automation script failed
leaving the directory there so when it re-ran THEN it caused this weirdness.

I guess copyToLocal if it sees a directory already existing it then appends
the directory as a child to the local (so in my first scenario
/data/2010_06_10/81ae7c24745211df9f6d002590008422 already was existing
because my script created it the first time).

Still odd behavior, whatever it is fine sorry to bother... i just added to
my automation script to remove the directory before i do a copyToLocal.


On Thu, Jun 10, 2010 at 7:12 PM, Joseph Stein <cryptcom@gmail.com> wrote:

> Hi, so ok  am using copyToLocal through an automation script we have and
> seeing odd results.
>
> I am not sure if this is something I am doing wrong, defect, or known good
> reason for it.  Let me know I would like to correct this in either my own
> script, happy to give a try in the fs code fixing a bug or my own brain in
> understanding because there is some good reason for this.
>
> scenario
>
> hadoop fs -copyToLocal event/2010_06_10/81ae7c24745211df9f6d002590008422
> /data/2010_06_10/81ae7c24745211df9f6d002590008422
>
> is resulting in my part files showing up in *
> /data/2010_06_10/81ae7c24745211df9f6d002590008422/81ae7c24745211df9f6d002590008422
> *
> if i try [Hadoop 0.20.1]
>
> hadoop fs -copyToLocal event/2010_06_10/81ae7c24745211df9f6d002590008422
> /data/2010_06_10
>
> is resulting in my part files showing up in just */data/2010_06_10*
> (with no creation of the UUID directory like it did before)
>
> my desired result is to have the files
>
> from *event/2010_06_10/81ae7c24745211df9f6d002590008422
>
> *and end up in */data/2010_06_10/81ae7c24745211df9f6d002590008422*
>
> with the trailing directory creating itself like it does in the first
> scenario but not duplicating it as it does....weirdly.
>
> /*
> Joe Stein
> http://www.linkedin.com/in/charmalloc
> Twitter: @allthingshadoop
> */
>
>


-- 
/*
Joe Stein
http://www.linkedin.com/in/charmalloc
*/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message