Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type:Message-ID;
  b=KcbKWldhuTqhR4CYuW/0yO4fT2byMc2SrziuqcBDS9suTSM1rRXrr1Kh1/9hITsou66FPzXW1jgWnzGgv+gIeYYDNKqrTqTnKK4g3UQIdxvkBnSg0lFRn2shIPdnwOcrp3+zsSOdEh1uLJmuPAu9qYaagkroSv4WFA9WFEqCJR8=;
Date: Wed, 17 Dec 2008 19:18:51 -0800 (PST)
From: C G <parallelguy@yahoo.com>
Reply-To: parallelguy@yahoo.com
Subject: Copy data between HDFS instances...
To: core-user@hadoop.apache.org
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Message-ID: <741080.9764.qm@web45402.mail.sp1.yahoo.com>

Hi All:

I am setting up 2 grids, each with its own HDFS.  The grids are unaware of each other but exist on the same network.

I'd like to copy data from one HDFS to the other.  Is there a way to do this simply, or do I need to cobble together scripts to copy from HDFS on one side and pipe to a dfs -cp on the other side?

I tried something like this:

     hadoop dfs -ls hdfs://grid1NameNode:portNo/

from grid2 trying to ls on grid1 but got a "wrong FS" error message.  I also tried:

    hadoop dfs -ls hdfs://grid1NameNode:portNo/foo

on grid2 where "/foo" exists on grid1 and got 0 files found.

I assume there is some way to do this and I just don't have the right command line magic.  This is Hadoop 0.15.0.

Any help appreciated.

Thanks,
C G