Return-Path: 
 <hdfs-user-return-771-apmail-hadoop-hdfs-user-archive=hadoop.apache.org@hadoop.apache.org>
Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org
Received: (qmail 17824 invoked from network); 12 Jan 2011 19:58:52 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3)
  by minotaur.apache.org with SMTP; 12 Jan 2011 19:58:52 -0000
Received: (qmail 84268 invoked by uid 500); 12 Jan 2011 19:58:52 -0000
Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org
Received: (qmail 84197 invoked by uid 500); 12 Jan 2011 19:58:51 -0000
Mailing-List: contact hdfs-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:hdfs-user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:hdfs-user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:hdfs-user@hadoop.apache.org>
List-Id: <hdfs-user.hadoop.apache.org>
Reply-To: hdfs-user@hadoop.apache.org
Delivered-To: mailing list hdfs-user@hadoop.apache.org
Received: (qmail 84189 invoked by uid 99); 12 Jan 2011 19:58:51 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 Jan 2011 19:58:51 +0000
X-ASF-Spam-Status: No, hits=-2.3 required=10.0
	tests=RCVD_IN_DNSWL_MED,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of awittenauer@linkedin.com
 designates 69.28.149.24 as permitted sender)
Received: from [69.28.149.24] (HELO esv4-mav02.corp.linkedin.com)
 (69.28.149.24)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 Jan 2011 19:58:46 +0000
DomainKey-Signature: s=prod; d=linkedin.com; c=nofws; q=dns;
  h=X-IronPort-AV:Received:From:To:Subject:Thread-Topic:
   Thread-Index:Date:Message-ID:References:In-Reply-To:
   Accept-Language:Content-Language:X-MS-Has-Attach:
   X-MS-TNEF-Correlator:x-originating-ip:Content-Type:
   Content-ID:Content-Transfer-Encoding:MIME-Version;
  b=c2oPovqi9HBl/xe+tu1DF7Ac+Hkjz/JNgcB7kCyWpgzlnACuGmpSzazG
   drHjMAQOuiS4UfIDrW1rE6lq+oVV6zsaKIDyjf1H16eyZTcanx6jjNqhi
   yYxnK9g9R6ZDczq;
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=linkedin.com; i=awittenauer@linkedin.com; q=dns/txt;
  s=proddkim; t=1294862323; x=1326398323;
  h=from:sender:reply-to:subject:date:message-id:to:cc:
   mime-version:content-transfer-encoding:content-id:
   content-description:resent-date:resent-from:resent-sender:
   resent-to:resent-cc:resent-message-id:in-reply-to:
   references:list-id:list-help:list-unsubscribe:
   list-subscribe:list-post:list-owner:list-archive;
  z=From:=20Allen=20Wittenauer=20<awittenauer@linkedin.com>
   |Subject:=20Re:=20rsync=20+=20hdfs|Date:=20Wed,=2012=20Ja
   n=202011=2019:58:25=20+0000|Message-ID:=20<A37C4672-39DF-
   4405-B79B-3CBE17B2D357@linkedin.com>|To:=20"<hdfs-user@ha
   doop.apache.org>"=20<hdfs-user@hadoop.apache.org>
   |MIME-Version:=201.0|Content-Transfer-Encoding:=20quoted-
   printable|Content-ID:=20<0F99E450C9894143B907A46D064D8F42
   @linkedin.com>|In-Reply-To:=20<AANLkTinPsDoC1hbEws2cPTbnB
   Jp0Ae-DJ6rDo1WJFpMR@mail.gmail.com>|References:=20<AANLkT
   inPsDoC1hbEws2cPTbnBJp0Ae-DJ6rDo1WJFpMR@mail.gmail.com>;
  bh=Byy/KcLiLPbISVScrFKq/a3PZ6rA2JlQ/WWJdcuGu0w=;
  b=BaYOOJ5+OiZl7VQsgwWHd96N7eC3IPBG5qOfkbhFnJxeE1lPWVtDXiRR
   cfbXUAmhaFJfaHcFWVSxKdWXSYoSxs/IUo/m/i4sTWZuG2jqrf0ELEyQw
   omrv3WXTulIwxTz;
X-IronPort-AV: E=Sophos;i="4.60,314,1291622400";
   d="scan'208";a="19705215"
Received: from ESV4-EXC01.linkedin.biz ([fe80::d7c:dc04:aea1:97d7]) by
 esv4-cas02.linkedin.biz ([172.18.46.142]) with mapi id 14.01.0251.000; Wed,
 12 Jan 2011 11:58:26 -0800
From: Allen Wittenauer <awittenauer@linkedin.com>
To: "<hdfs-user@hadoop.apache.org>" <hdfs-user@hadoop.apache.org>
Subject: Re: rsync + hdfs
Thread-Topic: rsync + hdfs
Thread-Index: AQHLsgESKPPfngtnDkS96mOqHV4PGZPOSD+A
Date: Wed, 12 Jan 2011 19:58:25 +0000
Message-ID: <A37C4672-39DF-4405-B79B-3CBE17B2D357@linkedin.com>
References: <AANLkTinPsDoC1hbEws2cPTbnBJp0Ae-DJ6rDo1WJFpMR@mail.gmail.com>
In-Reply-To: <AANLkTinPsDoC1hbEws2cPTbnBJp0Ae-DJ6rDo1WJFpMR@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [172.18.46.247]
Content-Type: text/plain; charset="us-ascii"
Content-ID: <0F99E450C9894143B907A46D064D8F42@linkedin.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0


On Jan 11, 2011, at 6:32 PM, Mag Gam wrote:

> Is there a tool similar to rsync for hdfs? I would like to check for:
> ctime and size (the default behavior of rsync) and then sync if
> necessary. This would be a nice feature to add in the dfs utilities.


distcp is the closest.