hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pv...@iiit.ac.in
Subject Re: scp to namenode faster than dfs put?
Date Wed, 17 Sep 2008 14:34:27 GMT
thanks for the replies. So looks like replication might be the real
overhead when compared to scp.

> Also dfs put copies multiple replicas unlike scp.
>
> Lohit
>
> On Sep 17, 2008, at 6:03 AM, "叶双明" <yeshuangming@gmail.com> wrote:
>
> Actually, No.
> As you said, I understand that "dfs -put" breaks the data into blocksand
> then copies to datanodes,
> but scp do not breaks the data into blocksand , and just copy the data to
> the namenode!
>
>
> 2008/9/17, Prasad Pingali <pvvpr@research.iiit.ac.in>:
>
> Hello,
>  I observe that scp of data to the namenode is faster than actually
> putting
> into dfs (all nodes coming from same switch and have same ethernet cards,
> homogenous nodes)? I understand that "dfs -put" breaks the data into
> blocks
> and then copies to datanodes, but shouldn't that be atleast as fast as
> copying data to namenode from a single machine, if not faster?
>
> thanks and regards,
> Prasad Pingali,
> IIIT Hyderabad.
>
>
>
>
>
> --
> Sorry for my english!!  明
> Please help me to correct my english expression and error in syntax
>
>
>




Mime
View raw message