Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of harsh@cloudera.com designates
 209.85.192.180 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAFWc0y0+PLVMV7aig5ZcesWr=huOeDg7P1phv+VzPSkgkb9TPw@mail.gmail.com>
References: 
 <CAFWc0y0+PLVMV7aig5ZcesWr=huOeDg7P1phv+VzPSkgkb9TPw@mail.gmail.com>
From: Harsh J <harsh@cloudera.com>
Date: Fri, 31 May 2013 22:33:31 +0530
Message-ID: 
 <CAOcnVr2hY7xSkMVX9NqVHhR=yfjt0x8S=xwqX3J2YyBrOLDCUg@mail.gmail.com>
Subject: Re: possible to change replication factor at file creation time (with
 copyFromLocal)?
To: "<user@hadoop.apache.org>" <user@hadoop.apache.org>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

Hi Julian,

Yes, "dfs" subcommand accepts config overrides via -D. Just do "hadoop
dfs -Ddfs.replication=3DX -copyFromLocal =85".

On Fri, May 31, 2013 at 10:27 PM, Julian Bui <julianbui@gmail.com> wrote:
> Hi hadoop users,
>
> I am aware that you can set the replication factor of a file after it's b=
een
> created, but can you do it as you copy files to the HDFS?  My hope/intuit=
ion
> is that if you were able to reduce the replication factor of a file while
> copying, the copy time would decrease.  I'm finding it difficult waiting =
for
> large data sets to copy over.
>
> I am currently doing:
>
> hadoop dfs -copyFromLocal "/copy/from/path/" input
>
> and am wondering if it's possible to also specify something like -setrep =
on
> the same line.  -setsrep requires you to specify the file, which implies
> that it has to exist first, requiring two separate commands.
>
> Thanks in advance,
> -Julian


--=20
Harsh J