Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of harsh@cloudera.com designates
 209.85.213.182 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <007501cf62cd$60603810$2120a830$@126.com>
References: <004e01cf6291$104040a0$30c0c1e0$@126.com>
 <CAOcnVr0ZPz14C_rXbePwhc2r4OdwNj94o_bCi2ya_-6PKH_jUQ@mail.gmail.com>
 <007501cf62cd$60603810$2120a830$@126.com>
From: Harsh J <harsh@cloudera.com>
Date: Tue, 29 Apr 2014 05:01:29 +0530
Message-ID: 
 <CAOcnVr2gZozebUkW0cN5kRAS9t_hBRknKPO4t5coR8m7pjkepg@mail.gmail.com>
Subject: =?GB2312?B?UmU6ILTwuLQ6IGhkZnMgd3JpdGUgcGFydGlhbGx5?=
To: "<user@hadoop.apache.org>" <user@hadoop.apache.org>
Content-Type: text/plain; charset=GB2312
Content-Transfer-Encoding: quoted-printable

You do not need to alter the packet size to write files - why do you
think you need larger packets than the default one?

On Mon, Apr 28, 2014 at 4:04 PM,  <tdhkx@126.com> wrote:
> Hi Harsh,
>
>
>
> I=A1=AFm using HDFS client to write GZIP compressed files, I want to writ=
e once a
> file, in order to not uncompressing  it. So I should make every write
> completely, otherwise file will corrupted.
>
> I=A1=AFm raising up the client=A1=AFs write packet size to avoid partiall=
y write. But
> it doesn=A1=AFt work, since it can=A1=AFt bigger than 16M(file size > 16M=
).
>
> That=A1=AFs my problem.
>
>
>
> Thank a lot for replying.
>
>
>
> Regards,
>
> Ken Huang
>
>
>
> =B7=A2=BC=FE=C8=CB: user-return-15182-tdhkx=3D126.com@hadoop.apache.org
> [mailto:user-return-15182-tdhkx=3D126.com@hadoop.apache.org] =B4=FA=B1=ED=
 Harsh J
> =B7=A2=CB=CD=CA=B1=BC=E4: 2014=C4=EA4=D4=C228=C8=D5 13:30
> =CA=D5=BC=FE=C8=CB: <user@hadoop.apache.org>
> =D6=F7=CC=E2: Re: hdfs write partially
>
>
>
> Packets are chunks of the input you try to pass to the HDFS writer. What
> problem are you exactly facing (or, why are you trying to raise up the
> client's write packet size)?
>
>
>
> On Mon, Apr 28, 2014 at 8:52 AM, <tdhkx@126.com> wrote:
>
> Hello everyone,
>
>
>
> Since the default dfs.client-write-packet-size is 64K and it can=A1=AFt b=
e bigger
> than 16M.
>
> So if write bigger than 16M a time, how to make sure it doesn=A1=AFt writ=
e
> partially ?
>
>
>
> Does anyone knows how to fix this?
>
>
>
> Thanks a lot.
>
>
>
> --
>
> Ken Huang
>
>
>
>
>
> --
> Harsh J


--=20
Harsh J