Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 76CF311446 for ; Mon, 28 Apr 2014 05:31:00 +0000 (UTC) Received: (qmail 53699 invoked by uid 500); 28 Apr 2014 05:30:51 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 52949 invoked by uid 500); 28 Apr 2014 05:30:42 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 52942 invoked by uid 99); 28 Apr 2014 05:30:41 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Apr 2014 05:30:41 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of harsh@cloudera.com designates 209.85.223.172 as permitted sender) Received: from [209.85.223.172] (HELO mail-ie0-f172.google.com) (209.85.223.172) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Apr 2014 05:30:36 +0000 Received: by mail-ie0-f172.google.com with SMTP id at1so2746881iec.3 for ; Sun, 27 Apr 2014 22:30:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=D9/hWwC+a3ARxCU4HqU47rBhr06eSOMxOmEZOpmSwjI=; b=g11NQUfr2ouMzMyswvA+wEi39U8W8j9vSMVRP6PdHLahqaiv0TOjzVQ8RHinRslS3K bZvaJsLb7Ig58+j1g1GsxuAAWzuwyCmjjjlzpHxo9KHrtmU04idl0tP/0tFMtGnHmalC gSzdnNQX1mdeEqXlQASk3QE0tJGydL/4GXET9/O237RC5JaDmnWg5c9/XM8Jt8NISbB4 28eLmcwko8oUsXDfvi3jj1WiOPwCY9y65hnSdtDXbIhe8TAQE8tEF1vxa8g2V8D1aDP1 Dd3X3hufKKghU+OCZ3AMAHjolcm2hoBoTZu2UEKCROFA5dEuMEANpFaYTC9PZZO79tPe aC+Q== X-Gm-Message-State: ALoCoQk4ejicecHKXjYgE0rSAGLKK5WDsrevNvXvh6kKcErDeoVb9BlEB1yWWdP0g3qkhWlSg6LE X-Received: by 10.42.99.138 with SMTP id w10mr21143750icn.42.1398663012993; Sun, 27 Apr 2014 22:30:12 -0700 (PDT) MIME-Version: 1.0 Received: by 10.50.50.168 with HTTP; Sun, 27 Apr 2014 22:29:52 -0700 (PDT) In-Reply-To: <004e01cf6291$104040a0$30c0c1e0$@126.com> References: <004e01cf6291$104040a0$30c0c1e0$@126.com> From: Harsh J Date: Mon, 28 Apr 2014 10:59:52 +0530 Message-ID: Subject: Re: hdfs write partially To: "" Content-Type: multipart/alternative; boundary=20cf301cc4c60ce08d04f8139c7f X-Virus-Checked: Checked by ClamAV on apache.org --20cf301cc4c60ce08d04f8139c7f Content-Type: text/plain; charset=ISO-8859-1 Packets are chunks of the input you try to pass to the HDFS writer. What problem are you exactly facing (or, why are you trying to raise up the client's write packet size)? On Mon, Apr 28, 2014 at 8:52 AM, wrote: > Hello everyone, > > > > Since the default dfs.client-write-packet-size is 64K and it can't be > bigger than 16M. > > So if write bigger than 16M a time, how to make sure it doesn't write > partially ? > > > > Does anyone knows how to fix this? > > > > Thanks a lot. > > > > -- > > Ken Huang > -- Harsh J --20cf301cc4c60ce08d04f8139c7f Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Packets are chunks of the input you try to pass to the HDF= S writer. What problem are you exactly facing (or, why are you trying to ra= ise up the client's write packet size)?


On Mon, Apr 28, 2014 at 8:52 AM, <tdhkx@12= 6.com> wrote:

Hello everyone,

 

Since <= a name=3D"145a65b9598b698c_dfs.client-write-packet-size">the default dfs.client-write-packet-size is 64K and it can’t be bigger than 16M.<= /p>

So if write bigger than 16M a t= ime, how to make sure it doesn’t write partially ?

 <= u>

Does anyone knows ho= w to fix this?

 

Thanks a lot.

 =

--

<= span lang=3D"EN-US" style=3D"font-size:10.5pt;font-family:"Calibri&quo= t;,"sans-serif";color:#1f497d">Ken Huang




-= -
Harsh J
--20cf301cc4c60ce08d04f8139c7f--