Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2F03210D23 for ; Thu, 8 Aug 2013 10:51:16 +0000 (UTC) Received: (qmail 81509 invoked by uid 500); 8 Aug 2013 10:51:11 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 81415 invoked by uid 500); 8 Aug 2013 10:51:10 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 81408 invoked by uid 99); 8 Aug 2013 10:51:10 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 08 Aug 2013 10:51:10 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of harsh@cloudera.com designates 209.85.219.52 as permitted sender) Received: from [209.85.219.52] (HELO mail-oa0-f52.google.com) (209.85.219.52) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 08 Aug 2013 10:51:03 +0000 Received: by mail-oa0-f52.google.com with SMTP id n12so417911oag.11 for ; Thu, 08 Aug 2013 03:50:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=MBtOdNk1kaeAaGAJZ9yVUNWWHPN+kQbFRHNQbg9tklk=; b=RKrD+csOrkERpK9QuHbO0pMLvopyIOY0oId7oLfDY+/BdKSnZ6GJxx8NUOusK7LqxO OFajAf8ojB6GJ/46EGocpbIskTtRclIl1cDg/aJTg0Mpduc2RI4ykHImRzOIYCrQQW4o aAYPIZ3nN+o/Neg42vTfByvi7FRrLEcSEiJ9Vf4roOaneZAfbV1uVlAzflGdtUY8Qq7R Tr8BF7eiw1S3MPhh6HEg1Up+I6SJLhbZpSiGXIW+7tFIkPnVXkgNOMzyKDCWiChVnGTn zdWpiuyCVZjDhQPdTo4Xq7ZhMa0f3ow4HT96HZdAh/ZKN8OSPX1EILHjWeORzLvOjG+s Grkg== X-Gm-Message-State: ALoCoQldBlzzJd4sm4rNpkl8zG2SBYa8oV12Vpm0dqQzYvLh4gw2Sq8nFAruCmO9QVSnXfZ2Rb/H MIME-Version: 1.0 X-Received: by 10.60.161.136 with SMTP id xs8mr3802614oeb.77.1375959042684; Thu, 08 Aug 2013 03:50:42 -0700 (PDT) Received: by 10.182.236.136 with HTTP; Thu, 8 Aug 2013 03:50:42 -0700 (PDT) Received: by 10.182.236.136 with HTTP; Thu, 8 Aug 2013 03:50:42 -0700 (PDT) In-Reply-To: References: Date: Thu, 8 Aug 2013 16:20:42 +0530 Message-ID: Subject: Re: Why LineRecordWriter.write(..) is synchronized From: Harsh J To: "" Content-Type: multipart/alternative; boundary=089e01183a18f70b7004e36d6d5e X-Virus-Checked: Checked by ClamAV on apache.org --089e01183a18f70b7004e36d6d5e Content-Type: text/plain; charset=ISO-8859-1 While we don't fork by default, we do provide a MultithreadedMapper implementation that would require such synchronization. But if you are asking is it necessary, then perhaps the answer is no. On Aug 8, 2013 3:43 PM, "Azuryy Yu" wrote: > its not hadoop forked threads, we may create a line record writer, then > call this writer concurrently. > On Aug 8, 2013 4:00 PM, "Sathwik B P" wrote: > >> Hi, >> Thanks for your reply. >> May I know where does hadoop fork multiple threads to use a single >> RecordWriter. >> >> regards, >> sathwik >> >> On Thu, Aug 8, 2013 at 7:06 AM, Azuryy Yu wrote: >> >>> because we may use multi-threads to write a single file. >>> On Aug 8, 2013 2:54 PM, "Sathwik B P" wrote: >>> >>>> Hi, >>>> >>>> LineRecordWriter.write(..) is synchronized. I did not find any other >>>> RecordWriter implementations define the write as synchronized. >>>> Any specific reason for this. >>>> >>>> regards, >>>> sathwik >>>> >>> >> --089e01183a18f70b7004e36d6d5e Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

While we don't fork by default, we do provide a Multithr= eadedMapper implementation that would require such synchronization. But if = you are asking is it necessary, then perhaps the answer is no.

On Aug 8, 2013 3:43 PM, "Azuryy Yu" &l= t;azuryyyu@gmail.com> wrote:

its not hadoop forked threads, we may create a line record w= riter, then call this writer concurrently.

On Aug 8, 2013 4:00 PM, "Sathwik B P" = <sathwik.bp@gm= ail.com> wrote:
Hi,
Thanks for your reply.
May I know where does hadoop fork multiple= threads to use a single RecordWriter.

regards,
sathwik

On Thu, Aug 8, 2013 at 7:06 AM, Azuryy Yu <azury= yyu@gmail.com> wrote:

because we may use multi-thre= ads to write a single file.

On Aug 8, 2013 2:54 PM, "Sathwik B P" = <sathwik@apache.= org> wrote:
Hi,

LineRecordWriter.write(..) is synchronized. I did not find any o= ther RecordWriter implementations define the write as synchronized.
Any = specific reason for this.

regards,
sathwik

--089e01183a18f70b7004e36d6d5e--