Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4FD3FD9AD for ; Thu, 14 Mar 2013 11:29:56 +0000 (UTC) Received: (qmail 11669 invoked by uid 500); 14 Mar 2013 11:29:51 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 11296 invoked by uid 500); 14 Mar 2013 11:29:48 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 11259 invoked by uid 99); 14 Mar 2013 11:29:47 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 14 Mar 2013 11:29:47 +0000 X-ASF-Spam-Status: No, hits=2.9 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [98.139.212.167] (HELO nm8.bullet.mail.bf1.yahoo.com) (98.139.212.167) by apache.org (qpsmtpd/0.29) with SMTP; Thu, 14 Mar 2013 11:29:39 +0000 Received: from [98.139.215.140] by nm8.bullet.mail.bf1.yahoo.com with NNFMP; 14 Mar 2013 11:29:18 -0000 Received: from [98.139.212.213] by tm11.bullet.mail.bf1.yahoo.com with NNFMP; 14 Mar 2013 11:29:17 -0000 Received: from [127.0.0.1] by omp1022.mail.bf1.yahoo.com with NNFMP; 14 Mar 2013 11:29:17 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 973968.72328.bm@omp1022.mail.bf1.yahoo.com Received: (qmail 74487 invoked by uid 60001); 14 Mar 2013 11:29:17 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rocketmail.com; s=s1024; t=1363260557; bh=1FNKK6I5X0rQP0mdiaPwvxSPT9YK2bAMYNrIlgUUkiY=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=exSdOVwnhlhQZ4TzXa9DCsdF+08TXk8ZZDKd2ruMMIvfiGZoUs32CTqU7Yld2+ey68YguOzaa/TlO8uOnqjB5YDYexsidGZEtG42/d+scPujCcs9tIxkQemMBmQVJnG/TJ50FI/KaeAx6nYqpiyDs4IOWWpMtg8cRuWk8KsdNS0= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=rocketmail.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=FhYek46IvgeszmMHPgMfIkG8UpjwRol8SOUSLrYTmMe07eiy4LhdcgmIo4iQUPSHqiPBv6EFf7EBmiBLLwKx8sqzsmAo88mvHoVrfEDoV44oH01GDhby0NwoOQr1mhO6U1kHM0aLL7WSEXnfRD8wH/K880v/Jzi7rbDyqPK1hVQ=; X-YMail-OSG: eXplP6MVM1lWnO_osnMBLFCHR99k6a8lb00j1h3kixu_mrc _HCZJTr3S2WundldpSz0mYik.ekNYGektpuUPasojQeduVeIJIkJyryOvIgq YF7gwT80pWKkh2rYazfQ9XZBBSoAoq8mesvUtFJeYXcuv.qwENnh29wkMnEC L5hvDQQguKQnWiwqaNIaxemhw5.djk66IrO9nPa8E6bskJxw7VtLsC80xqIV LFAdEhN9dLO2TbdydSWXzBkkDGJLqBBoaJOHQ0FnOf81f_Sne8zhGuFH1UKk VZNERbkyD4n9rYBZcQAw4QXlJfgXdpxDuWQ7njA456qG1hbeQexc2d1Bau5H n.3CvMxQ9hpI08_azxqSp9qbnWDpDuImU5h6m0E_twuw5CN9yPNs_uWSeWK6 kirCErIh7gUVssvyLoM72Hkl0t5GRcjuESdrsTGtZdJsAXWb3MPXc5N.v.BF xPhO71cicSCC7DgzXyAhp Received: from [112.79.40.38] by web161904.mail.bf1.yahoo.com via HTTP; Thu, 14 Mar 2013 04:29:17 PDT X-Rocket-MIMEInfo: 002.001,U2FpLApFYWNoIGZpbGUgaXMgZGl2aWRlZCBpbnRvIHNwbGl0IGFzIHBlciB0aGUgbWFwIGlucHV0IGZvcm1hdCwgZWFjaCBzcGxpdCBpcyBlcXVhbCB0byBhIG1hcC4gWW91IHJpZ2h0bHkgc3RhdGVkIDEgc3BsaXQ9MSBibG9jaz0xIG1hcC4gUmVjb3JkIGNhbiBiZSBjb21iaW5hdGlvbiBvZiBibG9jayBkZWZpbmVkIGJ5IHJlY29yZHJlYWRlciBjb2RlLiBPbmUgcmVjb3JkIGNhbiBiZSBzZXJpZXMgb2YgbWFwcyBvciBzcGxpdHMgb3IgYmxvY2tzLiAKCkhvcGUgdGhpcyB3aWxsIGNsZWFyLiAKClNlbnQgZnIBMAEBAQE- X-Mailer: YahooMailWebService/0.8.137.519 Message-ID: <1363260557.74062.androidMobile@web161904.mail.bf1.yahoo.com> Date: Thu, 14 Mar 2013 04:29:17 -0700 (PDT) From: Manish Bhoge Subject: Re: Block vs FileSplit vs record vs line To: "user@hadoop.apache.org" , "saigraph@yahoo.in" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="1435348337-1483368872-1363260557=:74062" X-Virus-Checked: Checked by ClamAV on apache.org --1435348337-1483368872-1363260557=:74062 Content-Type: text/plain; charset=us-ascii Sai, Each file is divided into split as per the map input format, each split is equal to a map. You rightly stated 1 split=1 block=1 map. Record can be combination of block defined by recordreader code. One record can be series of maps or splits or blocks. Hope this will clear. Sent from HTC via Rocket! excuse typo. --1435348337-1483368872-1363260557=:74062 Content-Type: text/html; charset=us-ascii

Sai,
Each file is divided into split as per the map input format, each split is equal to a map. You rightly stated 1 split=1 block=1 map. Record can be combination of block defined by recordreader code. One record can be series of maps or splits or blocks.

Hope this will clear.

Sent from HTC via Rocket! excuse typo.



From: Sai Sai <saigraph@yahoo.in>;
To: user@hadoop.apache.org <user@hadoop.apache.org>;
Subject: Re: Block vs FileSplit vs record vs line
Sent: Thu, Mar 14, 2013 8:45:53 AM

Just wondering if this is right way to understand this:
A large file is split into multiple blocks and each block is split into multiple file splits and each file split has multiple records and each record has multiple lines. Each line is processed by 1 instance of mapper.
Any help is appreciated.
Thanks
Sai



--1435348337-1483368872-1363260557=:74062--