Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 91FF4EAA3 for ; Wed, 6 Mar 2013 05:55:01 +0000 (UTC) Received: (qmail 18066 invoked by uid 500); 6 Mar 2013 05:54:56 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 17951 invoked by uid 500); 6 Mar 2013 05:54:55 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 17932 invoked by uid 99); 6 Mar 2013 05:54:55 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 Mar 2013 05:54:55 +0000 X-ASF-Spam-Status: No, hits=-1.8 required=5.0 tests=FRT_ADOBE2,HTML_MESSAGE,RCVD_IN_DNSWL_HI,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [81.252.56.10] (HELO airbus-sf2.airbus.gmessaging.net) (81.252.56.10) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 Mar 2013 05:54:48 +0000 Received: from airbus-sf2.airbus.gmessaging.net (localhost.localdomain [127.0.0.1]) by localhost.airbus.gmessaging.net (Postfix) with SMTP id 7079312684D9 for ; Wed, 6 Mar 2013 06:54:26 +0100 (CET) Received: from de0-mailhub-p11.res.airbus.corp (unknown [44.225.3.152]) by airbus-sf2.airbus.gmessaging.net (Postfix) with ESMTP id 253471268215 for ; Wed, 6 Mar 2013 06:54:26 +0100 (CET) Received: from DE0-MAILMBX-P21.res.airbus.corp ([44.225.3.150]) by de0-mailhub-p11.res.airbus.corp ([44.225.3.152]) with mapi; Wed, 6 Mar 2013 06:54:26 +0100 From: "AMARNATH, Balachandar" To: "user@hadoop.apache.org" Date: Wed, 6 Mar 2013 06:54:23 +0100 Subject: RE: Map reduce technique Thread-Topic: Map reduce technique Thread-Index: Ac4aLKfQwFEyjg2oRPi50aFTOdx8LwAAVkHwAABARVA= Message-ID: <26415_1362549266_5136DA12_26415_5996_1_E6472BD072676843ADB8D37E50F4C88D0BF865686D@DE0-MAILMBX-P21.res.airbus.corp> References: <29267_1362548240_5136D610_29267_11841_1_E6472BD072676843ADB8D37E50F4C88D0BF8656859@DE0-MAILMBX-P21.res.airbus.corp> <29267_1362549128_5136D987_29267_12576_1_535B028B4629374D86894192BE5184DD273B4328@indiambx02.corp.adobe.com> In-Reply-To: <29267_1362549128_5136D987_29267_12576_1_535B028B4629374D86894192BE5184DD273B4328@indiambx02.corp.adobe.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_E6472BD072676843ADB8D37E50F4C88D0BF865686DDE0MAILMBXP21_" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_E6472BD072676843ADB8D37E50F4C88D0BF865686DDE0MAILMBXP21_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Thanks for the mail, Can u please share few links to start with? Regards Bala From: Samir Kumar Das Mohapatra [mailto:dasmohap@adobe.com] Sent: 06 March 2013 11:21 To: user@hadoop.apache.org Subject: RE: Map reduce technique I think you have to look the sequence file as input format . Basically, the way this works is, you will have a separate Java process tha= t takes several image files, reads the ray bytes into memory, then stores t= he data into a key-value pair in a SequenceFile. Keep going and keep writin= g into HDFS. This may take a while, but you'll only have to do it once. Regards, Samir. From: AMARNATH, Balachandar [mailto:BALACHANDAR.AMARNATH@airbus.com] Sent: 06 March 2013 11:07 To: user@hadoop.apache.org Subject: Map reduce technique Hi, I am new to map reduce paradigm. I read in a tutorial that says that 'map' = function splits the data and into key value pairs. This means, the map-redu= ce framework automatically splits the data into pieces or do we need to exp= licitly provide the method to split the data into pieces. If it does automa= tically, how it splits an image file (size etc)? I see, processing of an im= age file as a whole will give different results than processing them in chu= nks. With thanks and regards Balachandar The information in this e-mail is confidential. The contents may not be dis= closed or used by anyone other than the addressee. Access to this e-mail by= anyone else is unauthorised. If you are not the intended recipient, please notify Airbus immediately and= delete this e-mail. Airbus cannot accept any responsibility for the accuracy or completeness of= this e-mail as it has been sent over public networks. If you have any conc= erns over the content of this message or its Accuracy or Integrity, please = contact Airbus immediately. All outgoing e-mails from Airbus are checked using regularly updated virus = scanning software but you should take whatever measures you deem to be appr= opriate to ensure that this message and any attachments are virus free. The information in this e-mail is confidential. The contents may not be dis= closed or used by anyone other than the addressee. Access to this e-mail by= anyone else is unauthorised. If you are not the intended recipient, please notify Airbus immediately and= delete this e-mail. Airbus cannot accept any responsibility for the accuracy or completeness of= this e-mail as it has been sent over public networks. If you have any conc= erns over the content of this message or its Accuracy or Integrity, please = contact Airbus immediately. All outgoing e-mails from Airbus are checked using regularly updated virus = scanning software but you should take whatever measures you deem to be appr= opriate to ensure that this message and any attachments are virus free. --_000_E6472BD072676843ADB8D37E50F4C88D0BF865686DDE0MAILMBXP21_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Thanks for = the mail,

 =

Can u please share few links to start = with?

 

 

Regards

Bala=

 

From:= Samir Kumar Das Mohapatra [mailto:dasmohap@adobe.com]
= Sent: 06 March 2013 11:21
To: user@hadoop.apache.org
Subject: RE: Map reduce technique

 

I think &nbs= p;you have to look the sequence file  as input format .

 

= Basically, the way this works is, you will have a separate Java process tha= t takes several image files, reads the ray bytes into memory, then stores t= he data into a key-value pair in a SequenceFile. Keep going and keep writin= g into HDFS. This may take a while, but you'll only have to do it once.

 <= /o:p>

Regards,<= /o:p>

Samir.<= span lang=3DEN-US style=3D'font-size:11.0pt;font-family:"Calibri","sans-ser= if";color:#1F497D'>

 <= /o:p>

From: AMARNATH, Balachandar [mailto:BALACHANDA= R.AMARNATH@airbus.com]
Sent: 06 March 2013 11:07
To: u= ser@hadoop.apache.org
Subject: Map reduce technique

 

Hi,

 

I am new to ma= p reduce paradigm. I read in a tutorial that says that ‘map’ fu= nction splits the data and into key value pairs. This means, the map-reduce= framework automatically splits the data into pieces or do we need to expli= citly provide the method to split the data into pieces. If it does automati= cally, how it splits an image file (size etc)? I see, processing of an imag= e file as a whole will give different results than processing them in chunk= s.

 

 

<= /div>

 

With thanks and regards<= /p>

Bala= chandar

 

 

 

The information in this e-mail i=
s confidential. The contents may not be disclosed or used by anyone other t=
han the addressee. Access to this e-mail by anyone else is unauthorised.
I=
f you are not the intended recipient, please notify Airbus immediately and =
delete this e-mail.
Airbus cannot accept any responsibility for the accura=
cy or completeness of this e-mail as it has been sent over public networks.=
 If you have any concerns over the content of this message or its Accuracy =
or Integrity, please contact Airbus immediately.
All outgoing e-mails from= Airbus are checked using regularly updated virus scanning software but you= should take whatever measures you deem to be appropriate to ensure that th= is message and any attachments are virus free.
The information in this e-mail is confidential. The contents may not =
be disclosed or used by anyone other than the addressee. Access to this e-m=
ail by anyone else is unauthorised.
If you are not the intended recipient, please notify Airbus immediately and=
 delete this e-mail.
Airbus cannot accept any responsibility for the accuracy or completeness of=
 this e-mail as it has been sent over public networks. If you have any conc=
erns over the content of this message or its Accuracy or Integrity, please =
contact Airbus immediately.
All outgoing e-mails from Airbus are checked using regularly updated virus =
scanning software but you should take whatever measures you deem to be appr=
opriate to ensure that this message and any attachments are virus free.
= --_000_E6472BD072676843ADB8D37E50F4C88D0BF865686DDE0MAILMBXP21_--