Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id F3947F89D for ; Tue, 26 Mar 2013 04:20:15 +0000 (UTC) Received: (qmail 70175 invoked by uid 500); 26 Mar 2013 04:20:10 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 69870 invoked by uid 500); 26 Mar 2013 04:20:05 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 69833 invoked by uid 99); 26 Mar 2013 04:20:04 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Mar 2013 04:20:04 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [81.252.56.22] (HELO airbus-sf6.airbus.gmessaging.net) (81.252.56.22) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Mar 2013 04:19:58 +0000 Received: from airbus-sf6.airbus.gmessaging.net (localhost.localdomain [127.0.0.1]) by localhost.airbus.gmessaging.net (Postfix) with SMTP id 57C851FC0313 for ; Tue, 26 Mar 2013 05:19:36 +0100 (CET) Received: from de0-mailhub-p08.res.airbus.corp (unknown [44.225.0.176]) by airbus-sf6.airbus.gmessaging.net (Postfix) with ESMTP id 023181FC019C for ; Tue, 26 Mar 2013 05:19:36 +0100 (CET) Received: from DE0-MAILMBX-P21.res.airbus.corp ([44.225.3.142]) by de0-mailhub-p08.res.airbus.corp ([44.225.0.176]) with mapi; Tue, 26 Mar 2013 05:19:36 +0100 From: "AMARNATH, Balachandar" To: "user@hadoop.apache.org" Date: Tue, 26 Mar 2013 05:19:32 +0100 Subject: RE: Any answer ? Candidate application for map reduce Thread-Topic: Any answer ? Candidate application for map reduce Thread-Index: Ac4pklWs9R7vQK1VTAKZ3FVa0qhu3AARsC0A Message-ID: <7077_1364271576_515121D8_7077_9727_1_E6472BD072676843ADB8D37E50F4C88D0EB178647A@DE0-MAILMBX-P21.res.airbus.corp> References: <25038_1364190456_514FE4F8_25038_3301_1_E6472BD072676843ADB8D37E50F4C88D0EB16F2BC2@DE0-MAILMBX-P21.res.airbus.corp> <3A5F60F2-5FA6-40FC-9B8C-3C2211806D5F@gmail.com> <23734_1364203506_515017F2_23734_17018_1_CALDQvddp4A8WSCLsDyWkHEYq0R6u8XaBuMQAyS3PFMVSTMT7jQ@mail.gmail.com> <3607_1364209930_5150310A_3607_12056_1_E6472BD072676843ADB8D37E50F4C88D0EB16F3096@DE0-MAILMBX-P21.res.airbus.corp> <31693_1364241184_5150AB1F_31693_16251_1_CACBYxK+omrnZSb0LUSq2EeP55XowOKB34cMHq9-+hAo9S0biVQ@mail.gmail.com> In-Reply-To: <31693_1364241184_5150AB1F_31693_16251_1_CACBYxK+omrnZSb0LUSq2EeP55XowOKB34cMHq9-+hAo9S0biVQ@mail.gmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_E6472BD072676843ADB8D37E50F4C88D0EB178647ADE0MAILMBXP21_" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_E6472BD072676843ADB8D37E50F4C88D0EB178647ADE0MAILMBXP21_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable This is encouraging ! Thanks a lot, From: Sandy Ryza [mailto:sandy.ryza@cloudera.com] Sent: 26 March 2013 01:22 To: user@hadoop.apache.org Subject: Re: Any answer ? Candidate application for map reduce Hi Bala, A standard benchmark program for mapreduce is terasort, which is included i= n the hadoop examples jar. You can generate data for it using teragen, whi= ch runs a map-only job: hadoop jar path-to-examples-jar.jar and then sort the data using terasort: hadoop jar path-to-examples-jar.jar -Sandy On Mon, Mar 25, 2013 at 4:12 AM, AMARNATH, Balachandar > wrote: Hello Yanbo Liang, My issue is that I neither do not have data nor I have an application! I = was looking for an open source benchmark application to test this, preferab= ly, if the application (or simple test code) is related to image processing= domain Regards Bala From: Yanbo Liang [mailto:yanbohappy@gmail.com] Sent: 25 March 2013 14:55 To: user@hadoop.apache.org Subject: Re: Any answer ? Candidate application for map reduce >From your description "split the data in to chunks, feed the chunks to the = application, and merge the processed chunks to get A back" is just suit for= the MapReduce paradigm. First you can feed the split chunks to Mapper and = merge the processed chunks at Reducer. Why did you not use MapReduce paradi= gm? 2013/3/25 Raymond Tay > Hello Bala, I bumped into your email by chance. I'm not sure what you are after, really because if you wanted something lik= e image processing and feature detection then hadoop's mailing list is prob= ably not the place and you might want to join NVIDIA's or ATI's CUDA / Open= CL forums. Hope this helps Ray On 25 Mar, 2013, at 1:47 PM, "AMARNATH, Balachandar" > wrote: Any answers from anyone of you :) Regards Bala From: AMARNATH, Balachandar [mailto:BALACHANDAR.AMARNATH@airbus.com] Sent: 22 March 2013 10:25 To: user@hadoop.apache.org Subject: Candidate application for map reduce Hello, I am looking for an sample application (preferably image processing, featur= e detection etc) that can be good candidate of map reduce paradigm. To very= specific, I am looking for an open source simple application that process = data and produce some results (A). When you split the data in to chunks, fe= ed the chunks to the application, and merge the processed chunks to get A b= ack. Is there any website where can I look into such kind of benchmark appl= ications ? Any pointers and thoughts will be helpful here. With thanks and regards Balachandar The information in this e-mail is confidential. The contents may not be dis= closed or used by anyone other than the addressee. Access to this e-mail by= anyone else is unauthorised. If you are not the intended recipient, please notify Airbus immediately and= delete this e-mail. Airbus cannot accept any responsibility for the accuracy or completeness of= this e-mail as it has been sent over public networks. If you have any conc= erns over the content of this message or its Accuracy or Integrity, please = contact Airbus immediately. All outgoing e-mails from Airbus are checked using regularly updated virus = scanning software but you should take whatever measures you deem to be appr= opriate to ensure that this message and any attachments are virus free. The information in this e-mail is confidential. The contents may not be dis= closed or used by anyone other than the addressee. Access to this e-mail by= anyone else is unauthorised. If you are not the intended recipient, please notify Airbus immediately and= delete this e-mail. Airbus cannot accept any responsibility for the accuracy or completeness of= this e-mail as it has been sent over public networks. If you have any conc= erns over the content of this message or its Accuracy or Integrity, please = contact Airbus immediately. All outgoing e-mails from Airbus are checked using regularly updated virus = scanning software but you should take whatever measures you deem to be appr= opriate to ensure that this message and any attachments are virus free. The information in this e-mail is confidential. The contents may not be dis= closed or used by anyone other than the addressee. Access to this e-mail by= anyone else is unauthorised. If you are not the intended recipient, please notify Airbus immediately and= delete this e-mail. Airbus cannot accept any responsibility for the accuracy or completeness of= this e-mail as it has been sent over public networks. If you have any conc= erns over the content of this message or its Accuracy or Integrity, please = contact Airbus immediately. All outgoing e-mails from Airbus are checked using regularly updated virus = scanning software but you should take whatever measures you deem to be appr= opriate to ensure that this message and any attachments are virus free. The information in this e-mail is confidential. The contents may not be dis= closed or used by anyone other than the addressee. Access to this e-mail by= anyone else is unauthorised. If you are not the intended recipient, please notify Airbus immediately and= delete this e-mail. Airbus cannot accept any responsibility for the accuracy or completeness of= this e-mail as it has been sent over public networks. If you have any conc= erns over the content of this message or its Accuracy or Integrity, please = contact Airbus immediately. All outgoing e-mails from Airbus are checked using regularly updated virus = scanning software but you should take whatever measures you deem to be appr= opriate to ensure that this message and any attachments are virus free. --_000_E6472BD072676843ADB8D37E50F4C88D0EB178647ADE0MAILMBXP21_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

This is enc= ouraging ! Thanks a lot,

 

From: Sandy Ryza [mailto:sandy.ryza@cloude= ra.com]
Sent: 26 March 2013 01:22
To: user@hadoop.apac= he.org
Subject: Re: Any answer ? Candidate application for map re= duce

 

Hi Bala,

 

A standard benchmark program for mapreduce is terasort, which = is included in the hadoop examples jar.  You can generate data for it = using teragen, which runs a map-only job:

 

<= p class=3DMsoNormal style=3D'margin-left:36.0pt'>hadoop jar path-to-ex= amples-jar.jar <number of records> <directory to put them in>

<= o:p> 

and then sort the data using terasort:

 

hadoop jar path-to-exam= ples-jar.jar <input directory> <output directory>

 

-Sandy

On Mon, Mar 25, 201= 3 at 4:12 AM, AMARNATH, Balachandar <BALACHANDAR.AMARNATH@airbus.com> w= rote:

Hello Yanbo Li= ang,

 =

My issue is  that I neither d= o not have data nor I have an application!  I was looking for an open = source benchmark application to test this, preferably, if the application (= or simple test code) is related to image processing domain

 

Regards

= Bala

 =

 

<= p class=3DMsoNormal style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:= auto;margin-left:36.0pt'> 

From: Yanbo Liang [mailto:yanbohappy@gmail.com= ]
Sent: 25 March 2013 14:55
To: user@hadoop.apache.org
Subj= ect: Re: Any answer ? Candidate application for map reduce<= /o:p>

 

From your description "split the data in to chunks, feed the chunks= to the application, and merge the processed chunks to get A back" is = just suit for the MapReduce paradigm. First you can feed the split chunks t= o Mapper and merge the processed chunks at Reducer. Why did you not use Map= Reduce paradigm? 

2013/3/25 Raymond Tay <raymondtay1974@gmail.com><= /p>

Hello Bala,

 

I bumped into your email by chance. 

 

I'm not sure what you are after, really because if you wanted something l= ike image processing and feature detection then hadoop's mailing list is pr= obably not the place and you might want to join NVIDIA's or ATI's CUDA / Op= enCL forums.

 =

Hope this helps

Ray

 

On 25 Mar, 2013, at 1:47 PM, "AMARNATH, Balachandar" &= lt;BAL= ACHANDAR.AMARNATH@airbus.com> wrote:

 

Any answers from anyone of you J

 

 

Regards

Bala

 

From: AMARNATH, Balachandar [mailto= :BALACHANDAR.AMA= RNATH@airbus.com] 
Sent: 22 March 2013= 10:25
To: user@hadoop.apache.org<= br>Subject: Candidate application for map reduce

 

Hello,

=

 

I am looking for an sample application (preferably image= processing, feature detection etc) that can be good candidate of map reduc= e paradigm. To very specific, I am looking for an open source simple applic= ation that process data and produce some results (A). When you split the da= ta in to chunks, feed the chunks to the application, and merge the processe= d chunks to get A back. Is there any website where can I look into such kin= d of benchmark applications ? Any pointers and thoughts will be helpful her= e.

 

With thanks and = regards

Balac= handar

 =

 

 = 

The information in =
this e-mail is confidential. The contents may not be disclosed or used by a=
nyone other than the addressee. Access to this e-mail by anyone else is una=
uthorised.
If you are not=
 the intended recipient, please notify Airbus immediately and delete this e=
-mail.
 <=
/pre>
 
Airbus cannot accept any responsibility for the accura=
cy or completeness of this e-mail as it has been sent over public networks.=
 If you have any concerns over the content of this message or its Accuracy =
or Integrity, please contact Airbus immediately.
All outgoing e-mails from Airbus are checked using =
regularly updated virus scanning software but you should take whatever meas=
ures you deem to be appropriate to ensure that this message and any attachm=
ents are virus free.
The information in this e-mail is confidential. The contents may not be d=
isclosed or used by anyone other than the addressee. Access to this e-mail =
by anyone else is unauthorised.
If you are not the intended recipient, please notify Airbus immediat=
ely and delete this e-mail.
Airbus cannot accept any responsibility for the accuracy or completeness=
 of this e-mail as it has been sent over public networks. If you have any c=
oncerns over the content of this message or its Accuracy or Integrity, plea=
se contact Airbus immediately.
All outgoing e-mails from Airbus are checked using regularly updated =
virus scanning software but you should take whatever measures you deem to b=
e appropriate to ensure that this message and any attachments are virus fre=
e.

 <= /p>

 

The info=
rmation in this e-mail is confidential. The contents may not be disclosed o=
r used by anyone other than the addressee. Access to this e-mail by anyone =
else is unauthorised.
If =
you are not the intended recipient, please notify Airbus immediately and de=
lete this e-mail.
Airbus =
cannot accept any responsibility for the accuracy or completeness of this e=
-mail as it has been sent over public networks. If you have any concerns ov=
er the content of this message or its Accuracy or Integrity, please contact=
 Airbus immediately.
All =
outgoing e-mails from Airbus are checked using regularly updated virus scan=
ning software but you should take whatever measures you deem to be appropri=
ate to ensure that this message and any attachments are virus free.

 

The information in this e-mai=
l is confidential. The contents may not be disclosed or used by anyone othe=
r than the addressee. Access to this e-mail by anyone else is unauthorised.
If you are not the intended recipient, please notify Airbus immediately and=
 delete this e-mail.
Airbus cannot accept any responsibility for the accuracy or completeness of=
 this e-mail as it has been sent over public networks. If you have any conc=
erns over the content of this message or its Accuracy or Integrity, please =
contact Airbus immediately.
All outgoing e-mails from Airbus are checked using regularly updated virus =
scanning software but you should take whatever measures you deem to be appr=
opriate to ensure that this message and any attachments are virus free.
= --_000_E6472BD072676843ADB8D37E50F4C88D0EB178647ADE0MAILMBXP21_--