Return-Path: Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: (qmail 15950 invoked from network); 19 Apr 2010 11:46:24 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 19 Apr 2010 11:46:24 -0000 Received: (qmail 44357 invoked by uid 500); 19 Apr 2010 11:46:23 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 44307 invoked by uid 500); 19 Apr 2010 11:46:22 -0000 Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-user@hadoop.apache.org Delivered-To: mailing list mapreduce-user@hadoop.apache.org Received: (qmail 44299 invoked by uid 99); 19 Apr 2010 11:46:22 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 19 Apr 2010 11:46:22 +0000 X-ASF-Spam-Status: No, hits=2.9 required=10.0 tests=HTML_MESSAGE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [74.125.82.48] (HELO mail-ww0-f48.google.com) (74.125.82.48) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 19 Apr 2010 11:46:15 +0000 Received: by wwb29 with SMTP id 29so2709040wwb.35 for ; Mon, 19 Apr 2010 04:45:54 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.220.141 with HTTP; Mon, 19 Apr 2010 04:45:31 -0700 (PDT) From: =?UTF-8?Q?Utku_Can_Top=C3=A7u?= Date: Mon, 19 Apr 2010 13:45:31 +0200 Received: by 10.216.91.16 with SMTP id g16mr7172263wef.102.1271677554280; Mon, 19 Apr 2010 04:45:54 -0700 (PDT) Message-ID: Subject: OutPutFormat for running the execution on each key:value pair To: mapreduce-user@hadoop.apache.org Content-Type: multipart/alternative; boundary=0016e6d7ec6ce451020484958040 X-Virus-Checked: Checked by ClamAV on apache.org --0016e6d7ec6ce451020484958040 Content-Type: text/plain; charset=UTF-8 Hello All, I've been looking for the documentation and the API docs, however I couldn't find an OutPutFormat that will go through all the reduce emits as key:value pairs by running an execute method for each pair. Say, I've reduced hostnames as keys, and want to ping every hostname that I've reduced as key and store the results of pings in a MySQL table or so. So basically, What I'm thinking of was such a method like: outputExec(K key, V value){ ... pingResult = ping(key) writetoMysql(value,pingResult) .... } Is there such an OutPutFormat implementation currently in Hadoop? If not so, I'd like to start doing something that fits my needs and try to contribute in to the project, but I think I'll need some guidance beforehand dealing with the OutPutFormat. Best Regards, Utku --0016e6d7ec6ce451020484958040 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hello All,

I've been looking for the documentation and the API d= ocs,
however I couldn't find an OutPutFormat that will go through a= ll the reduce emits as key:value pairs by running an execute method for eac= h pair.

Say, I've reduced hostnames as keys, and want to ping every hostnam= e that I've reduced as key and store the results of pings in a MySQL ta= ble or so.

So basically,
What I'm thinking of was such a meth= od like:

outputExec(K key, V value){
...
pingResult =3D ping(key)
write= toMysql(value,pingResult)
....
}

Is there such an OutPutFormat= implementation currently in Hadoop? If not so, I'd like to start doing= something that fits my needs and try to contribute in to the project, but = I think I'll need some guidance beforehand dealing with the OutPutForma= t.

Best Regards,
Utku




--0016e6d7ec6ce451020484958040--