hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venky Iyer (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-4590) User-definable handlers for MAP and REDUCE transforms
Date Wed, 05 Nov 2008 09:23:44 GMT
User-definable handlers for MAP and REDUCE transforms
-----------------------------------------------------

                 Key: HADOOP-4590
                 URL: https://issues.apache.org/jira/browse/HADOOP-4590
             Project: Hadoop Core
          Issue Type: Wish
          Components: contrib/hive
            Reporter: Venky Iyer


Mappers can be specified (as before) like:

.... MAP USING 'uri' .....

uris are in a format to be decided upon; possibilities are

protocol://resource/param=value,param2=value2

or

protocol: resource_string

For example, shell commands are like 

sh://uniq or 
sh: sort | uniq

When no protocol is specified, we assume the default to be sh://.

Another example is pyfunc://foo.bar/baz=2 , which points to the bar(baz=2) function from the
foo module. 

We can add handlers for these protocols like

add handler sh shell (default)
add handler pyfunc "python pyhive.py"

and replace these handlers using appropriate syntax.

Map and Reduce handlers can be distinct. 



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message