Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of harsh@cloudera.com designates
 209.85.210.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <51038155.2010700@cse.ohio-state.edu>
References: <51038155.2010700@cse.ohio-state.edu>
From: Harsh J <harsh@cloudera.com>
Date: Sat, 26 Jan 2013 21:30:53 +0530
Message-ID: 
 <CAOcnVr1QMtmPa2ryeT=JaJ_Y==7ncpoEB5o5OcgQFnJYD=DM9Q@mail.gmail.com>
Subject: Re: Executing a Python program inside Map Function
To: "<user@hadoop.apache.org>" <user@hadoop.apache.org>
Content-Type: text/plain; charset=ISO-8859-1

Java provides the Process class to help you launch and read/write
from/to processes:
http://docs.oracle.com/javase/6/docs/api/java/lang/Process.html. You
can use this to spawn your program from your code, to write input into
the process's stdin, and to read its output via its stdout/etc.. The
hadoop-streaming parts of Apache Hadoop is very similar in its
operations - but allows little control back on the launched java map
class which you seem to require.

The tasks (both M and R types) provide entry and exit API points
(configure()/setup(), and cleanup()) - allowing you to spawn a process
before map-reads start, and end it after, letting you manage your
spawned process more cleanly.

On Sat, Jan 26, 2013 at 12:40 PM, Sundeep Kambhampati
<kambhamp@cse.ohio-state.edu> wrote:
> Is it possible to run a python script inside a Map function which is in
> java?
>
> I what to to run a python script which is on my local disk and I want to use
> the output of that script for further processing in Map Function to produce
> <key/Value> Pairs.
> Can some give me some idea how to do it.
>
>
> Regards
> Sundeep


-- 
Harsh J