hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jamal sasha <jamalsha...@gmail.com>
Subject executing linux command from hadoop (python)
Date Thu, 15 Aug 2013 23:23:52 GMT
 Lets say that I have a data which interacts with a rest api like

%curl hostname data

Now, I have the following script:

#!/usr/bin/env python
import sys,os

cmd = """curl http://localhost  --data  '"""
string = " "
for line in sys.stdin:
    line = line.rstrip(os.linesep)
    string += line

os.system(cmd + string+"'")

Now, if i give a sample file for data, and run the above script with

cat data.txt | python mapper.py

It works perfectly. But will this work if i execute on hadoop as well?
I am trying to set up hadoop on local mode to check it out but I think it
will take me sometime to get there?
Any experiences, suggestions?

View raw message