incubator-oozie-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deepak Reddy <dre...@couponsinc.com>
Subject RE: Oozie streaming script not found on nodes
Date Sun, 02 Oct 2011 23:32:03 GMT
I believe <file> refers to the hdfs location.

This pattern works for me

<mapper>filename</mapper>
...
...
<file>filename</file>
<file>depfile1</file>
<file>depfile2>/file>



-DR

-----Original Message-----
From: Sridhar Narayanan [mailto:srinar@yahoo-inc.com] 
Sent: Friday, September 30, 2011 4:39 AM
To: oozie-users@incubator.apache.org
Subject: Oozie streaming script not found on nodes

I am running a workflow- currently only single actions at a time for diagnostics.
                                    Workflow looks like(relevant section):
<action name='preclean'>
            <streaming>
                <mapper>python mapper1 </mapper>
                <reducer>python reducer1 </reducer>
            </streaming>
                     ...
                     ...
 <file>$path/mapper1#mapper1</file>
            <file>$path/reducer1#reducer1</file>
            <file>$path/tools.py#tools.py</file>
            <file>$path/streaming.py#streaming.py</file>
Note: $path and all other variables are defined in the config file.
         Files exist at these locations
         All action stages work executed through a script On running the workflow(OOZIE URL
set, correct config file,
updated(3.0.2) version of oozie, examples run fine), ALL map tasks fail and the following
is the error :
                                                          Error:
File "reducer1", line 10, in ?
    import streaming
ImportError: No module named streaming
java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1
                                                        Questions:
1. Does the path of the file point to a HDFS location or home?
(currently points to the HDFS location, which is a mirror image of home, for this purpose)
2. Is this a permissions problem?
3. Does forcing sim-links create this problem?(I have had earlier problems because the mapper
cannot follow sim-links, but either way doesn't change the error here) (All paths are absolute,
and correct. All scripts work. All archives(where used) are uncorrupted) Advice is much appreciated.
Please feel free to ask for additional information.


Mime
View raw message