incubator-oozie-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sridhar Narayanan <sri...@yahoo-inc.com>
Subject Oozie streaming script not found on nodes
Date Fri, 30 Sep 2011 11:39:29 GMT
I am running a workflow- currently only single actions at a time for
diagnostics.
                                    Workflow looks like(relevant section):
<action name='preclean'>
            <streaming>
                <mapper>python mapper1 </mapper>
                <reducer>python reducer1 </reducer>
            </streaming>
                     ...
                     ...
 <file>$path/mapper1#mapper1</file>
            <file>$path/reducer1#reducer1</file>
            <file>$path/tools.py#tools.py</file>
            <file>$path/streaming.py#streaming.py</file>
Note: $path and all other variables are defined in the config file.
         Files exist at these locations
         All action stages work executed through a script
On running the workflow(OOZIE URL set, correct config file,
updated(3.0.2) version of oozie, examples run fine), ALL map tasks
fail and the following is the error :
                                                          Error:
File "reducer1", line 10, in ?
    import streaming
ImportError: No module named streaming
java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess
failed with code 1
                                                        Questions:
1. Does the path of the file point to a HDFS location or home?
(currently points to the HDFS location, which is a mirror image of
home, for this purpose)
2. Is this a permissions problem?
3. Does forcing sim-links create this problem?(I have had earlier
problems because the mapper cannot follow sim-links, but either way
doesn't change the error here)
(All paths are absolute, and correct. All scripts work. All
archives(where used) are uncorrupted)
Advice is much appreciated. Please feel free to ask for additional
information.


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message