incubator-oozie-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Angelo K. Huang" <ange...@apache.org>
Subject Re: Oozie streaming script not found on nodes
Date Mon, 03 Oct 2011 00:45:25 GMT
Yes, it is relative to your application folder at hdfs.

On Sun, Oct 2, 2011 at 4:32 PM, Deepak Reddy <dreddy@couponsinc.com> wrote:

> I believe <file> refers to the hdfs location.
>
> This pattern works for me
>
> <mapper>filename</mapper>
> ...
> ...
> <file>filename</file>
> <file>depfile1</file>
> <file>depfile2>/file>
>
>
>
> -DR
>
> -----Original Message-----
> From: Sridhar Narayanan [mailto:srinar@yahoo-inc.com]
> Sent: Friday, September 30, 2011 4:39 AM
> To: oozie-users@incubator.apache.org
> Subject: Oozie streaming script not found on nodes
>
> I am running a workflow- currently only single actions at a time for
> diagnostics.
>                                    Workflow looks like(relevant section):
> <action name='preclean'>
>            <streaming>
>                <mapper>python mapper1 </mapper>
>                <reducer>python reducer1 </reducer>
>            </streaming>
>                     ...
>                     ...
>  <file>$path/mapper1#mapper1</file>
>            <file>$path/reducer1#reducer1</file>
>            <file>$path/tools.py#tools.py</file>
>            <file>$path/streaming.py#streaming.py</file>
> Note: $path and all other variables are defined in the config file.
>         Files exist at these locations
>         All action stages work executed through a script On running the
> workflow(OOZIE URL set, correct config file,
> updated(3.0.2) version of oozie, examples run fine), ALL map tasks fail and
> the following is the error :
>                                                          Error:
> File "reducer1", line 10, in ?
>    import streaming
> ImportError: No module named streaming
> java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess
> failed with code 1
>                                                        Questions:
> 1. Does the path of the file point to a HDFS location or home?
> (currently points to the HDFS location, which is a mirror image of home,
> for this purpose) 2. Is this a permissions problem?
> 3. Does forcing sim-links create this problem?(I have had earlier problems
> because the mapper cannot follow sim-links, but either way doesn't change
> the error here) (All paths are absolute, and correct. All scripts work. All
> archives(where used) are uncorrupted) Advice is much appreciated. Please
> feel free to ask for additional information.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message