hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rares Vernica <rvern...@gmail.com>
Subject Re: permission denied on additional binaries
Date Wed, 08 Jul 2009 15:09:18 GMT
On Tue, Jul 7, 2009 at 10:26 PM, jason hadoop <jason.hadoop@gmail.com> wrote:
>
> The mapper has no control at the point where your mymapper.sh script is
> running.
>
> You may wish to have the following commands in mymapper.sh before you
> attempt your pipeline
> (pwd
> ls -l ./binary1 ./binary2
> echo $PATH) 1>&2

I tried this, and because I specify "-file mymapper.sh -file binary1
-file binary2" as arguments to the streaming command, the two binaries
and the script end up in the same directory.

> The 1>&2 redirects stdout to stderr.
>
> The commands output should end up in your job's userlog.
> You could redirect it to a temporary file to make it available in /tmp.
>
> Chapter 8 of Pro Hadoop covers some fine details of streaming jobs.
>
> It may be that there is something going on in the environment that is
> resulting in your permission denied error.
>
>
> On Tue, Jul 7, 2009 at 9:04 PM, rvernica <rvernica@gmail.com> wrote:
>
> >
> >
> > Ashish Venugopal wrote:
> > >
> > > I have a question regarding a mapper task that needs to call 2 binaries.
> > > Ideally I would be able to do the following:
> > > mymapper.sh
> > >
> > > ./binary1 | ./binary2
> > >
> > > stream -mapper mymapper.sh ... -file binary1 -file binary2 -file
> > > mymapper.sh
> > >
> > > But when this runs, I get "permission denied" executing binary1. I read
> > on
> > > the Wiki that Unix pipers are not
> > > allowed, but it seemed to indicated that simply putting them on the
> > > command
> > > line was not allowed, but in a
> > > script it was acceptable. Is this the case?
> > >
> > > Also, I have also tried to make binary1 call binary2 via forking, but I
> > > get
> > > the same permission denied
> > > error. Is this a fundamental design decision (to allow only 1 binary in
> > > the
> > > mapper)? Or do I just need
> > > to explicitly change the permissions somehow?
> > >
> > > Ashish
> > >
> > >
> >
> > Does anybody has any solution to this?
> >
> > I tried it in 17.2.1 and I still get Permission Denied.
> >
> > On the other hand, if I say:
> >
> > stream ... -reducer "binary1 | binary2"
> >
> > "|" and "binary2" are treated as arguments to "binary1"
> >
> > Thanks!
> > --
> > View this message in context:
> > http://www.nabble.com/permission-denied-on-additional-binaries-tp16551104p24384957.html
> > Sent from the Hadoop core-user mailing list archive at Nabble.com.
> >
> >
>
>
> --
> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> http://www.amazon.com/dp/1430219424?tag=jewlerymall
> www.prohadoopbook.com a community for Hadoop Professionals

Mime
View raw message