hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jason hadoop <jason.had...@gmail.com>
Subject Re: permission denied on additional binaries
Date Thu, 09 Jul 2009 05:52:46 GMT
Just out of curiosity, what happens when you run your script by hand?


On Wed, Jul 8, 2009 at 8:09 AM, Rares Vernica <rvernica@gmail.com> wrote:

> On Tue, Jul 7, 2009 at 10:26 PM, jason hadoop <jason.hadoop@gmail.com>
> wrote:
> >
> > The mapper has no control at the point where your mymapper.sh script is
> > running.
> >
> > You may wish to have the following commands in mymapper.sh before you
> > attempt your pipeline
> > (pwd
> > ls -l ./binary1 ./binary2
> > echo $PATH) 1>&2
>
> I tried this, and because I specify "-file mymapper.sh -file binary1
> -file binary2" as arguments to the streaming command, the two binaries
> and the script end up in the same directory.
>
> > The 1>&2 redirects stdout to stderr.
> >
> > The commands output should end up in your job's userlog.
> > You could redirect it to a temporary file to make it available in /tmp.
> >
> > Chapter 8 of Pro Hadoop covers some fine details of streaming jobs.
> >
> > It may be that there is something going on in the environment that is
> > resulting in your permission denied error.
> >
> >
> > On Tue, Jul 7, 2009 at 9:04 PM, rvernica <rvernica@gmail.com> wrote:
> >
> > >
> > >
> > > Ashish Venugopal wrote:
> > > >
> > > > I have a question regarding a mapper task that needs to call 2
> binaries.
> > > > Ideally I would be able to do the following:
> > > > mymapper.sh
> > > >
> > > > ./binary1 | ./binary2
> > > >
> > > > stream -mapper mymapper.sh ... -file binary1 -file binary2 -file
> > > > mymapper.sh
> > > >
> > > > But when this runs, I get "permission denied" executing binary1. I
> read
> > > on
> > > > the Wiki that Unix pipers are not
> > > > allowed, but it seemed to indicated that simply putting them on the
> > > > command
> > > > line was not allowed, but in a
> > > > script it was acceptable. Is this the case?
> > > >
> > > > Also, I have also tried to make binary1 call binary2 via forking, but
> I
> > > > get
> > > > the same permission denied
> > > > error. Is this a fundamental design decision (to allow only 1 binary
> in
> > > > the
> > > > mapper)? Or do I just need
> > > > to explicitly change the permissions somehow?
> > > >
> > > > Ashish
> > > >
> > > >
> > >
> > > Does anybody has any solution to this?
> > >
> > > I tried it in 17.2.1 and I still get Permission Denied.
> > >
> > > On the other hand, if I say:
> > >
> > > stream ... -reducer "binary1 | binary2"
> > >
> > > "|" and "binary2" are treated as arguments to "binary1"
> > >
> > > Thanks!
> > > --
> > > View this message in context:
> > >
> http://www.nabble.com/permission-denied-on-additional-binaries-tp16551104p24384957.html
> > > Sent from the Hadoop core-user mailing list archive at Nabble.com.
> > >
> > >
> >
> >
> > --
> > Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > www.prohadoopbook.com a community for Hadoop Professionals
>



-- 
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message