Mailing-List: contact oozie-users-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: oozie-users@incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <4E24448B-9764-49A6-AF65-D2645DF84AAE@gmail.com>
References: <C20A058B-4805-4156-A354-75308BF3AEE1@gmail.com>
	<C57369F6-9B5A-4007-88D8-183C26524B4A@gmail.com>
	<CAMkn0zYXDVo7_aGLTm78qjsU+dqgf1LMjE2hMJ5HxQtuk__o1g@mail.gmail.com>
	<4E24448B-9764-49A6-AF65-D2645DF84AAE@gmail.com>
Date: Sat, 22 Oct 2011 16:38:04 -0700
Message-ID: 
 <CAMkn0zb34QyHixzBu6=z8nNimBUhCtN6ftXenoaxpdJjD+yDzQ@mail.gmail.com>
Subject: Re: using oozie with datastax brisk
From: Mayank Bansal <mayank@apache.org>
To: oozie-users@incubator.apache.org
Content-Type: multipart/alternative; boundary=00151750e42a604dc604afebae9b

--00151750e42a604dc604afebae9b
Content-Type: text/plain; charset=ISO-8859-1

Yeah this is the issue. It is not fixed in 3.0.2.

I am sending you the commit as of now and compile the code yourself.

https://github.com/bansalmayank/oozie/commit/6a8aa2d799b34b2aa2f10e4133eaee2898f66053

just take the changes from the following files.
 [image: Txt]core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java
 [image: Txt] core/src/main/resources/oozie-default.xml

just add

oozie.filesystems.supported
hdfs,cfs


Thanks,
Mayank

On Sat, Oct 22, 2011 at 7:26 AM, Jeremy Hanna <jeremy.hanna1234@gmail.com>wrote:

> Thanks for the response Mayank.  Is there a particular fix on trunk or
> something you're talking about?
>
> The full exception in the logs from the error I mentioned is:
> 2011-10-21 21:03:55,627  INFO BaseJobServlet:536 - USER[?] GROUP[users]
> TOKEN[-] APP[-] JOB[-] ACTION[-] AuthorizationException
> org.apache.oozie.service.AuthorizationException: E0902: Exception occured:
> [java.io.IOException: No FileSystem for scheme: cfs]
>        at
> org.apache.oozie.service.AuthorizationService.authorizeForApp(AuthorizationService.java:320)
>        at
> org.apache.oozie.servlet.BaseJobServlet.checkAuthorizationForApp(BaseJobServlet.java:185)
>        at
> org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:89)
>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
>        at
> org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:281)
>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>        at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
>        at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>        at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>        at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
>        at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>        at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>        at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>        at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
>        at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
>        at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
>        at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
>        at java.lang.Thread.run(Thread.java:662)
> Caused by: org.apache.oozie.service.HadoopAccessorException: E0902:
> Exception occured: [java.io.IOException: No FileSystem for scheme: cfs]
>        at
> org.apache.oozie.service.KerberosHadoopAccessorService.createFileSystem(KerberosHadoopAccessorService.java:208)
>        at
> org.apache.oozie.service.AuthorizationService.authorizeForApp(AuthorizationService.java:285)
>        ... 17 more
> Caused by: java.io.IOException: No FileSystem for scheme: cfs
>        at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1307)
>        at org.apache.hadoop.fs.FileSystem.access$100(FileSystem.java:65)
>        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1328)
>        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:226)
>        at
> org.apache.oozie.service.KerberosHadoopAccessorService$3.run(KerberosHadoopAccessorService.java:200)
>        at
> org.apache.oozie.service.KerberosHadoopAccessorService$3.run(KerberosHadoopAccessorService.java:192)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at javax.security.auth.Subject.doAs(Subject.java:396)
>        at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>        at
> org.apache.oozie.service.KerberosHadoopAccessorService.createFileSystem(KerberosHadoopAccessorService.java:192)
>        ... 18 more
>
> I seem to have gotten past that part, though I've needed to get some of the
> Cassandra libs in there that cfs is dependent on.  But I may not be seeing
> all the errors yet.  Is there a release planned with the fix that you're
> talking about or a revision or ticket or patch I can apply?
>
> Thanks,
>
> Jeremy
>
> On Oct 22, 2011, at 12:32 AM, Mayank Bansal wrote:
>
> > I am not sure whether you will be able to run any other file system
> except
> > hdfs. I do not think we have put the configurable file systems fix in
> 3.0.2.
> >
> > Send the Oozie log for more details.
> >
> > Thanks,
> > Mayank
> >
> > On Fri, Oct 21, 2011 at 7:37 PM, Jeremy Hanna <
> jeremy.hanna1234@gmail.com>wrote:
> >
> >> Looks like I need to put that core-site.xml and the jar that contains
> >> CassandraFileSystem into the oozie/oozie-server/lib directory.  Still
> trying
> >> to track down a 500 error now, but at least I've gotten past that
> initial
> >> issue.
> >>
> >> On Oct 21, 2011, at 4:20 PM, Jeremy Hanna wrote:
> >>
> >>> I've been trying to setup oozie with datastax's brisk product.  I am
> >> using oozie 3.0.2 and set it up by using:
> >>> bin/oozie-setup.sh -hadoop 0.20.200 /opt/brisk/resources/hadoop -extjs
> >> ./ext-2.2.zip
> >>>
> >>> Technically brisk uses hadoop 0.20.203 but oozie didn't like that.
> >>>
> >>> So in the brisk core-site.xml it defines:
> >>> <property>
> >>>   <name>fs.cfs.impl</name>
> >>>   <value>com.datastax.bdp.hadoop.cfs.CassandraFileSystem</value>
> >>> </property>
> >>>
> >>> and then I use that in my fs.default.name
> >>> <property>
> >>>   <name>fs.default.name</name>
> >>>   <value>cfs://myserver.com</value>
> >>> </property>
> >>>
> >>> however when I try to run one of the examples such as this (with
> nameNode
> >> set to cfs://myserver.com):
> >>> oozie job -oozie http://localhost:11000/oozie -config
> >> examples/apps/map-reduce/job.properties -run
> >>>
> >>> it gives the error:
> >>> Error: E0902 : E0902: Exception occured: [java.io.IOException: No
> >> FileSystem for scheme: cfs]
> >>>
> >>> Shouldn't oozie have picked up the core-site.xml from the hadoop that
> it
> >> picked up when running oozie-setup.sh and therefore been able to pick up
> the
> >> fs.cfs.impl property?
> >>>
> >>> Thanks!
> >>>
> >>> Jeremy
> >>
> >>
> >
> >
> > --
> > Thanks and Regards,
> > Mayank
> > Cell: 408-718-9370
>
>

--00151750e42a604dc604afebae9b--