Mailing-List: contact dev-help@falcon.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@falcon.incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of shwetha.gs@inmobi.com
 designates 74.125.82.49 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAN0mEbx1Q2bNQ+vLa5jYj-Tg1xkxedoqaET8nmZaCFz_o=74-A@mail.gmail.com>
References: 
 <CAN0mEbxreBBB5qJQN2a9wML_u=QFoQAah6R3a9wXY+8sfKAh_A@mail.gmail.com>
	<CAGhRFTkPTwhSOtVC=3-HSb3r74dLOiTwTuHAHJhGrNp3mifoHQ@mail.gmail.com>
	<CAN0mEbwjRY_4WLR_HrPy3w4WLy7PZxzYZrWtbi1vM=uA5c21wg@mail.gmail.com>
	<BLU179-W11187B668125104A817F5BA4710@phx.gbl>
	<CAN0mEbyfohkHZjQhv4serdN6rPiL8xkJ4bVKLJKdyTbq8UOEoQ@mail.gmail.com>
	<CAN0mEbx1Q2bNQ+vLa5jYj-Tg1xkxedoqaET8nmZaCFz_o=74-A@mail.gmail.com>
Date: Thu, 27 Nov 2014 11:50:05 +0530
Message-ID: 
 <CAGhRFTkKVwhCBSpqN8TuXk9JyDaYxRnyhN6kM-jrxzhFJt9dEg@mail.gmail.com>
Subject: Re: Need help for the problem that i faced
From: Shwetha GS <shwetha.gs@inmobi.com>
To: "dev@falcon.incubator.apache.org" <dev@falcon.incubator.apache.org>
Content-Type: multipart/alternative; boundary=089e01419fc09f7c6d0508d12202

--089e01419fc09f7c6d0508d12202
Content-Type: text/plain; charset=UTF-8

2 ways of doing:
1. Falcon process exposes workflow lib(just like workflow path) which
should contain the path to the directory that contains the jar
example:

<workflow name="test" version="1.0.0" engine="oozie"
path="/user/guest/myapp" lib="/user/guest/mylib"/>

2. If the workflow is in <path>/workflow.xml, the jar should be in
<path>/lib/ directory


On Thu, Nov 27, 2014 at 11:42 AM, Gaurav Agarwal <gzagarwal@gmail.com>
wrote:

> Apart from this I have two more questions?
>
> 1. How to invoke java class from a workflow ,where to keep the jar in which
> the class present?I kept the jar at the same location where the workflow is
> present still got the same error. ClassNotfound exception
>
> Thanks
>
> On Thu, Nov 27, 2014 at 11:39 AM, Gaurav Agarwal <gzagarwal@gmail.com>
> wrote:
>
> > Srikanth
> >
> > What do we mean by that in oozie context?
> >
> > Thanks
> >
> > On Thu, Nov 27, 2014 at 11:36 AM, Srikanth Sundarrajan <
> > sriksun@hotmail.com> wrote:
> >
> >> Looks like the script ingest.sh is returning non-zero exit code.
> >>
> >> Regards
> >> Srikanth Sundarrajan
> >>
> >> > Date: Thu, 27 Nov 2014 11:31:30 +0530
> >> > Subject: Re: Need help for the problem that i faced
> >> > From: gzagarwal@gmail.com
> >> > To: dev@falcon.incubator.apache.org
> >> >
> >> > These are the log
> >> >
> >> > 2014-11-27 05:37:02,726 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@:start:] Start action
> >> > [0000129-141126162609456-oozie-oozi-W@:start:] with user-retry state
> :
> >> > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> >> > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
> >> > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action
> status=DONE
> >> > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
> >> > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action updated in
> >> DB!
> >> > 2014-11-27 05:37:03,151 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Start action
> >> > [0000129-141126162609456-oozie-oozi-W@shell-node] with user-retry
> >> state :
> >> > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> >> > 2014-11-27 05:37:05,377 INFO ShellActionExecutor:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] checking
> >> action,
> >> > hadoop job ID [job_1417019189193_0117] status [RUNNING]
> >> > 2014-11-27 05:37:05,378 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
> >> > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action
> >> > status=RUNNING
> >> > 2014-11-27 05:37:05,379 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
> >> > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action updated
> >> in
> >> > DB!
> >> > 2014-11-27 05:37:49,260 INFO CallbackServlet:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] callback for
> >> action
> >> > [0000129-141126162609456-oozie-oozi-W@shell-node]
> >> > 2014-11-27 05:37:50,753 INFO ShellActionExecutor:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] action
> >> completed,
> >> > external ID [job_1417019189193_0117]
> >> > 2014-11-27 05:37:50,761 WARN ShellActionExecutor:546 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Launcher
> ERROR,
> >> > reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit
> code
> >> > [1]
> >> > 2014-11-27 05:37:50,865 INFO ActionEndXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] ERROR is
> >> considered
> >> > as FAILED for SLA
> >> > 2014-11-27 05:37:50,941 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@fail] Start action
> >> > [0000129-141126162609456-oozie-oozi-W@fail] with user-retry state :
> >> > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> >> > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
> >> > [***0000129-141126162609456-oozie-oozi-W@fail***]Action status=DONE
> >> > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
> >> > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> APP[shell-wf]
> >> > JOB[0000129-141126162609456-oozie-oozi-W]
> >> > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
> >> > [***0000129-141126162609456-oozie-oozi-W@fail***]Action updated in
> DB!
> >> >
> >> > On Thu, Nov 27, 2014 at 10:25 AM, Shwetha GS <shwetha.gs@inmobi.com>
> >> wrote:
> >> >
> >> > >   <exec>ingest.sh</exec>
> >> > > Try using absolute path.
> >> > >
> >> > > None of the logs imply any error. Can you attach all the oozie logs
> >> for the
> >> > > workflow?
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > On Thu, Nov 27, 2014 at 7:50 AM, Gaurav Agarwal <
> gzagarwal@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > Hello I am running apache falcon and oozie with shell script to
> run
> >> the
> >> > > > workflow.
> >> > > >
> >> > > >
> >> > > >
> >> > > > getting this error
> >> > > >
> >> > > > *ell-node] Launcher ERROR, reason: Main class
> >> > > > [org.apache.oozie.action.hadoop.ShellMain], exit code [1]*
> >> > > > *2014-11-26 17:24:06,645  INFO ActionEndXCommand:543 -
> >> > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> >> > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> >> > > > JOB[0000020-141126162609456-oozie-oozi-W]
> >> > > > ACTION[0000020-141126162609456-oozie-oozi-W@shell-node] ERROR is
> >> > > > considered
> >> > > > as FAILED for SLA*
> >> > > > *2014-11-26 17:24:06,702  INFO ActionStartXCommand:543 -
> >> > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> >> > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> >> > > > JOB[0000020-141126162609456-oozie-oozi-W]
> >> > > > ACTION[0000020-141126162609456-oozie-oozi-W@fail] Start action
> >> > > > [0000020-141126162609456-oozie-*
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > *Workflow*
> >> > > >
> >> > > > <!--
> >> > > >   Licensed to the Apache Software Foundation (ASF) under one
> >> > > >   or more contributor license agreements.  See the NOTICE file
> >> > > >   distributed with this work for additional information
> >> > > >   regarding copyright ownership.  The ASF licenses this file
> >> > > >   to you under the Apache License, Version 2.0 (the
> >> > > >   "License"); you may not use this file except in compliance
> >> > > >   with the License.  You may obtain a copy of the License at
> >> > > >
> >> > > >        http://www.apache.org/licenses/LICENSE-2.0
> >> > > >
> >> > > >   Unless required by applicable law or agreed to in writing,
> >> software
> >> > > >   distributed under the License is distributed on an "AS IS"
> BASIS,
> >> > > >   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> >> > > implied.
> >> > > >   See the License for the specific language governing permissions
> >> and
> >> > > >   limitations under the License.
> >> > > > -->
> >> > > > <workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf">
> >> > > >     <start to="shell-node"/>
> >> > > >     <action name="shell-node">
> >> > > >         <shell xmlns="uri:oozie:shell-action:0.2">
> >> > > >             <job-tracker>${jobTracker}</job-tracker>
> >> > > >             <name-node>${nameNode}</name-node>
> >> > > >             <configuration>
> >> > > >                 <property>
> >> > > >                     <name>mapred.job.queue.name</name>
> >> > > >                     <value>${queueName}</value>
> >> > > >                 </property>
> >> > > >             </configuration>
> >> > > >             <exec>ingest.sh</exec>
> >> > > >             <argument>${feedInstancePaths}</argument>
> >> > > >             <file>${wf:appPath()}/ingest.sh#ingest.sh</file>
> >> > > >             <!-- <file>/tmp/ingest.sh#ingest.sh</file> -->
> >> > > >             <!-- <capture-output/> -->
> >> > > >         </shell>
> >> > > >         <ok to="end"/>
> >> > > >         <error to="fail"/>
> >> > > >     </action>
> >> > > >     <kill name="fail">
> >> > > >         <message>Shell action failed, error
> >> > > > message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> >> > > >     </kill>
> >> > > >     <end name="end"/>
> >> > > > </workflow-app>
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > This is the shell script
> >> > > >
> >> > > >
> >> > > >
> >> > > > #!/bin/bash
> >> > > >
> >> > > >
> >> > > > java -cp
> >> > > >
> >> > >
> >>
> "/media/sf_ToSandbox/BigData-0.0.1-SNAPSHOT.jar:/usr/hdp/2.2.0.0-913/sqoop/lib/*:/usr/hdp/2.2.0.0-913/hadoop/lib/*"
> >> > > > com.bigdata.falcon.StaticDataProvider
> >> > > >
> >> > > >
> >> > > >
> >> > > > These are the feed and process and cluster information
> >> > > >
> >> > > >
> >> > > >
> >> > > > <feed description="Raw customer email feed"
> name="FINALrawEmailFeed"
> >> > > >     xmlns="uri:falcon:feed:0.1">
> >> > > >
> >> > > >
> >>  <tags>externalSystem=USWestEmailServers,classification=secure</tags>
> >> > > >     <groups>churnAnalysisDataPipeline</groups>
> >> > > >     <frequency>minutes(3)</frequency>
> >> > > >     <late-arrival cut-off="hours(4)"/>
> >> > > >     <clusters>
> >> > > >         <cluster name="SprimaryCluster" type="source">
> >> > > >             <validity start="2014-11-21T00:00Z"
> >> end="2016-03-31T00:00Z"/>
> >> > > >             <retention limit="days(90)" action="delete"/>
> >> > > >         </cluster>
> >> > > >     </clusters>
> >> > > >
> >> > > >     <locations>
> >> > > >         <location type="data"
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> path="/user/gaurav123/falcon/demo/primary/input/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
> >> > > >         <location type="stats" path="/none"/>
> >> > > >         <location type="meta" path="/none"/>
> >> > > >     </locations>
> >> > > >
> >> > > >     <ACL owner="ambari-qa" group="users" permission="0777"/>
> >> > > >     <schema location="/none" provider="none"/>
> >> > > > </feed>
> >> > > >
> >> > > >
> >> > > >
> >> > > > *Process*
> >> > > >
> >> > > > *-----------*
> >> > > >
> >> > > >
> >> > > >
> >> > > > <?xml version="1.0" encoding="UTF-8"?>
> >> > > > <!--
> >> > > >  Hourly process, ingests raw emails from all US west email servers
> >> > > >  -->
> >> > > > <process name="IngestProcess" xmlns="uri:falcon:process:0.1">
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> <tags>pipeline=churnAnalysisDataPipeline,owner=ETLGroup,externalSystem=USWestEmailServers</tags>
> >> > > >
> >> > > >
> >> > > >     <clusters>
> >> > > >         <cluster name="SprimaryCluster">
> >> > > >             <validity start="2014-11-24T09:35Z"
> >> end="2014-11-25T10:00Z"/>
> >> > > >         </cluster>
> >> > > >     </clusters>
> >> > > >
> >> > > >     <parallel>1</parallel>
> >> > > >     <order>FIFO</order>
> >> > > >     <frequency>minutes(3)</frequency>
> >> > > >
> >> > > >     <outputs>
> >> > > >         <output name="output" feed="FINALrawEmailFeed"
> >> > > instance="now(0,0)"
> >> > > > />
> >> > > >     </outputs>
> >> > > >
> >> > > >     <workflow name="IngestWorkflow" version="2.0.0"
> >> > > >     engine="oozie"
> >> path="/user/gaurav123/falcon/demo/apps/ingest/fs" />
> >> > > >
> >> > > >     <retry policy="periodic" delay="minutes(15)" attempts="3" />
> >> > > >
> >> > > > </process>
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > Can you please let me know what is the issue , I have tried since
> >> long
> >> > > two
> >> > > > days? but unable to run the workflow with falcon process
> >> > > >
> >> > > >  Also got this exception
> >> > > >
> >> > > > 2014-11-26 17:47:42,548 ERROR CompletedActionXCommand:540 -
> >> > > > SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[] APP[-]
> >> > > > JOB[0000099-141126162609456-oozie-oozi-W]
> >> > > > ACTION[0000099-141126162609456-oozie-oozi-W@user-action]
> >> XException,
> >> > > > org.apache.oozie.command.CommandException: E0800: Action it is not
> >> > > > running its in [PREP] state, action
> >> > > > [0000099-141126162609456-oozie-oozi-W@user-action]
> >> > > >
> >> > >
> >> > > --
> >> > > _____________________________________________________________
> >> > > The information contained in this communication is intended solely
> >> for the
> >> > > use of the individual or entity to whom it is addressed and others
> >> > > authorized to receive it. It may contain confidential or legally
> >> privileged
> >> > > information. If you are not the intended recipient you are hereby
> >> notified
> >> > > that any disclosure, copying, distribution or taking any action in
> >> reliance
> >> > > on the contents of this information is strictly prohibited and may
> be
> >> > > unlawful. If you have received this communication in error, please
> >> notify
> >> > > us immediately by responding to this email and then delete it from
> >> your
> >> > > system. The firm is neither liable for the proper and complete
> >> transmission
> >> > > of the information contained in this communication nor for any delay
> >> in its
> >> > > receipt.
> >> > >
> >>
> >>
> >
> >
>

-- 
_____________________________________________________________
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

--089e01419fc09f7c6d0508d12202--