Mailing-List: contact dev-help@falcon.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@falcon.incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of gzagarwal@gmail.com designates
 209.85.214.180 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAGhRFTnKNqURkQoMuG0N+_5C+LVXXSzfs=BHz=P9sAydBr3BSA@mail.gmail.com>
References: 
 <CAN0mEbxreBBB5qJQN2a9wML_u=QFoQAah6R3a9wXY+8sfKAh_A@mail.gmail.com>
	<CAGhRFTkPTwhSOtVC=3-HSb3r74dLOiTwTuHAHJhGrNp3mifoHQ@mail.gmail.com>
	<CAN0mEbwjRY_4WLR_HrPy3w4WLy7PZxzYZrWtbi1vM=uA5c21wg@mail.gmail.com>
	<BLU179-W11187B668125104A817F5BA4710@phx.gbl>
	<CAN0mEbyfohkHZjQhv4serdN6rPiL8xkJ4bVKLJKdyTbq8UOEoQ@mail.gmail.com>
	<CAGhRFTnKNqURkQoMuG0N+_5C+LVXXSzfs=BHz=P9sAydBr3BSA@mail.gmail.com>
Date: Thu, 27 Nov 2014 11:49:12 +0530
Message-ID: 
 <CAN0mEby5S9wO6kaUZfwwfjy99wTCg=mn3Ao9tyV=Emt+zrazXg@mail.gmail.com>
Subject: Re: Need help for the problem that i faced
From: Gaurav Agarwal <gzagarwal@gmail.com>
To: dev@falcon.incubator.apache.org
Content-Type: multipart/alternative; boundary=001a113cc7f27398620508d11f06

--001a113cc7f27398620508d11f06
Content-Type: text/plain; charset=UTF-8

Shwetha

Apologise to trouble everyone but i am new to this falcon. Do i need to
check the logs in JobBrowser?

there is stderror

Log Length: 188
Error: Could not find or load main class com.jpmc.cib.is.bigdata.
falcon.SemsStaticDataProvider
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.ShellMain],
exit code [1]


On Thu, Nov 27, 2014 at 11:43 AM, Shwetha GS <shwetha.gs@inmobi.com> wrote:

> oozie launches a hadoop job which executes the shell script. Check the
> launcher job logs
>
> On Thu, Nov 27, 2014 at 11:39 AM, Gaurav Agarwal <gzagarwal@gmail.com>
> wrote:
>
> > Srikanth
> >
> > What do we mean by that in oozie context?
> >
> > Thanks
> >
> > On Thu, Nov 27, 2014 at 11:36 AM, Srikanth Sundarrajan <
> > sriksun@hotmail.com>
> > wrote:
> >
> > > Looks like the script ingest.sh is returning non-zero exit code.
> > >
> > > Regards
> > > Srikanth Sundarrajan
> > >
> > > > Date: Thu, 27 Nov 2014 11:31:30 +0530
> > > > Subject: Re: Need help for the problem that i faced
> > > > From: gzagarwal@gmail.com
> > > > To: dev@falcon.incubator.apache.org
> > > >
> > > > These are the log
> > > >
> > > > 2014-11-27 05:37:02,726 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:] Start action
> > > > [0000129-141126162609456-oozie-oozi-W@:start:] with user-retry
> state :
> > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > > > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
> > > > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action
> status=DONE
> > > > 2014-11-27 05:37:02,727 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@:start:]
> > > > [***0000129-141126162609456-oozie-oozi-W@:start:***]Action updated
> in
> > > DB!
> > > > 2014-11-27 05:37:03,151 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Start action
> > > > [0000129-141126162609456-oozie-oozi-W@shell-node] with user-retry
> > state
> > > :
> > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > > > 2014-11-27 05:37:05,377 INFO ShellActionExecutor:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] checking
> > action,
> > > > hadoop job ID [job_1417019189193_0117] status [RUNNING]
> > > > 2014-11-27 05:37:05,378 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
> > > > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action
> > > > status=RUNNING
> > > > 2014-11-27 05:37:05,379 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node]
> > > > [***0000129-141126162609456-oozie-oozi-W@shell-node***]Action
> updated
> > in
> > > > DB!
> > > > 2014-11-27 05:37:49,260 INFO CallbackServlet:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] callback for
> > > action
> > > > [0000129-141126162609456-oozie-oozi-W@shell-node]
> > > > 2014-11-27 05:37:50,753 INFO ShellActionExecutor:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] action
> > > completed,
> > > > external ID [job_1417019189193_0117]
> > > > 2014-11-27 05:37:50,761 WARN ShellActionExecutor:546 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] Launcher
> > ERROR,
> > > > reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit
> > code
> > > > [1]
> > > > 2014-11-27 05:37:50,865 INFO ActionEndXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@shell-node] ERROR is
> > > considered
> > > > as FAILED for SLA
> > > > 2014-11-27 05:37:50,941 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail] Start action
> > > > [0000129-141126162609456-oozie-oozi-W@fail] with user-retry state :
> > > > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > > > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
> > > > [***0000129-141126162609456-oozie-oozi-W@fail***]Action status=DONE
> > > > 2014-11-27 05:37:50,942 INFO ActionStartXCommand:543 - SERVER[
> > > > sandbox.hortonworks.com] USER[ambari-qa] GROUP[-] TOKEN[]
> > APP[shell-wf]
> > > > JOB[0000129-141126162609456-oozie-oozi-W]
> > > > ACTION[0000129-141126162609456-oozie-oozi-W@fail]
> > > > [***0000129-141126162609456-oozie-oozi-W@fail***]Action updated in
> DB!
> > > >
> > > > On Thu, Nov 27, 2014 at 10:25 AM, Shwetha GS <shwetha.gs@inmobi.com>
> > > wrote:
> > > >
> > > > >   <exec>ingest.sh</exec>
> > > > > Try using absolute path.
> > > > >
> > > > > None of the logs imply any error. Can you attach all the oozie logs
> > > for the
> > > > > workflow?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Nov 27, 2014 at 7:50 AM, Gaurav Agarwal <
> gzagarwal@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Hello I am running apache falcon and oozie with shell script to
> run
> > > the
> > > > > > workflow.
> > > > > >
> > > > > >
> > > > > >
> > > > > > getting this error
> > > > > >
> > > > > > *ell-node] Launcher ERROR, reason: Main class
> > > > > > [org.apache.oozie.action.hadoop.ShellMain], exit code [1]*
> > > > > > *2014-11-26 17:24:06,645  INFO ActionEndXCommand:543 -
> > > > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> > > > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > > > > > JOB[0000020-141126162609456-oozie-oozi-W]
> > > > > > ACTION[0000020-141126162609456-oozie-oozi-W@shell-node] ERROR is
> > > > > > considered
> > > > > > as FAILED for SLA*
> > > > > > *2014-11-26 17:24:06,702  INFO ActionStartXCommand:543 -
> > > > > > SERVER[sandbox.hortonworks.com <http://sandbox.hortonworks.com>]
> > > > > > USER[ambari-qa] GROUP[-] TOKEN[] APP[shell-wf]
> > > > > > JOB[0000020-141126162609456-oozie-oozi-W]
> > > > > > ACTION[0000020-141126162609456-oozie-oozi-W@fail] Start action
> > > > > > [0000020-141126162609456-oozie-*
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > *Workflow*
> > > > > >
> > > > > > <!--
> > > > > >   Licensed to the Apache Software Foundation (ASF) under one
> > > > > >   or more contributor license agreements.  See the NOTICE file
> > > > > >   distributed with this work for additional information
> > > > > >   regarding copyright ownership.  The ASF licenses this file
> > > > > >   to you under the Apache License, Version 2.0 (the
> > > > > >   "License"); you may not use this file except in compliance
> > > > > >   with the License.  You may obtain a copy of the License at
> > > > > >
> > > > > >        http://www.apache.org/licenses/LICENSE-2.0
> > > > > >
> > > > > >   Unless required by applicable law or agreed to in writing,
> > software
> > > > > >   distributed under the License is distributed on an "AS IS"
> BASIS,
> > > > > >   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> > > > > implied.
> > > > > >   See the License for the specific language governing permissions
> > and
> > > > > >   limitations under the License.
> > > > > > -->
> > > > > > <workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf">
> > > > > >     <start to="shell-node"/>
> > > > > >     <action name="shell-node">
> > > > > >         <shell xmlns="uri:oozie:shell-action:0.2">
> > > > > >             <job-tracker>${jobTracker}</job-tracker>
> > > > > >             <name-node>${nameNode}</name-node>
> > > > > >             <configuration>
> > > > > >                 <property>
> > > > > >                     <name>mapred.job.queue.name</name>
> > > > > >                     <value>${queueName}</value>
> > > > > >                 </property>
> > > > > >             </configuration>
> > > > > >             <exec>ingest.sh</exec>
> > > > > >             <argument>${feedInstancePaths}</argument>
> > > > > >             <file>${wf:appPath()}/ingest.sh#ingest.sh</file>
> > > > > >             <!-- <file>/tmp/ingest.sh#ingest.sh</file> -->
> > > > > >             <!-- <capture-output/> -->
> > > > > >         </shell>
> > > > > >         <ok to="end"/>
> > > > > >         <error to="fail"/>
> > > > > >     </action>
> > > > > >     <kill name="fail">
> > > > > >         <message>Shell action failed, error
> > > > > > message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> > > > > >     </kill>
> > > > > >     <end name="end"/>
> > > > > > </workflow-app>
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > This is the shell script
> > > > > >
> > > > > >
> > > > > >
> > > > > > #!/bin/bash
> > > > > >
> > > > > >
> > > > > > java -cp
> > > > > >
> > > > >
> > >
> >
> "/media/sf_ToSandbox/BigData-0.0.1-SNAPSHOT.jar:/usr/hdp/2.2.0.0-913/sqoop/lib/*:/usr/hdp/2.2.0.0-913/hadoop/lib/*"
> > > > > > com.bigdata.falcon.StaticDataProvider
> > > > > >
> > > > > >
> > > > > >
> > > > > > These are the feed and process and cluster information
> > > > > >
> > > > > >
> > > > > >
> > > > > > <feed description="Raw customer email feed"
> > name="FINALrawEmailFeed"
> > > > > >     xmlns="uri:falcon:feed:0.1">
> > > > > >
> > > > > >
> > >  <tags>externalSystem=USWestEmailServers,classification=secure</tags>
> > > > > >     <groups>churnAnalysisDataPipeline</groups>
> > > > > >     <frequency>minutes(3)</frequency>
> > > > > >     <late-arrival cut-off="hours(4)"/>
> > > > > >     <clusters>
> > > > > >         <cluster name="SprimaryCluster" type="source">
> > > > > >             <validity start="2014-11-21T00:00Z"
> > > end="2016-03-31T00:00Z"/>
> > > > > >             <retention limit="days(90)" action="delete"/>
> > > > > >         </cluster>
> > > > > >     </clusters>
> > > > > >
> > > > > >     <locations>
> > > > > >         <location type="data"
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > >
> >
> path="/user/gaurav123/falcon/demo/primary/input/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
> > > > > >         <location type="stats" path="/none"/>
> > > > > >         <location type="meta" path="/none"/>
> > > > > >     </locations>
> > > > > >
> > > > > >     <ACL owner="ambari-qa" group="users" permission="0777"/>
> > > > > >     <schema location="/none" provider="none"/>
> > > > > > </feed>
> > > > > >
> > > > > >
> > > > > >
> > > > > > *Process*
> > > > > >
> > > > > > *-----------*
> > > > > >
> > > > > >
> > > > > >
> > > > > > <?xml version="1.0" encoding="UTF-8"?>
> > > > > > <!--
> > > > > >  Hourly process, ingests raw emails from all US west email
> servers
> > > > > >  -->
> > > > > > <process name="IngestProcess" xmlns="uri:falcon:process:0.1">
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > >
> >
> <tags>pipeline=churnAnalysisDataPipeline,owner=ETLGroup,externalSystem=USWestEmailServers</tags>
> > > > > >
> > > > > >
> > > > > >     <clusters>
> > > > > >         <cluster name="SprimaryCluster">
> > > > > >             <validity start="2014-11-24T09:35Z"
> > > end="2014-11-25T10:00Z"/>
> > > > > >         </cluster>
> > > > > >     </clusters>
> > > > > >
> > > > > >     <parallel>1</parallel>
> > > > > >     <order>FIFO</order>
> > > > > >     <frequency>minutes(3)</frequency>
> > > > > >
> > > > > >     <outputs>
> > > > > >         <output name="output" feed="FINALrawEmailFeed"
> > > > > instance="now(0,0)"
> > > > > > />
> > > > > >     </outputs>
> > > > > >
> > > > > >     <workflow name="IngestWorkflow" version="2.0.0"
> > > > > >     engine="oozie"
> > path="/user/gaurav123/falcon/demo/apps/ingest/fs"
> > > />
> > > > > >
> > > > > >     <retry policy="periodic" delay="minutes(15)" attempts="3" />
> > > > > >
> > > > > > </process>
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Can you please let me know what is the issue , I have tried since
> > > long
> > > > > two
> > > > > > days? but unable to run the workflow with falcon process
> > > > > >
> > > > > >  Also got this exception
> > > > > >
> > > > > > 2014-11-26 17:47:42,548 ERROR CompletedActionXCommand:540 -
> > > > > > SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[] APP[-]
> > > > > > JOB[0000099-141126162609456-oozie-oozi-W]
> > > > > > ACTION[0000099-141126162609456-oozie-oozi-W@user-action]
> > XException,
> > > > > > org.apache.oozie.command.CommandException: E0800: Action it is
> not
> > > > > > running its in [PREP] state, action
> > > > > > [0000099-141126162609456-oozie-oozi-W@user-action]
> > > > > >
> > > > >
> > > > > --
> > > > > _____________________________________________________________
> > > > > The information contained in this communication is intended solely
> > for
> > > the
> > > > > use of the individual or entity to whom it is addressed and others
> > > > > authorized to receive it. It may contain confidential or legally
> > > privileged
> > > > > information. If you are not the intended recipient you are hereby
> > > notified
> > > > > that any disclosure, copying, distribution or taking any action in
> > > reliance
> > > > > on the contents of this information is strictly prohibited and may
> be
> > > > > unlawful. If you have received this communication in error, please
> > > notify
> > > > > us immediately by responding to this email and then delete it from
> > your
> > > > > system. The firm is neither liable for the proper and complete
> > > transmission
> > > > > of the information contained in this communication nor for any
> delay
> > > in its
> > > > > receipt.
> > > > >
> > >
> > >
> >
>
> --
> _____________________________________________________________
> The information contained in this communication is intended solely for the
> use of the individual or entity to whom it is addressed and others
> authorized to receive it. It may contain confidential or legally privileged
> information. If you are not the intended recipient you are hereby notified
> that any disclosure, copying, distribution or taking any action in reliance
> on the contents of this information is strictly prohibited and may be
> unlawful. If you have received this communication in error, please notify
> us immediately by responding to this email and then delete it from your
> system. The firm is neither liable for the proper and complete transmission
> of the information contained in this communication nor for any delay in its
> receipt.
>

--001a113cc7f27398620508d11f06--