falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srikanth Sundarrajan <srik...@hotmail.com>
Subject RE: Hive Tables as Falcon Feed Input Error
Date Tue, 15 Apr 2014 12:34:43 GMT
I would suspect that Oozie has issues connecting to the hive metastore while determining availability
of instance. Some logs might help nail down the issue.
RegardsSrikanth Sundarrajan

> From: suniln.kumar@impetus.co.in
> To: dev@falcon.incubator.apache.org
> Subject: RE: Hive Tables as Falcon Feed Input Error
> Date: Tue, 15 Apr 2014 12:29:29 +0000
> 
> Hi Srikanth,
> It is still in waiting state at oozie.
> 
> 
> Thanks and Regard.
> Sunil Kumar
> 
> 
> -----Original Message-----
> From: Srikanth Sundarrajan [mailto:sriksun@hotmail.com]
> Sent: Tuesday, April 15, 2014 5:53 PM
> To: dev@falcon.incubator.apache.org
> Subject: RE: Hive Tables as Falcon Feed Input Error
> 
> Hi Sunil,    Can you confirm if the process has kicked off in hadoop or is it still in
PREP / WAITING state in oozie ?
> RegardsSrikanth Sundarrajan
> 
> > From: suniln.kumar@impetus.co.in
> > To: dev@falcon.incubator.apache.org
> > Subject: RE: Hive Tables as Falcon Feed Input Error
> > Date: Tue, 15 Apr 2014 06:37:01 +0000
> >
> > Hi,
> > I am able to resolve Feed definition issue for the Hive tables. It is mandatory
to have Hive partition for you hive tables and it also needs to build Oozie with the Hive
and Hcatalog jars. Please refer to below blog for building oozie with Hive Dependencies:
> > http://blog.nanthrax.net/2014/03/hadoop-cdc-and-processes-notification
> > -with-apache-falcon-apache-activemq-and-apache-camel/
> >
> >
> > But now I am facing issue with the Process execution of falcon for the Hive tables.
It remain running state from last 18 hours . Not sure what is the issue . can somebody help
me to resolve this.
> >
> >  Following is the process definition of the Hive process I have created :
> > <?xml version="1.0" encoding="UTF-8"?> <process name="hiveprocess2"
> > xmlns="uri:falcon:process:0.1">
> >     <clusters>
> >         <cluster name="demoClsHive1">
> >             <validity end="2014-04-22T00:00Z" start="2014-04-01T00:00Z"/>
> >         </cluster>
> >     </clusters>
> >
> >     <parallel>1</parallel>
> >     <order>FIFO</order>
> >     <frequency>days(1)</frequency>
> >     <timezone>UTC</timezone>
> >
> >     <inputs>
> >         <input end="today(0,0)" start="today(0,0)" feed="reservation2" name="input"/>
> >     </inputs>
> >
> >     <outputs>
> >         <output instance="now(0,0)" feed="reservationmale2" name="output"/>
> >     </outputs>
> >
> >     <properties>
> >         <property name="queueName" value="default"/>
> >     </properties>
> >
> >     <workflow engine="hive" path="/falcon/hive_run/script.hql"/>
> >
> >     <retry policy="periodic" delay="minutes(10)" attempts="3"/>
> >
> >     <late-process policy="exp-backoff" delay="hours(2)">
> >         <late-input input="input" workflow-path="/falcon/hive_run/"/>
> >     </late-process>
> > </process>
> >
> >
> >
> >
> > -----Original Message-----
> > From: Sunil N Kumar
> > Sent: Monday, April 14, 2014 12:58 PM
> > To: dev@falcon.incubator.apache.org
> > Subject: Hive Tables as Falcon Feed Input Error
> >
> > Hi,
> >
> > I am trying to build sample around executing Hive Script as part of Falcon Oozie
Process. I have created two tables in the Hive and want to join data from these tables and
put data into third Hive table using falcon Process. But I am not able to submit Input feed
into Falcon Server. I am using Falcon 0.4 release.
> >
> >
> >
> > Input Feed XML:
> >
> > <?xml version="1.0" encoding="UTF-8"?>
> >
> > <feed description="clicks log table " name="input-table"
> > xmlns="uri:falcon:feed:0.1">
> >
> >     <frequency>hours(1)</frequency>
> >
> >     <timezone>UTC</timezone>
> >
> >     <late-arrival cut-off="hours(3)"/>
> >
> >
> >
> >     <clusters>
> >
> >         <cluster name="demoClsHive">
> >
> >             <validity start="2010-01-01T00:00Z"
> > end="2012-04-21T00:00Z"/>
> >
> >             <retention limit="hours(24)" action="delete"/>
> >
> >         </cluster>
> >
> >     </clusters>
> >
> >
> >
> >     <table uri="catalog:default:reservationnew:gender='Male'" />
> >
> >
> >
> >     <ACL owner="cloud" group="cloud" permission="0x755"/>
> >
> >     <schema location="hcat" provider="hcat"/>
> >
> > </feed>
> >
> >
> >
> > Error:
> >
> > [cloud@Impetus-942 falcon-distributed-0.4-incubating]$ bin/falcon
> > entity -submit -type feed -file examples/entity/in-feed.xml
> >
> > Error: java.net.URISyntaxException: URI path is not in expected format: database:table:
catalog:default:reservationnew:gender='Male'
> >
> > ________________________________
> >
> >
> >
> >
> >
> >
> > NOTE: This message may contain information that is confidential, proprietary, privileged
or otherwise protected by law. The message is intended solely for the named addressee. If
received in error, please destroy and notify the sender. Any use of this email is prohibited
when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity
of this communication has been maintained nor that the communication is free of errors, virus,
interception or interference.
> >
> > ________________________________
> >
> >
> >
> >
> >
> >
> > NOTE: This message may contain information that is confidential, proprietary, privileged
or otherwise protected by law. The message is intended solely for the named addressee. If
received in error, please destroy and notify the sender. Any use of this email is prohibited
when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity
of this communication has been maintained nor that the communication is free of errors, virus,
interception or interference.
> 
> 
> ________________________________
> 
> 
> 
> 
> 
> 
> NOTE: This message may contain information that is confidential, proprietary, privileged
or otherwise protected by law. The message is intended solely for the named addressee. If
received in error, please destroy and notify the sender. Any use of this email is prohibited
when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity
of this communication has been maintained nor that the communication is free of errors, virus,
interception or interference.
 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message