falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peng Zhang <pengzh...@yahoo-inc.com>
Subject Re: Retention does not work?
Date Tue, 07 Oct 2014 15:35:52 GMT
Hi Venkatesh,
I see the staging path and there only exists a workflow dir which contains the process info
and logs. I didn't see the Is there anything wrong with my conf?
One of the log file is like below. It's in the json format. What the problem would be then?Looking
forward for your reply.
Thanks,Peng


{"timeStamp":"2014-10-03-15-57","status":"SUCCEEDED","feedNames":"out","runId":"0","entityType":"PROCESS","nominalTime":"2013-11-15-00-55","userWorkflowName":"oozie-co-workflow","workflowUser":"pengzhang","entityName":"oozie-co","feedInstancePaths":"hdfs:\/\/nn\/data\/out\/2013\/11\/15\/00","operation":"GENERATE","logDir":"hdfs:\/\/nn:8020\/projects\/falcon\/staging\/falcon\/workflows\/process\/oozie-co\/logs\/job-2013-11-15-00-55\/","falconInputPaths":"hdfs:\/\/nn:8020\/data\/in\/2013\/11\/15\/00\/54,hdfs:\/\/nn:8020\/data\/in\/2013\/11\/15\/00\/53,hdfs:\/\/nn:8020\/data\/in\/2013\/11\/15\/00\/52,hdfs:\/\/nng:8020\/data\/in\/2013\/11\/15\/00\/51,hdfs:\/\/nn:8020\/data\/in\/2013\/11\/15\/00\/50","falconInputFeeds":"in","userWorkflowEngine":"oozie","userWorkflowVersion":"1.0","workflowId":"0000086-140929160402261-oozie-oozi-W","cluster":"local","workflowEngineUrl":"http:\/\/oozie:4080\/oozie","subflowId":"0000086-140929160402261-oozie-oozi-W@user-oozie-workflow"}
 

     On Friday, October 3, 2014 7:11 PM, Seetharam Venkatesh <venkatesh@innerzeal.com>
wrote:
   

 Can you please check the instancePaths*.csv under staging dir and see if it
had any contents in it.

On Fri, Oct 3, 2014 at 9:38 AM, Peng Zhang <pengzhang@yahoo-inc.com.invalid>
wrote:

> Hi Falcon Devs,
>
> I run the example apps in the src package and the work flow does work
> well. However, after successfully finishing the job, I did’t see any
> retention job scheduled neither does my input and output path get deleted.
> Attachment is the sample in-feed.xml I have. In addition, I can not see any
> errors in ooze log/job logs. Could you please help me figure out what the
> problem might be?
> Thanks in advance.
>
> Best,
> Peng
>
> feed description="input" name="in" xmlns="uri:falcon:feed:0.1">
>    <groups>input</groups>
>
>    <frequency>minutes(1)</frequency>
>    <timezone>UTC</timezone>
>    <late-arrival cut-off="minutes(3)"/>
>
>    <clusters>
>        <cluster name="local">
>            <validity start="2013-11-15T00:00Z" end="2013-11-15T00:50Z"/>
>            <retention limit="minutes(15)" action="delete"/>
>        </cluster>
>    </clusters>
>
>    <locations>
>        <location type="data"
> path="/data/in/${YEAR}/${MONTH}/${DAY}/${HOUR}/${MINUTE}"/>
>    </locations>
>
>    <ACL owner="pengzhang" group="hdfs" permission="0x777"/>
>    <schema location="/schema/log/log.format.csv" provider="csv"/>
>    <properties>
>
>


-- 
Regards,
Venkatesh

“Perfection (in design) is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.”
- Antoine de Saint-Exupéry

   
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message