falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From prashant madaan <prashantmadaan...@gmail.com>
Subject Replication Problem
Date Thu, 26 Nov 2015 15:24:33 GMT
Hi team,
Hope you are doing good.

I was doing some testing on falcon but stuck up with below issue. looking
for some quick response on this.

The issue I am facing are :

* Replication Issues :*

I have a process that runs an oozie workflow and passes the time parameters
of 7 hours before the current time of falcon job . Below is the value i
pass :
<property name="feed_date" value="${formatTime(dateOffset(instanceTime(),
-7, 'HOUR'),'yyyy-MM-dd')}" />

<property name="feed_hour" value="${formatTime(dateOffset(instanceTime(),
-7, 'HOUR'),'HH')}" />

I have built a feed for replication and mentioned the late arrival cutoff
to be hours(10).

But the issue is that the the data comes late, approximately 7 hours from
the current running time and the feed entity runs on the current time . The
late cut off works for 4 5 iterations of this but then it stops the
replication .

Note: The frequency of the feed entity as well as the process entity  is
hours(1).

Example :

Suppose Current time is 10 AM

The data comes for 3 AM hour.

I schedule this process at 10 am and the data gets picked up and is
processed and the feed replication runs fine .

Similar things happen for the next 3 4 iterations , but after that the feed
replication does not kick off at all (Thus i cannot check the logs) but the
process entity runs fine .


Please let me know how to fix this issue . Also can we make falcon run 7
hours late from the current time . For example

Feed Replication Path contains /xyz/${YEAR}-${MONTH}-${DAY}/${HOUR}/abc

This picks up the current falcon time . But i want it to always run 7 hours
behind the current time .


Any Help is appreciated .

Thanks and Regards

Prashant Madaan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message