falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venkatesh Seetharam (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-306) Feed late replication is not working
Date Tue, 08 Apr 2014 18:25:18 GMT

    [ https://issues.apache.org/jira/browse/FALCON-306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13963275#comment-13963275
] 

Venkatesh Seetharam commented on FALCON-306:
--------------------------------------------

Thanks [~shaik.idris] and [~karishmag9] for jumping in and helping us out. SIncerely appreciate
it.

However, I ran an end to end test yesterday on latest code and I was able to see late reruns
were working as expected. 

Verified that it works fine.

{code}
2014-04-08 02:59:59,098 DEBUG - [LaterunHandler:] ~ Dequeued Message:clusterName=bcp-cluster*wfId=0000037-140405205856593-oozie-oozi-W*msgInsertTime=1396918800000*delayInMilliSec=7200000*entityType=feed*entityName=late-replicating-feed*instance=2014-04-08T01:00Z*runId=0*workflowUser=hrt_qa
(DelayedQueue:85)
2014-04-08 02:59:59,098 INFO  - [LaterunHandler:] ~ Logging in hrt_qa (CurrentUser:52)
2014-04-08 02:59:59,174 INFO  - [LaterunHandler:] ~ fs.default.name is deprecated. Instead,
use fs.defaultFS (deprecation:1009)
2014-04-08 02:59:59,248 INFO  - [LaterunHandler:] ~ Recorded size:50 is different from new
size200 (LateDataHandler:283)
2014-04-08 02:59:59,250 INFO  - [LaterunHandler:] ~ Late changes detected in the following
feeds: late-replicating-feed (AbstractRerunConsumer:75)
2014-04-08 02:59:59,539 INFO  - [LaterunHandler:] ~ Rerun job 0000037-140405205856593-oozie-oozi-W
on cluster bcp-cluster (OozieWorkflowEngine:1136)
2014-04-08 02:59:59,539 INFO  - [LaterunHandler:] ~ Scheduled late rerun for wf-id: 0000037-140405205856593-oozie-oozi-W
on cluster: bcp-cluster (AbstractRerunConsumer:78)
{code}

Also, notice that 0000037-140405205856593-oozie-oozi-W has runId of 1 instead of 0.

> Feed late replication is not working
> ------------------------------------
>
>                 Key: FALCON-306
>                 URL: https://issues.apache.org/jira/browse/FALCON-306
>             Project: Falcon
>          Issue Type: Bug
>            Reporter: Raghav Kumar Gautam
>              Labels: falcon-regression
>         Attachments: falcon-306-upload.zip, falcon.application.log
>
>
> *STR*
> - Source clusters have following folders:
>   at colo=ua3, partition=UK/${cluster.colo} folder=input-data/2014/02/05/22/08/UK/ua3
>   at colo=ua2, partition=US/${cluster.colo} folder=input-data/2014/02/05/22/08/US/ua2
> - Target cluster
>   at colo=ua1: folder=input-data/2014/02/05/22/08/
> - Upload some data in source folders
> - Replication goes through
> - Upload some more data in source folders
> - Late replication does not happen
> *Test*
> PrismFeedLateReplicationTest#multipleSourceOneTarget_pastData
> PrismFeedLateReplicationTest#multipleSourceOneTarget_futureData



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message