falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajay Yadava (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-1406) Effective time in Entity updates.
Date Sun, 28 Aug 2016 19:14:20 GMT

    [ https://issues.apache.org/jira/browse/FALCON-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15443958#comment-15443958

Ajay Yadava commented on FALCON-1406:


Please update the old review request instead of creating a new one, otherwise the old context
and review comments get lost after the update. For now, I have given my comments, on the new
review board request.

Some correctness and functional issues which I noticed and I feel are worth capturing here
for everyone's consideration.
1. This feature will break entity sla features as it contains some instances for monitoring
sla and they might be affected with entity update(and sla update along with it)
2. Touch command is also a way to do an update, however it is not updated to accept effectiveTime
3. This feature will also not work in cases where input feed instances may have been deleted
by retention and on passing effectiveTime from past those instances will not succeed.
4. Instance search feature will also break by these changes as we are not taking care of existing
graph db instances.
5. After several effective updates, it will be hard to keep track of which workflow/code worked
with which input/jars.

> Effective time in Entity updates.
> ---------------------------------
>                 Key: FALCON-1406
>                 URL: https://issues.apache.org/jira/browse/FALCON-1406
>             Project: Falcon
>          Issue Type: New Feature
>            Reporter: sandeep samudrala
>            Assignee: sandeep samudrala
>         Attachments: FALCON-1406-initial.patch, effective_time_in_entity_updates.pdf
> Effective time with entity updates needs to be provided even with past time too. There
was effective time capability provided in the past which gives the functionality to set an
effective time for an entity with only current or future time(now + delay), which could not
solve all the issues. 
> Following are few scenarios which would require effective time to be available with time
back in past.
> a) New code being deployed for an incompatible input data set which would leave instances
with old code and new data.
> b) Bad code being pushed for which, the entity should be able to go back in time to replay(rerun)
with new code.
> c) Orchestration level changes(good/bad) would need functionality to go back in time
to start with.
> For reference: Linking all the Jiras that have been worked upon around effective time
> https://issues.apache.org/jira/browse/FALCON-374
> https://issues.apache.org/jira/browse/FALCON-297

This message was sent by Atlassian JIRA

View raw message