falcon-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From b...@apache.org
Subject falcon git commit: FALCON-1937 Cluster update documentation
Date Fri, 20 May 2016 22:14:33 GMT
Repository: falcon
Updated Branches:
  refs/heads/master 823c7d1d8 -> b53cc7090


FALCON-1937 Cluster update documentation

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Ying Zheng <yzheng@hortonworks.com>"

Closes #153 from bvellanki/FALCON-1937


Project: http://git-wip-us.apache.org/repos/asf/falcon/repo
Commit: http://git-wip-us.apache.org/repos/asf/falcon/commit/b53cc709
Tree: http://git-wip-us.apache.org/repos/asf/falcon/tree/b53cc709
Diff: http://git-wip-us.apache.org/repos/asf/falcon/diff/b53cc709

Branch: refs/heads/master
Commit: b53cc7090a372b24f1adef45f5321648a2add8cc
Parents: 823c7d1
Author: bvellanki <bvellanki@hortonworks.com>
Authored: Fri May 20 15:14:29 2016 -0700
Committer: bvellanki <bvellanki@hortonworks.com>
Committed: Fri May 20 15:14:29 2016 -0700

----------------------------------------------------------------------
 docs/src/site/twiki/FalconDocumentation.twiki | 41 +++++++++++++++++++---
 1 file changed, 37 insertions(+), 4 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/falcon/blob/b53cc709/docs/src/site/twiki/FalconDocumentation.twiki
----------------------------------------------------------------------
diff --git a/docs/src/site/twiki/FalconDocumentation.twiki b/docs/src/site/twiki/FalconDocumentation.twiki
index 1411773..a8ea70d 100644
--- a/docs/src/site/twiki/FalconDocumentation.twiki
+++ b/docs/src/site/twiki/FalconDocumentation.twiki
@@ -8,6 +8,7 @@
    * <a href="#Retention">Retention</a>
    * <a href="#Replication">Replication</a>
    * <a href="#Cross_entity_validations">Cross entity validations</a>
+   * <a href="#Updating_cluster_entity_definition">Updating cluster entity</a>
    * <a href="#Updating_process_and_feed_definition">Updating process and feed definition</a>
    * <a href="#Handling_late_input_data">Handling late input data</a>
    * <a href="#Idempotency">Idempotency</a>
@@ -249,9 +250,14 @@ entity from the falcon configuration store. Delete operation on an entity
would
 no dependent entities on the deleted entity.
 
 ---+++ Update
-Update operation allows an already submitted/scheduled entity to be updated. Cluster update
is currently
-not allowed. Feed update can cause cascading update to all the processes already scheduled.
Process update triggers
-update in falcon if entity is updated. The following set of actions are performed in scheduler
to realize an update:
+Update operation allows an already submitted/scheduled entity to be updated. Feed update
can cause cascading update to
+all the processes already scheduled. Process update triggers update in falcon if entity is
scheduled.
+
+Cluster update will require user to  update dependent Feed and Process entities that are
already scheduled.
+Cluster update needs to be performed in safemode. We provide a CLI command for the user to
update the scheduled
+dependent entities after cluster update and exiting safemode.
+
+The following set of actions are performed in scheduler to realize an update:
    * Update the old scheduled entity to set the end time to "now"
    * Schedule as per the new process/feed definition with the start time as "now"
 
@@ -615,10 +621,37 @@ Failure to follow any of the above rules would result in a process submission
fa
 present in the system for the specified time period, the process can be submitted and scheduled,
but all instances
 created would result in a WAITING state unless data is actually provided in the cluster.
 
+---++ Updating cluster entity definition
+Cluster entities can be updated when the user wants to change their interface endpoints or
properties,
+e.g. hadoop clusters updated from unsecure to secure; hadoop cluster moved from non high-availability
to high-availability, etc.
+
+In these scenarios, user would want to change the cluster entity to reflect updated interface
endpoints or properties.
+Updating cluster would require cascading update to dependent feed/process jobs scheduled
on this cluster. So Falcon only allows
+Cluster update when
+   * Falcon server is in safemode.
+   * The update is requested by superuser
+   * The underlying namenode or workflow engine referenced by interface URI is the same.
It is only the URI that has changed to reflect secure/HA environments.
+
+Cluster entity should be updated by superuser using following CLI command.
+<verbatim>
+bash$ falcon entity -type cluster -name primaryCluster -update -file ~/primary-updated.xml
+</verbatim>
+
+Once the cluster entity is updated, user should exit FalconServer from safemode and update
the scheduled entities that are
+dependent on this Cluster. In case of an error during update, user should address the root
cause of failure and retry
+the command. For example : if the cluster has 10 dependent entities and the updateClusterDependents
command failed
+after updating 6th entity, rerun of this command will only update entities 7 to 10.
+<verbatim>
+bash$ falcon entity -updateClusterDependents -cluster primaryCluster
+</verbatim>
 
+Please Refer to [[falconcli/FalconCLI][Falcon CLI]] for more details on usage of CLI commands.
 
 ---++ Updating process and feed definition
-Any changes in feed/process can be done by updating its definition. After the update, any
new workflows which are to be scheduled after the update call will pick up the new changes.
Feed/process name and start time can't be updated. Updating a process triggers updates to
the workflow that is triggered in the workflow engine. Updating feed updates feed workflows
like retention, replication etc. and also updates the processes that reference the feed.
+Any changes in feed/process can be done by updating its definition. After the update, any
new workflows which are to
+be scheduled after the update call will pick up the new changes. Feed/process name and start
time can't be updated.
+Updating a process triggers updates to the workflow that is triggered in the workflow engine.
Updating feed updates
+feed workflows like retention, replication etc. and also updates the processes that reference
the feed.
 
 
 ---++ Handling late input data


Mime
View raw message