falcon-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pall...@apache.org
Subject svn commit: r1730446 [5/6] - in /falcon/trunk/releases/0.9: ./ src/ src/site/ src/site/resources/ src/site/resources/images/ src/site/resources/images/logos/ src/site/resources/images/profiles/ src/site/twiki/ src/site/twiki/falconcli/ src/site/twiki/r...
Date Mon, 15 Feb 2016 05:08:34 GMT
Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/RunningInstance.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/RunningInstance.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/RunningInstance.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/RunningInstance.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,13 @@
+---+++Running
+
+[[CommonCLI][Common CLI Options]]
+
+Running option provides all the running instances of the mentioned process.
+
+Usage:
+$FALCON_HOME/bin/falcon instance -type <<feed/process>> -name <<name>> -running
+
+Optional Args : -colo <<colo>> -lifecycle <<lifecycles>>
+-filterBy <<field1:value1,field2:value2>> -orderBy <<field>> -sortOrder <<sortOrder>> -offset 0 -numResults 10
+
+<a href="../Restapi/InstanceRunning.html">Optional params described here.</a>
\ No newline at end of file

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/SLAAlert.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/SLAAlert.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/SLAAlert.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/SLAAlert.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,49 @@
+---+++SLAAlert
+
+[[CommonCLI][Common CLI Options]]
+
+<verbatim>
+Since: 0.8
+</verbatim>
+
+This command lists all the feed instances which have missed sla and are still not available. If a feed instance missed
+sla but is now available, then it will not be reported in results. The purpose of this API is alerting and hence it
+ doesn't return feed instances which missed SLA but are available as they don't require any action.
+
+* Currently sla monitoring is supported only for feeds.
+
+* Option end is optional and will default to current time if missing.
+
+* Option name is optional, if provided only instances of that feed will be considered.
+
+Usage:
+
+*Example 1*
+
+*$FALCON_HOME/bin/falcon entity -type feed -start 2014-09-05T00:00Z -slaAlert  -end 2016-05-03T00:00Z -colo local*
+
+name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T11:59Z, tags: Missed SLA High
+name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:00Z, tags: Missed SLA High
+name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:01Z, tags: Missed SLA High
+name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:02Z, tags: Missed SLA High
+name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:03Z, tags: Missed SLA High
+name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:04Z, tags: Missed SLA High
+name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:05Z, tags: Missed SLA High
+name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:06Z, tags: Missed SLA High
+name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:07Z, tags: Missed SLA High
+name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:08Z, tags: Missed SLA Low
+
+
+Response: default/Success!
+
+Request Id: default/216978070@qtp-830047511-4 - f5a6c129-ab42-4feb-a2bf-c3baed356248
+
+*Example 2*
+
+*$FALCON_HOME/bin/falcon entity -type feed -start 2014-09-05T00:00Z -slaAlert  -end 2016-05-03T00:00Z -colo local -name in*
+
+name: in, type: FEED, cluster: local, instanceTime: 2015-09-26T06:00Z, tags: Missed SLA High
+
+Response: default/Success!
+
+Request Id: default/1580107885@qtp-830047511-7 - f16cbc51-5070-4551-ad25-28f75e5e4cf2

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/Schedule.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/Schedule.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/Schedule.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/Schedule.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,22 @@
+---+++Schedule
+
+[[CommonCLI][Common CLI Options]]
+
+Once submitted, an entity can be scheduled using schedule option. Process and feed can only be scheduled.
+
+Usage:
+$FALCON_HOME/bin/falcon entity  -type [process|feed] -name <<name>> -schedule
+
+Optional Args :
+
+-skipDryRun When this argument is specified, Falcon skips oozie dryrun.
+
+-doAs <username>
+
+-properties <<key1:val1,...,keyN:valN>>. Specifying 'falcon.scheduler:native' as a property will schedule the entity on the the native scheduler of Falcon. Else, it will default to the engine specified in startup.properties. For details on Native scheduler, refer to [[FalconNativeScheduler][Falcon Native Scheduler]]
+
+Examples:
+
+ $FALCON_HOME/bin/falcon entity  -type process -name sampleProcess -schedule
+
+ $FALCON_HOME/bin/falcon entity  -type process -name sampleProcess -schedule -properties falcon.scheduler:native

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/StatusAdmin.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/StatusAdmin.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/StatusAdmin.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/StatusAdmin.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,8 @@
+---+++Status
+
+[[CommonCLI][Common CLI Options]]
+
+Status returns the current state of Falcon (running or stopped).
+Usage:
+$FALCON_HOME/bin/falcon admin -status
+

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/StatusEntity.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/StatusEntity.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/StatusEntity.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/StatusEntity.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,8 @@
+---+++Status
+
+[[CommonCLI][Common CLI Options]]
+
+Status returns the current status of the entity.
+
+Usage:
+$FALCON_HOME/bin/falcon entity -type [cluster|datasource|feed|process] -name <<name>> -status
\ No newline at end of file

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/StatusInstance.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/StatusInstance.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/StatusInstance.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/StatusInstance.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,20 @@
+---+++Status
+
+[[CommonCLI][Common CLI Options]]
+
+Status option via CLI can be used to get the status of a single or multiple instances.  If the instance is not yet materialized but is within the process validity range, WAITING is returned as the state. Along with the status of the instance time is also returned. Log location gives the oozie workflow url
+If the instance is in WAITING state, missing dependencies are listed.
+The job urls are populated for all actions of user workflow and non-succeeded actions of the main-workflow. The user then need not go to the underlying scheduler to get the job urls when needed to debug an issue in the job.
+
+Example : Suppose a process has 3 instance, one has succeeded,one is in running state and other one is waiting, the expected output is:
+
+{"status":"SUCCEEDED","message":"getStatus is successful","instances":[{"instance":"2012-05-07T05:02Z","status":"SUCCEEDED","logFile":"http://oozie-dashboard-url"},{"instance":"2012-05-07T05:07Z","status":"RUNNING","logFile":"http://oozie-dashboard-url"}, {"instance":"2010-01-02T11:05Z","status":"WAITING"}]
+
+Usage:
+$FALCON_HOME/bin/falcon instance -type <<feed/process>> -name <<name>> -status
+
+Optional Args : -start "yyyy-MM-dd'T'HH:mm'Z'" -end "yyyy-MM-dd'T'HH:mm'Z'" -colo <<colo>>
+-filterBy <<field1:value1,field2:value2>> -lifecycle <<lifecycles>>
+-orderBy field -sortOrder <<sortOrder>> -offset 0 -numResults 10
+
+<a href="../Restapi/InstanceStatus.html"> Optional params described here.</a>

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/Submit.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/Submit.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/Submit.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/Submit.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,13 @@
+---+++Submit
+
+[[CommonCLI][Common CLI Options]]
+
+Submit option is used to set up entity definition.
+
+Usage:
+$FALCON_HOME/bin/falcon entity -submit -type [cluster|datasource|feed|process] -file <entity-definition.xml>
+
+Example:
+$FALCON_HOME/bin/falcon entity -submit -type cluster -file /cluster/definition.xml
+
+Note: The url option in the above and all subsequent commands is optional. If not mentioned it will be picked from client.properties file. If the option is not provided and also not set in client.properties, Falcon CLI will fail.

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/SubmitRecipe.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/SubmitRecipe.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/SubmitRecipe.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/SubmitRecipe.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,17 @@
+---+++ Submit Recipe
+
+[[CommonCLI][Common CLI Options]]
+
+Submit the specified recipe.
+
+Usage:
+$FALCON_HOME/bin/falcon recipe -name <name>
+Name of the recipe. User should have defined <name>-template.xml and <name>.properties in the path specified by falcon.recipe.path in client.properties file. falcon.home path is used if its not specified in client.properties file.
+If its not specified in client.properties file and also if files cannot be found at falcon.home, Falcon CLI will fail.
+
+Optional Args : -tool <recipeToolClassName>
+Falcon provides a base tool that recipes can override. If this option is not specified the default Recipe Tool
+RecipeTool defined is used. This option is required if user defines his own recipe tool class.
+
+Example:
+$FALCON_HOME/bin/falcon recipe -name hdfs-replication
\ No newline at end of file

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/SummaryEntity.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/SummaryEntity.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/SummaryEntity.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/SummaryEntity.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,14 @@
+---+++Summary
+
+[[CommonCLI][Common CLI Options]]
+
+Summary of entities of a particular type and a cluster will be listed. Entity summary has N most recent instances of entity.
+
+Usage:
+$FALCON_HOME/bin/falcon entity -type [feed|process] -summary
+
+Optional Args : -start "yyyy-MM-dd'T'HH:mm'Z'" -end "yyyy-MM-dd'T'HH:mm'Z'" -fields <<field1,field2>>
+-filterBy <<field1:value1,field2:value2>> -tags <<tagkey=tagvalue,tagkey=tagvalue>>
+-orderBy <<field>> -sortOrder <<sortOrder>> -offset 0 -numResults 10 -numInstances 7
+
+<a href="../Restapi/EntitySummary.html">Optional params described here.</a>
\ No newline at end of file

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/SummaryInstance.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/SummaryInstance.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/SummaryInstance.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/SummaryInstance.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,20 @@
+---+++Summary
+
+[[CommonCLI][Common CLI Options]]
+
+Summary option via CLI can be used to get the consolidated status of the instances between the specified time period.
+Each status along with the corresponding instance count are listed for each of the applicable colos.
+The unscheduled instances between the specified time period are included as UNSCHEDULED in the output to provide more clarity.
+
+Example : Suppose a process has 3 instance, one has succeeded,one is in running state and other one is waiting, the expected output is:
+
+{"status":"SUCCEEDED","message":"getSummary is successful", instancesSummary:[{"cluster": <<name>> "map":[{"SUCCEEDED":"1"}, {"WAITING":"1"}, {"RUNNING":"1"}]}]}
+
+Usage:
+$FALCON_HOME/bin/falcon instance -type <<feed/process>> -name <<name>> -summary
+
+Optional Args : -start "yyyy-MM-dd'T'HH:mm'Z'" -end "yyyy-MM-dd'T'HH:mm'Z'" -colo <<colo>>
+-filterBy <<field1:value1,field2:value2>> -lifecycle <<lifecycles>>
+-orderBy field -sortOrder <<sortOrder>>
+
+<a href="../Restapi/InstanceSummary.html">Optional params described here.</a>

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/SuspendEntity.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/SuspendEntity.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/SuspendEntity.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/SuspendEntity.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,8 @@
+---+++Suspend
+
+[[CommonCLI][Common CLI Options]]
+
+Suspend on an entity results in suspension of the oozie bundle that was scheduled earlier through the schedule function. No further instances are executed on a suspended entity. Only schedule-able entities(process/feed) can be suspended.
+
+Usage:
+$FALCON_HOME/bin/falcon entity  -type [feed|process] -name <<name>> -suspend
\ No newline at end of file

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/SuspendInstance.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/SuspendInstance.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/SuspendInstance.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/SuspendInstance.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,8 @@
+---+++Suspend
+
+[[CommonCLI][Common CLI Options]]
+
+Suspend is used to suspend a instance or instances  for the given process. This option pauses the parent workflow at the state, which it was in at the time of execution of this command.
+
+Usage:
+$FALCON_HOME/bin/falcon instance -type <<feed/process>> -name <<name>> -suspend -start "yyyy-MM-dd'T'HH:mm'Z'" -end "yyyy-MM-dd'T'HH:mm'Z'"

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/Touch.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/Touch.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/Touch.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/Touch.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,10 @@
+---+++Touch
+
+[[CommonCLI][Common CLI Options]]
+
+Force Update operation allows an already submitted/scheduled entity to be updated.
+
+Usage:
+$FALCON_HOME/bin/falcon entity  -type [feed|process] -name <<name>> -touch
+
+Optional Arg : -skipDryRun. When this argument is specified, Falcon skips oozie dryrun.
\ No newline at end of file

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/TriageInstance.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/TriageInstance.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/TriageInstance.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/TriageInstance.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,9 @@
+---+++Triage
+
+[[CommonCLI][Common CLI Options]]
+
+Given a feed/process instance this command traces it's ancestors to find what all ancestors have failed. It's useful if
+lot of instances are failing in a pipeline as it then finds out the root cause of the pipeline being stuck.
+
+Usage:
+$FALCON_HOME/bin/falcon instance -triage -type <<feed/process>> -name <<name>> -start "yyyy-MM-dd'T'HH:mm'Z'"

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/UpdateEntity.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/UpdateEntity.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/UpdateEntity.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/UpdateEntity.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,14 @@
+---+++Update
+
+[[CommonCLI][Common CLI Options]]
+
+Update operation allows an already submitted/scheduled entity to be updated. Cluster and datasource updates are
+currently not allowed.
+
+Usage:
+$FALCON_HOME/bin/falcon entity  -type [feed|process] -name <<name>> -update -file <<path_to_file>>
+
+Optional Arg : -skipDryRun. When this argument is specified, Falcon skips oozie dryrun.
+
+Example:
+$FALCON_HOME/bin/falcon entity -type process -name hourly-reports-generator -update -file /process/definition.xml

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/VersionAdmin.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/VersionAdmin.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/VersionAdmin.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/VersionAdmin.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,7 @@
+---+++Version
+
+[[CommonCLI][Common CLI Options]]
+
+Version returns the current version of Falcon installed.
+Usage:
+$FALCON_HOME/bin/falcon admin -version

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/VertexEdgesMetadata.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/VertexEdgesMetadata.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/VertexEdgesMetadata.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/VertexEdgesMetadata.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,12 @@
+---+++ Vertex Edges
+
+[[CommonCLI][Common CLI Options]]
+
+Get the adjacent vertices or edges of the vertex with the specified direction.
+
+Usage:
+$FALCON_HOME/bin/falcon metadata -edges -id <<vertex-id>> -direction <<direction>>
+
+Example:
+$FALCON_HOME/bin/falcon metadata -edges -id 4 -direction both
+$FALCON_HOME/bin/falcon metadata -edges -id 4 -direction inE

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/VertexMetadata.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/VertexMetadata.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/VertexMetadata.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/VertexMetadata.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,11 @@
+---+++ Vertex
+
+[[CommonCLI][Common CLI Options]]
+
+Get the vertex with the specified id.
+
+Usage:
+$FALCON_HOME/bin/falcon metadata -vertex -id <<id>>
+
+Example:
+$FALCON_HOME/bin/falcon metadata -vertex -id 4

Added: falcon/trunk/releases/0.9/src/site/twiki/falconcli/VerticesMetadata.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/falconcli/VerticesMetadata.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/falconcli/VerticesMetadata.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/falconcli/VerticesMetadata.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,11 @@
+---+++ Vertices
+
+[[CommonCLI][Common CLI Options]]
+
+Get all vertices for a key index given the specified value.
+
+Usage:
+$FALCON_HOME/bin/falcon metadata -vertices -key <<key>> -value <<value>>
+
+Example:
+$FALCON_HOME/bin/falcon metadata -vertices -key type -value feed-instance

Added: falcon/trunk/releases/0.9/src/site/twiki/index.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/index.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/index.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/index.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,43 @@
+---+ Falcon - Feed management and data processing platform
+
+Falcon is a feed processing and feed management system aimed at making it
+easier for end consumers to onboard their feed processing and feed
+management on hadoop clusters.
+
+---++ Why?
+
+   * Establishes relationship between various data and processing elements on a Hadoop environment
+
+   * Feed management services such as feed retention, replications across clusters, archival etc.
+
+   * Easy to onboard new workflows/pipelines, with support for late data handling, retry policies
+
+   * Integration with metastore/catalog such as Hive/HCatalog
+
+   * Provide notification to end customer based on availability of feed groups
+     (logical group of related feeds, which are likely to be used together)
+
+   * Enables use cases for local processing in colo and global aggregations
+
+   * Captures Lineage information for feeds and processes
+
+---+ Getting Started
+
+Start with these simple steps to install an falcon instance [[InstallationSteps][Simple setup]]. Also refer
+to Falcon architecture and documentation in [[FalconDocumentation][Documentation]]. [[OnBoarding][On boarding]]
+describes steps to on-board a pipeline to Falcon. It also gives a sample pipeline for reference.
+[[EntitySpecification][Entity Specification]] gives complete details of all Falcon entities.
+
+[[falconcli/FalconCLI][Falcon CLI]] implements [[restapi/ResourceList][Falcon's RESTful API]] and
+describes various options for the command line utility provided by Falcon.
+
+Falcon provides OOTB [[HiveIntegration][lifecycle management for Tables in Hive (HCatalog)]]
+such as table replication for BCP and table eviction. Falcon also enforces
+[[Security][Security]] on protected resources and enables SSL.
+
+#LicenseInfo
+---+ Licensing Information
+
+Falcon is distributed under [[http://www.apache.org/licenses/LICENSE-2.0][Apache License 2.0]].
+
+

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/AdjacentVertices.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/AdjacentVertices.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/AdjacentVertices.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/AdjacentVertices.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,91 @@
+---++  GET api/metadata/lineage/vertices/:id/:direction
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Get a list of adjacent vertices or edges with a direction.
+
+---++ Parameters
+   * :id is the id of the vertex.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+   * :direction is the direction associated with the edges.
+
+   To get the adjacent out vertices of vertex pass direction as out, in to get adjacent in vertices
+   and both to get both in and out adjacent vertices. Similarly to get the out edges of vertex
+   pass outE, inE to get in edges and bothE to get the both in and out edges of vertex.
+
+      * out  : get the adjacent out vertices of vertex
+      * in   : get the adjacent in vertices of vertex
+      * both : get the both adjacent in and out vertices of vertex
+      * outCount  : get the number of out vertices of vertex
+      * inCount   : get the number of in vertices of vertex
+      * bothCount : get the number of adjacent in and out vertices of vertex
+      * outIds  : get the identifiers of out vertices of vertex
+      * inIds   : get the identifiers of in vertices of vertex
+      * bothIds : get the identifiers of adjacent in and out vertices of vertex
+
+---++ Results
+Adjacent vertices of the vertex for the specified direction.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/metadata/lineage/vertices/4/out
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "results": [
+        {
+            "timestamp":"2014-04-21T20:55Z",
+            "name":"sampleFeed",
+            "type":"feed-instance",
+            "_id":8,
+            "_type":"vertex"
+        }
+    ],
+    "totalSize":1}
+}
+</verbatim>
+
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/metadata/lineage/vertices/4/bothE
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "results":[
+        {
+            "_id":"Q5V-4-5g",
+            "_type":"edge",
+            "_outV":4,
+            "_inV":8,
+            "_label":"output"
+        }
+    ],
+    "totalSize":1
+}
+</verbatim>
+
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/metadata/lineage/vertices/4/bothE?doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "results":[
+        {
+            "_id":"Q5V-4-5g",
+            "_type":"edge",
+            "_outV":4,
+            "_inV":8,
+            "_label":"output"
+        }
+    ],
+    "totalSize":1
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/AdminConfig.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/AdminConfig.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/AdminConfig.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/AdminConfig.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,35 @@
+---++  GET /api/admin/config/:config-type
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Get configuration information of the falcon server.
+
+---++ Parameters
+   * :config-type can be build, deploy, startup or runtime
+
+---++ Results
+Configuration information of the server.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/admin/config/deploy
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "properties": [
+        {
+            "value": "embedded",
+            "key": "deploy.mode"
+        },
+        {
+            "value": "all",
+            "key": "domain"
+        }
+    ]
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/AdminStack.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/AdminStack.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/AdminStack.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/AdminStack.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,40 @@
+---++  GET /api/admin/stack
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Get stack trace of the falcon server.
+
+---++ Parameters
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Stack trace of the server.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/admin/stack?doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+Reference Handler
+State: WAITING
+java.lang.Object.wait(Native Method)
+java.lang.Object.wait(Object.java:485)
+java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)Finalizer
+
+...
+
+State: TIMED_WAITING
+sun.misc.Unsafe.park(Native Method)
+java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
+java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:424)
+java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:323)
+java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:874)
+java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:955)
+java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:917)
+java.lang.Thread.run(Thread.java:695)
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/AdminVersion.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/AdminVersion.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/AdminVersion.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/AdminVersion.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,35 @@
+---++  GET /api/admin/version
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Get version of the falcon server.
+
+---++ Parameters
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Version of the server.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/admin/version?doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "properties":[
+        {
+            "key":"Version",
+            "value":"0.4-incubating-SNAPSHOT-rb47788d1112fcf949c22a3860934167237b395b0"
+        },
+        {
+            "key":"Mode",
+            "value":"embedded"
+        }
+    ]
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/AllEdges.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/AllEdges.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/AllEdges.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/AllEdges.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,42 @@
+---++  GET pi/metadata/lineage//edges/all
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Get all edges.
+
+---++ Parameters
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+All edges in lineage graph.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/metadata/lineage/edges/all?doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "results": [
+        {
+            "_id":"Q5V-4-5g",
+            "_type":"edge",
+            "_outV":4,
+            "_inV":8,
+            "_label":"output"
+        },
+        {
+            "_id":"Q6t-c-5g",
+            "_type":"edge",
+            "_outV":12,
+            "_inV":16,
+            "_label":"output"
+        }
+    ],
+    "totalSize": 2
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/AllVertices.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/AllVertices.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/AllVertices.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/AllVertices.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,43 @@
+---++  GET api/metadata/lineage/vertices/all
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Get all vertices.
+
+---++ Parameters
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+All vertices in lineage graph.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/metadata/lineage/vertices/all?doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "results": [
+        {
+            "timestamp":"2014-04-21T20:55Z",
+            "name":"sampleIngestProcess\/2014-03-01T10:00Z",
+            "type":"process-instance",
+            "version":"2.0.0",
+            "_id":4,
+            "_type":"vertex"
+        },
+        {
+            "timestamp":"2014-04-21T20:55Z",
+            "name":"rawEmailFeed\/2014-03-01T10:00Z",
+            "type":"feed-instance",
+            "_id":8,
+            "_type":"vertex"
+        }
+    ],
+    "totalSize": 2
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/Edge.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/Edge.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/Edge.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/Edge.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,34 @@
+---++  GET api/metadata/lineage/edges/:id
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Gets the edge with specified id.
+
+---++ Parameters
+   * :id is the unique id of the edge.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Edge with the specified id.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/metadata/lineage/edges/Q6t-c-5g?doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "results":
+        {
+            "_id":"Q6t-c-5g",
+            "_type":"edge",
+            "_outV":12,
+            "_inV":16,
+            "_label":"output"
+        }
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityDefinition.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityDefinition.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityDefinition.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityDefinition.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,53 @@
+---++  GET /api/entities/definition/:entity-type/:entity-name
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Get definition of the entity.
+
+---++ Parameters
+   * :entity-type can be cluster, feed or process.
+   * :entity-name is name of the entity.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Definition of the entity.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/entities/definition/process/SampleProcess?doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
+<process name="SampleProcess" xmlns="uri:falcon:process:0.1">
+  <clusters>
+    <cluster name="primary-cluster">
+      <validity start="2012-04-03T06:00Z" end="2022-12-30T00:00Z"/>
+    </cluster>
+  </clusters>
+  <parallel>1</parallel>
+  <order>FIFO</order>
+  <frequency>hours(1)</frequency>
+  <timezone>UTC</timezone>
+  <inputs>
+    <input name="input" feed="SampleInput" start="yesterday(0,0)" end="today(-1,0)"/>
+  </inputs>
+  <outputs>
+    <output name="output" feed="SampleOutput" instance="yesterday(0,0)"/>
+  </outputs>
+  <properties>
+    <property name="queueName" value="default"/>
+    <property name="ssh.host" value="localhost"/>
+    <property name="fileTimestamp" value="${coord:formatTime(coord:nominalTime(), 'yyyy-MM-dd')}"/>
+  </properties>
+  <workflow engine="oozie" path="/examples/apps/aggregator"/>
+  <retry policy="exp-backoff" delay="minutes(5)" attempts="3"/>
+  <late-process policy="exp-backoff" delay="hours(1)">
+    <late-input input="input" workflow-path="/projects/bootcamp/workflow/lateinput"/>
+  </late-process>
+</process>
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityDelete.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityDelete.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityDelete.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityDelete.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,31 @@
+---++  DELETE /api/entities/delete/:entity-type/:entity-name
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Delete the specified entity.
+
+---++ Parameters
+   * :entity-type can be feed or process.
+   * :entity-name is name of the feed or process.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Results of the delete operation.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+DELETE http://localhost:15000/api/entities/delete/cluster/SampleProcess?doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "requestId": "falcon\/17ff6ca6-1c8a-459f-9ba8-8fec480e384a\n",
+    "message": "falcon\/SampleProcess(cluster) removed successfully\n",
+    "status": "SUCCEEDED"
+}
+</verbatim>
+

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityDependencies.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityDependencies.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityDependencies.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityDependencies.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,43 @@
+---++  GET /api/entities/dependencies/:entity-type/:entity-name
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Get dependencies of the entity.
+
+---++ Parameters
+   * :entity-type can be cluster, feed or process.
+   * :entity-name is name of the entity.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Dependenciess of the entity.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/entities/dependencies/process/SampleProcess?doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "entity": [
+        {
+            "name": "SampleInput",
+            "type": "feed",
+            "tag": [Input]
+        },
+        {
+            "name": "SampleOutput",
+            "type": "feed"
+            "tag": [Output]
+        },
+        {
+            "name": "primary-cluster",
+            "type": "cluster"
+        }
+    ]
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityLineage.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityLineage.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityLineage.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityLineage.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,40 @@
+---++  GET api/metadata/lineage/entities?pipeline=:pipeline
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+It returns the graph depicting the relationship between the various processes and feeds in a given pipeline.
+
+---++ Parameters
+   * :pipeline is the name of the pipeline
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+It returns a json graph
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/metadata/lineage/entities?pipeline=my-pipeline&doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "vertices": ["my-minutely-process", "my-hourly-process"],
+    "edges":
+    [
+        {
+         "from"  : "my-minutely-process",
+         "to"    : "my-hourly-process",
+         "label" : "my-minutely-feed"
+        },
+        {
+         "from"  : "my-hourly-process",
+         "to"    : "my-minutely-process",
+         "label" : "my-hourly-feedback"
+        }
+    ]
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityList.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityList.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityList.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityList.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,164 @@
+---++  GET /api/entities/list/:entity-type?fields=:fields
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Get list of the entities.
+
+---++ Parameters
+   * :entity-type Comma-separated entity types. Can be empty. Valid entity types are cluster, feed or process.
+   * fields <optional param> Fields of entity that the user wants to view, separated by commas.
+      * Valid options are STATUS, TAGS, PIPELINES, CLUSTERS.
+   * nameseq <optional param> Subsequence of entity name. Not case sensitive.
+      * The entity name needs to contain all the characters in the subsequence in the same order.
+      * Example 1: "sample1" will match the entity named "SampleFeed1-2".
+      * Example 2: "mhs" will match the entity named "New-My-Hourly-Summary".
+   * tagkeys <optional param> Keywords in tags, separated by comma. Not case sensitive.
+      * The returned entities will have tags that match all the tag keywords.
+   * filterBy <optional param> Filter results by list of field:value pairs. Example: filterBy=STATUS:RUNNING,PIPELINES:clickLogs
+      * Supported filter fields are NAME, STATUS, PIPELINES, CLUSTER.
+      * Query will do an AND among filterBy fields.
+   * tags <optional param> Return list of entities that have specified tags, separated by a comma. Query will do AND on tag values.
+      * Example: tags=consumer=consumer@xyz.com,owner=producer@xyz.com
+   * orderBy <optional param> Field by which results should be ordered.
+      * Supports ordering by "name".
+   * sortOrder <optional param> Valid options are "asc" and "desc"
+   * offset <optional param> Show results from the offset, used for pagination. Defaults to 0.
+   * numResults <optional param> Number of results to show per request, used for pagination. Only integers > 0 are valid, Default is 10.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+   * Note:
+      * We have two filtering parameters for entity tags: "tags" and "tagkeys". "tags" does the exact match in key=value fashion, while "tagkeys" finds all the entities with the given key as a substring in the tags. This "tagkeys" filter is introduced for the user who doesn't remember the exact tag but some keywords in the tag. It also helps users to save the time of typing long tags.
+      * The returned entities will match all the filtering criteria.
+
+---++ Results
+Total number of results and a list of entities.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/entities/list/feed
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "totalResults":"2”,
+    "entity": [
+        {
+            "name": "SampleOutput",
+            "type": "feed"
+        },
+        {
+            "name": "SampleInput",
+            "type": "feed"
+        }
+    ]
+}
+</verbatim>
+
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/entities/list
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "totalResults":"4”,
+    "entity": [
+        {
+            "name"  : "SampleCluster1",
+            "type"  : "cluster"
+        }
+        {
+            "name"  : "SampleOutput",
+            "type"  : "feed"
+        },
+        {
+            "name"  : "SampleInput",
+            "type"  : "feed"
+        }
+        {
+            "name"  : "SampleProcess1",
+            "type"  : "process"
+        }
+    ]
+}
+</verbatim>
+
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/entities/list/feed?fields=status
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "totalResults":"2”,
+    "entity": [
+        {
+            "name"  : "SampleOutput",
+            "type"  : "feed",
+            "status": "RUNNING"
+        },
+        {
+            "name": "SampleInput",
+            "type": "feed",
+            "status": "RUNNING"
+        }
+    ]
+}
+</verbatim>
+
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/entities/list/process?filterBy=STATUS:RUNNING,PIPELINES:dataReplication&fields=status,pipelines,tags&tags=consumer=consumer@xyz.com&orderBy=name&offset=2&numResults=2
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "totalResults":"10”,
+    "entity": [
+        {
+            "name"  : "SampleProcess1",
+            "type"  : "process",
+            "status": "RUNNING",
+            "pipelines": "dataReplication",
+            "tags": "consumer=consumer@xyz.com"
+        },
+        {
+            "name": "SampleProcess3",
+            "type": "process",
+            "status": "RUNNING",
+            "pipelines": "dataReplication",
+            "tags": "consumer=consumer@xyz.com,owner=producer@xyz.com"
+        }
+    ]
+}
+</verbatim>
+
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/entities/list/feed,process?nameseq=samplebill&tagkeys=billing,healthcare&numResults=2&offset=1&fields=status,clusters,tags&doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "totalResults":"4”,
+    "entity”:[
+        {
+            "type":"FEED”,
+            "name":"SampleUSHealthBill”,
+            "status":"SUBMITTED”,
+            "tags”: {"tag":["related=ushealthcare","department=billingDepartment"]},
+            "clusters": {"cluster":["SampleCluster1","primaryCluster”]}
+        },
+        {
+            "type":"PROCESS”,
+            "name":"SampleHealthBill”,
+            "status":"SUBMITTED”,
+            "tags”: {"tag":["related=healthcare","department=billingDepartment"]},
+            "clusters": {"cluster":"primaryCluster”}
+        }
+    ]
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityResume.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityResume.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityResume.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityResume.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,30 @@
+---++  POST /api/entities/resume/:entity-type/:entity-name
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Resume a supended entity.
+
+---++ Parameters
+   * :entity-type can either be a feed or a process.
+   * :entity-name is name of the entity.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Result of the resume command.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+POST http://localhost:15000/api/entities/resume/process/SampleProcess?doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "requestId": "default\/106582a9-130f-4903-8b8f-f95d7b286c30\n",
+    "message": "default\/SampleProcess(process) resumed successfully\n",
+    "status": "SUCCEEDED"
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySchedule.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySchedule.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySchedule.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySchedule.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,32 @@
+---++  POST /api/entities/schedule/:entity-type/:entity-name
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Schedule an entity.
+
+---++ Parameters
+   * :entity-type can either be a feed or a process.
+   * :entity-name is name of the entity.
+   * skipDryRun : Optional query param, Falcon skips oozie dryrun when value is set to true.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+
+---++ Results
+Result of the schedule command.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+POST http://localhost:15000/api/entities/schedule/process/SampleProcess?skipDryRun=false&doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "requestId": "default\/ee735c95-98bd-41b8-a705-2e78bcfcdcd9\n",
+    "message": "default\/SampleProcess(process) scheduled successfully\n",
+    "status": "SUCCEEDED"
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityStatus.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityStatus.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityStatus.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityStatus.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,30 @@
+---++  GET /api/entities/status/:entity-type/:entity-name
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Get status of the entity.
+
+---++ Parameters
+   * :entity-type can be cluster, feed or process.
+   * :entity-name is name of the entity.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Status of the entity.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/entities/status/process/SampleProcess?doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "requestId": "default\/4d35b382-852a-4bc7-9972-b9db3493322a\n",
+    "message": "default\/SUBMITTED\n",
+    "status": "SUCCEEDED"
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySubmit.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySubmit.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySubmit.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySubmit.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,105 @@
+---++ POST  api/entities/submit/:entity-type
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Submit the given entity.
+
+---++ Parameters
+   * :entity-type can be cluster, feed or process.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Result of the submission.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+POST http://localhost:15000/api/entities/submit/feed
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- Hourly sample input data -->
+
+<feed description="sample input data"
+      name="SampleInput" xmlns="uri:falcon:feed:0.1"
+      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
+    <groups>group</groups>
+    <frequency>hours(1)</frequency>
+    <late-arrival cut-off="hours(6)" />
+    <clusters>
+        <cluster name="primary-cluster" type="source">
+            <!--validity start="2009-01-01T00:00Z" end="2099-12-31T00:00Z" timezone="UTC" /-->
+            <validity start="2009-01-01T00:00Z" end="2099-12-31T00:00Z" />
+            <retention limit="months(24)" action="delete" />
+        </cluster>
+    </clusters>
+
+    <locations>
+        <location type="data" path="/projects/bootcamp/data/${YEAR}-${MONTH}-${DAY}-${HOUR}/SampleInput" />
+        <location type="stats" path="/projects/bootcamp/stats/SampleInput" />
+        <location type="meta" path="/projects/bootcamp/meta/SampleInput" />
+    </locations>
+
+    <ACL owner="suser" group="users" permission="0755" />
+
+    <schema location="/none" provider="none" />
+</feed>
+</verbatim>
+
+---+++ Result
+<verbatim>
+{
+    "requestId": "default\/d72a41f7-6420-487b-8199-62d66e492e35\n",
+    "message": "default\/Submit successful (feed) SampleInput\n",
+    "status": "SUCCEEDED"
+}
+</verbatim>
+
+---+++ Rest Call
+<verbatim>
+POST http://localhost:15000/api/entities/submit/process?doAs=joe
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- Daily sample process. Runs at 6th hour every day. Input - last day's hourly data. Generates output for yesterday -->
+<process xmlns="uri:falcon:process:0.1" name="SampleProcess" >
+    <clusters>
+      <cluster name="primary-cluster">
+	<validity start="2012-04-03T06:00Z" end="2022-12-30T00:00Z" />
+      </cluster>
+    </clusters>
+
+    <parallel>1</parallel>
+    <order>FIFO</order>
+    <frequency>hours(1)</frequency>
+
+    <inputs>
+        <input name="input" feed="SampleInput" start="yesterday(0,0)" end="today(-1,0)" />
+    </inputs>
+
+    <outputs>
+        <output name="output" feed="SampleOutput" instance="yesterday(0,0)" />
+    </outputs>
+
+    <properties>
+        <property name="queueName" value="default" />
+        <property name="ssh.host" value="localhost" />
+        <property name="fileTimestamp" value="${coord:formatTime(coord:nominalTime(), 'yyyy-MM-dd')}" />
+    </properties>
+
+    <workflow engine="oozie" path="/examples/apps/aggregator" />
+    <retry policy="exp-backoff" delay="minutes(5)" attempts="3" />
+    
+    <late-process policy="exp-backoff" delay="hours(1)">
+        <late-input input="input" workflow-path="/projects/bootcamp/workflow/lateinput" />
+    </late-process>
+</process>
+</verbatim>
+
+---+++ Result
+<verbatim>
+{
+    "requestId": "default\/e5cc8230-f356-4566-9b65-536abdff8aa3\n",
+    "message": "default\/Submit successful (process) SampleProcess\n",
+    "status": "SUCCEEDED"
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySubmitAndSchedule.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySubmitAndSchedule.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySubmitAndSchedule.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySubmitAndSchedule.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,64 @@
+---++  POST /api/entities/submitAndSchedule/:entity-type
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Submits and schedules an entity.
+
+---++ Parameters
+   * :entity-type can either be a feed or a process.
+   * skipDryRun : Optional query param, Falcon skips oozie dryrun when value is set to true.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Result of the submit and schedule command.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+POST http://localhost:15000/api/entities/submitAndSchedule/process?skipDryRun=false&doAs=joe
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- Daily sample process. Runs at 6th hour every day. Input - last day's hourly data. Generates output for yesterday -->
+<process xmlns="uri:falcon:process:0.1" name="SampleProcess" >
+    <clusters>
+      <cluster name="primary-cluster">
+	<validity start="2012-04-03T06:00Z" end="2022-12-30T00:00Z" />
+      </cluster>
+    </clusters>
+
+    <parallel>1</parallel>
+    <order>FIFO</order>
+    <frequency>hours(1)</frequency>
+
+    <inputs>
+        <input name="input" feed="SampleInput" start="yesterday(0,0)" end="today(-1,0)" />
+    </inputs>
+
+    <outputs>
+        <output name="output" feed="SampleOutput" instance="yesterday(0,0)" />
+    </outputs>
+
+    <properties>
+        <property name="queueName" value="default" />
+        <property name="ssh.host" value="localhost" />
+        <property name="fileTimestamp" value="${coord:formatTime(coord:nominalTime(), 'yyyy-MM-dd')}" />
+    </properties>
+
+    <workflow engine="oozie" path="/examples/apps/aggregator" />
+    <retry policy="exp-backoff" delay="minutes(5)" attempts="3" />
+    
+    <late-process policy="exp-backoff" delay="hours(1)">
+        <late-input input="input" workflow-path="/projects/bootcamp/workflow/lateinput" />
+    </late-process>
+</process>
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "requestId": "schedule\/default\/b5b40931-175b-4b15-8f2b-02ef2e66f06b\n\nsubmit\/default\/b5b40931-175b-4b15-8f2b-02ef2e66f06b\n\n",
+    "message": "schedule\/default\/SampleProcess(process) scheduled successfully\n\nsubmit\/default\/Submit successful (process) SampleProcess\n\n",
+    "status": "SUCCEEDED"
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySummary.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySummary.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySummary.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySummary.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,74 @@
+---++  GET /api/entities/summary/:entity-type
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Given an EntityType and cluster, get list of entities along with summary of N recent instances of each entity
+
+---++ Parameters
+   * :entity-type Valid options are feed or process.
+   * cluster Show entities that belong to this cluster.
+   * start <optional param> Show entity summaries from this date. Date format is yyyy-MM-dd'T'HH:mm'Z'.
+      * By default, it is set to (end - 2 days).
+   * end <optional param> Show entity summary up to this date. Date format is yyyy-MM-dd'T'HH:mm'Z'.
+      * Default is set to now.
+   * fields <optional param> Fields of entity that the user wants to view, separated by commas.
+      * Valid options are STATUS, TAGS, PIPELINES.
+   * filterBy <optional param> Filter results by list of field:value pairs. Example: filterBy=STATUS:RUNNING,PIPELINES:clickLogs
+      * Supported filter fields are NAME, STATUS, PIPELINES, CLUSTER.
+      * Query will do an AND among filterBy fields.
+   * tags <optional param> Return list of entities that have specified tags, separated by a comma. Query will do AND on tag values.
+      * Example: tags=consumer=consumer@xyz.com,owner=producer@xyz.com
+   * orderBy <optional param> Field by which results should be ordered.
+      * Supports ordering by "name".
+   * sortOrder <optional param> Valid options are "asc" and "desc"
+   * offset <optional param> Show results from the offset, used for pagination. Defaults to 0.
+   * numResults <optional param> Number of results to show per request, used for pagination. Only integers > 0 are valid, Default is 10.
+   * numInstances <optional param> Number of recent instances to show per entity. Only integers > 0 are valid, Default is 7.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Show entities along with summary of N instances for each entity.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/entities/summary/feed?cluster=primary-cluster&filterBy=STATUS:RUNNING&fields=status&tags=consumer=consumer@xyz.com&orderBy=name&offset=0&numResults=1&numInstances=2&doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "entitySummary": [
+        {
+            "name"  : "SampleOutput",
+            "type"  : "feed",
+            "status": "RUNNING",
+            "instances": [
+            {
+                "details": "",
+                "endTime": "2013-10-21T14:40:26-07:00",
+                "startTime": "2013-10-21T14:39:56-07:00",
+                "cluster": "primary-cluster",
+                "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933395-oozie-rgau-W",
+                "status": "RUNNING",
+                "instance": "2012-04-03T07:00Z"
+            },
+            {
+                "details": "",
+                "endTime": "2013-10-21T14:42:27-07:00",
+                "startTime": "2013-10-21T14:41:57-07:00",
+                "cluster": "primary-cluster",
+                "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933397-oozie-rgau-W",
+                "status": "RUNNING",
+                "instance": "2012-04-03T08:00Z"
+            },
+            ]
+        }
+    ]
+    "requestId": "default\/e15bb378-d09f-4911-9df2-5334a45153d2\n",
+    "message": "default\/STATUS\n",
+    "status": "SUCCEEDED"
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySuspend.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySuspend.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySuspend.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/EntitySuspend.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,30 @@
+---++  POST /api/entities/suspend/:entity-type/:entity-name
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Suspend an entity.
+
+---++ Parameters
+   * :entity-type can either be a feed or a process.
+   * :entity-name is name of the entity.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Status of the entity.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+POST http://localhost:15000/api/entities/suspend/process/SampleProcess?doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "requestId": "default\/fe5f2b6c-1f2e-49fc-af3a-342079f0b46b\n",
+    "message": "default\/SampleProcess(process) suspended successfully\n",
+    "status": "SUCCEEDED"
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityTouch.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityTouch.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityTouch.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityTouch.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,31 @@
+---++ POST  api/entities/touch/:entity-type/:entity-name
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Force updates the entity.
+
+---++ Parameters
+   * :entity-type can be feed or process.
+   * :entity-name is name of the feed or process.
+   * skipDryRun : Optional query param, Falcon skips oozie dryrun when value is set to true.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Result of the validation.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+POST http://localhost:15000/api/entities/touch/process/SampleProcess?skipDryRun=true&doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "requestId": "touch\/default\/d6aaa328-6836-4818-a212-515bb43d8b86\n\n",
+    "message": "touch\/default\/SampleProcess updated successfully\n\n",
+    "status": "SUCCEEDED"
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityUpdate.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityUpdate.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityUpdate.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityUpdate.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,66 @@
+---++ POST  api/entities/update/:entity-type/:entity-name
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Updates the submitted entity.
+
+---++ Parameters
+   * :entity-type can be feed or process.
+   * :entity-name is name of the feed or process.
+   * skipDryRun : Optional query param, Falcon skips oozie dryrun when value is set to true.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Result of the validation.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+POST http://localhost:15000/api/entities/update/process/SampleProcess?skipDryRun=false&doAs=joe
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- Daily sample process. Runs at 6th hour every day. Input - last day's hourly data. Generates output for yesterday -->
+<process xmlns="uri:falcon:process:0.1" name="SampleProcess" >
+    <clusters>
+      <cluster name="primary-cluster">
+	<validity start="2012-04-03T06:00Z" end="2022-12-30T00:00Z" />
+      </cluster>
+    </clusters>
+
+    <parallel>1</parallel>
+    <order>FIFO</order>
+    <frequency>hours(1)</frequency>
+
+    <inputs>
+        <input name="input" feed="SampleInput" start="yesterday(0,0)" end="today(-1,0)" />
+    </inputs>
+
+    <outputs>
+        <output name="output" feed="SampleOutput" instance="yesterday(0,0)" />
+    </outputs>
+
+    <properties>
+        <property name="queueName" value="default" />
+        <property name="ssh.host" value="localhost" />
+        <property name="fileTimestamp" value="${coord:formatTime(coord:nominalTime(), 'yyyy-MM-dd')}" />
+    </properties>
+
+    <workflow engine="oozie" path="/examples/apps/aggregator" />
+    <retry policy="exp-backoff" delay="minutes(5)" attempts="3" />
+    
+    <late-process policy="exp-backoff" delay="hours(1)">
+        <late-input input="input" workflow-path="/projects/bootcamp/workflow/lateinput" />
+    </late-process>
+</process>
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "requestId": "update\/default\/d6aaa328-6836-4818-a212-515bb43d8b86\n\n",
+    "message": "update\/default\/SampleProcess updated successfully\n\n",
+    "status": "SUCCEEDED"
+}
+</verbatim>
+

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityValidate.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityValidate.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityValidate.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/EntityValidate.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,170 @@
+---++ POST  api/entities/validate/entity-type
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Validates the submitted entity.
+
+---++ Parameters
+   * :entity-type can be cluster, feed or process.
+   * skipDryRun : Optional query param, Falcon skips oozie dryrun when value is set to true.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Result of the validation.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+POST http://localhost:15000/api/entities/validate/cluster
+<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
+<cluster xmlns="uri:falcon:cluster:0.1" name="primary-cluster" description="Primary Cluster" colo="west-coast">
+    <interfaces>
+        <interface type="readonly" endpoint="hftp://localhost:50070" version="1.1.1"/>
+        <interface type="write" endpoint="hdfs://localhost:9000" version="1.1.1"/>
+        <interface type="execute" endpoint="localhost:9001" version="1.1.1"/>
+        <interface type="workflow" endpoint="http://localhost:11000/oozie/" version="4.0.0"/>
+        <interface type="messaging" endpoint="tcp://localhost:61616?daemon=true" version="5.4.3"/>
+    </interfaces>
+    <locations>
+        <location name="staging" path="/apps/falcon/staging"/>
+        <location name="temp" path="/tmp"/>
+        <location name="working" path="/apps/falcon/working"/>
+    </locations>
+</cluster>
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "requestId": "dd3f6c3a-a6f1-4c50-97fb-3f9a3f698e10",
+    "message": "Validated successfully (CLUSTER) primary-cluster",
+    "status": "SUCCEEDED"
+}
+</verbatim>
+
+---+++ Rest Call
+<verbatim>
+POST http://localhost:15000/api/entities/validate/feed?skipDryRun=true
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- Hourly sample input data -->
+
+<feed description="sample input data"
+      name="SampleInput" xmlns="uri:falcon:feed:0.1"
+      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
+    <groups>group</groups>
+    <frequency>hours(1)</frequency>
+    <late-arrival cut-off="hours(6)" />
+    <clusters>
+        <cluster name="primary-cluster" type="source">
+            <!--validity start="2009-01-01T00:00Z" end="2099-12-31T00:00Z" timezone="UTC" /-->
+            <validity start="2009-01-01T00:00Z" end="2099-12-31T00:00Z" />
+            <retention limit="months(24)" action="delete" />
+        </cluster>
+    </clusters>
+
+    <locations>
+        <location type="data" path="/projects/bootcamp/data/${YEAR}-${MONTH}-${DAY}-${HOUR}/SampleInput" />
+        <location type="stats" path="/projects/bootcamp/stats/SampleInput" />
+        <location type="meta" path="/projects/bootcamp/meta/SampleInput" />
+    </locations>
+
+    <ACL owner="suser" group="users" permission="0755" />
+
+    <schema location="/none" provider="none" />
+</feed>
+</verbatim>
+
+---+++ Result
+<verbatim>
+{
+    "requestId": "c85b190e-e653-493a-a863-d62de9c2e3b0",
+    "message": "Validated successfully (FEED) SampleInput",
+    "status": "SUCCEEDED"
+}
+</verbatim>
+
+---+++ Rest Call
+<verbatim>
+POST http://localhost:15000/api/entities/validate/feed
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- Daily sample output data -->
+
+<feed description="sample output data" name="SampleOutput" xmlns="uri:falcon:feed:0.1"
+xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
+    <groups>group</groups>
+    <frequency>hours(1)</frequency>
+    <late-arrival cut-off="hours(6)" />
+    <clusters>
+        <cluster name="primary-cluster" type="source">
+            <!--validity start="2009-01-01T00:00Z" end="2099-12-31T00:00Z" timezone="UTC" /-->
+            <validity start="2009-01-01T00:00Z" end="2099-12-31T00:00Z" />
+            <retention limit="months(24)" action="delete" />
+        </cluster>
+    </clusters>
+    <locations>
+        <location type="data" path="/projects/bootcamp/output/${YEAR}-${MONTH}-${DAY}-${HOUR}/SampleOutput" />
+        <location type="stats" path="/projects/bootcamp/stats/SampleOutput" />
+        <location type="meta" path="/projects/bootcamp/meta/SampleOutput" />
+    </locations>
+    <ACL owner="suser" group="users" permission="0755" />
+    <schema location="/none" provider="none" />
+</feed>
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "requestId": "60781732-460e-4c6c-ba86-a75fae574b05",
+    "message": "Validated successfully (FEED) SampleOutput",
+    "status": "SUCCEEDED"
+}
+</verbatim>
+
+---+++ Rest Call
+<verbatim>
+POST http://localhost:15000/api/entities/validate/process?skipDryRun=false&doAs=joe
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- Daily sample process. Runs at 6th hour every day. Input - last day's hourly data. Generates output for yesterday -->
+<process xmlns="uri:falcon:process:0.1" name="SampleProcess" >
+    <clusters>
+      <cluster name="primary-cluster">
+	<validity start="2012-04-03T06:00Z" end="2022-12-30T00:00Z" />
+      </cluster>
+    </clusters>
+
+    <parallel>1</parallel>
+    <order>FIFO</order>
+    <frequency>hours(1)</frequency>
+
+    <inputs>
+        <input name="input" feed="SampleInput" start="yesterday(0,0)" end="today(-1,0)" />
+    </inputs>
+
+    <outputs>
+        <output name="output" feed="SampleOutput" instance="yesterday(0,0)" />
+    </outputs>
+
+    <properties>
+        <property name="queueName" value="default" />
+        <property name="ssh.host" value="localhost" />
+        <property name="fileTimestamp" value="${coord:formatTime(coord:nominalTime(), 'yyyy-MM-dd')}" />
+    </properties>
+
+    <workflow engine="oozie" path="/examples/apps/aggregator" />
+    <retry policy="exp-backoff" delay="minutes(5)" attempts="3" />
+    
+    <late-process policy="exp-backoff" delay="hours(1)">
+        <late-input input="input" workflow-path="/projects/bootcamp/workflow/lateinput" />
+    </late-process>
+</process>
+</verbatim>
+
+---+++ Result
+<verbatim>
+{
+    "requestId": "e4a965c6-c7a2-41d9-ba08-2e77f1c43f57",
+    "message": "Validated successfully (PROCESS) SampleProcess",
+    "status": "SUCCEEDED"
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/FeedInstanceListing.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/FeedInstanceListing.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/FeedInstanceListing.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/FeedInstanceListing.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,46 @@
+---++ GET /api/instance/listing/feed/:entity-name
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Get falcon feed instance availability.
+
+---++ Parameters
+   * :entity-name Name of the entity.
+   * start <optional param> Show instances from this date. Date format is yyyy-MM-dd'T'HH:mm'Z'.
+      * By default, it is set to (end - (10 * entityFrequency)).
+   * end <optional param> Show instances up to this date. Date format is yyyy-MM-dd'T'HH:mm'Z'.
+      * Default is set to now.
+   * colo <optional param> Colo on which the query should be run.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Feed instance availability status
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/instance/listing/feed/SampleFeed?colo=*&start=2012-04-03T07:00Z&doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "instances": [
+        {
+            "size": "450231212222",
+            "creationTime": "1236679827365",
+            "cluster": "primary-cluster",
+            "uri": "/data/SampleFeed/2012-04-03",
+            "status": "AVAILABLE",
+            "instance": "2012-04-03T07:00Z"
+        }
+    ],
+    "requestId": "default\/3527038e-8334-4e50-8173-76c4fa430d0b\n",
+    "message": "default\/STATUS\n",
+    "status": "SUCCEEDED"
+}
+</verbatim>
+
+

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/FeedLookup.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/FeedLookup.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/FeedLookup.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/FeedLookup.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,37 @@
+---++  GET api/entities/lookup/feed
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+
+---++ Parameters
+    * path path of the instance for which you want to determine the feed. e.g. /data/project1/2014/10/10/23/
+    Path has to be the complete path and can't be a part of it.
+    * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Returns the name of the feed along with the location type(meta/data/stats) and cluster on which the given path belongs to this feed.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/entities/lookup/feed?path=/data/project1/2014/10/10/23&doAs=joe
+</verbatim>
+---+++ Result
+{
+    "feeds":
+    [
+        {
+           "feedName": "My-Feed1",
+           "locationType": "DATA",
+           "clusterName": "My-cluster1"
+        },
+        {
+           "feedName": "My-Feed2",
+           "locationType": "DATA",
+           "clusterName": "My-cluster2"
+        }
+    ]
+}

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/FeedSLA.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/FeedSLA.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/FeedSLA.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/FeedSLA.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,56 @@
+---++ GET /api/entities/sla-alert/:entity-type
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+<verbatim>
+Since: 0.8
+</verbatim>
+This command lists all the feed instances which have missed sla and are still not available. If a feed instance missed
+sla but is now available, then it will not be reported in results. The purpose of this API is alerting and hence it
+ doesn't return feed instances which missed SLA but are available as they don't require any action.
+
+---++ Parameters
+   * :entity-type Only valid option is feed.
+   * entity-name <optional param> parameter to restrict results for a particular feed using feed's name.
+   * start <mandatory param> start of the time window for nominal instances, inclusive.
+   * end <mandatory param> end of the time window for nominal instances to be considered, default is treated as current time.
+   * colo <optional param> name of the colo
+
+
+---++ Results
+Pending feed instances which missed SLA.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/entities/sla-alert/feed?colo=*&start=2012-04-03T07:00Z
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "status":"SUCCEEDED",
+    "message":"default/Success!\n",
+    "requestId":"default/885720178@qtp-495452957-6 - f6e82e9b-d23f-466b-82df-4fb8293ce9cf\n",
+    "instances":[
+            {"cluster":"local","entityName":"out","entityType":"FEED","instanceTime":"2015-09-26T17:33:00+05:30","tags":"Missed SLA High"},
+            {"cluster":"local","entityName":"out","entityType":"FEED","instanceTime":"2015-09-26T17:29:00+05:30","tags":"Missed SLA High"},
+            {"cluster":"local","entityName":"out","entityType":"FEED","instanceTime":"2015-09-26T17:35:00+05:30","tags":"Missed SLA Low"},
+            {"cluster":"local","entityName":"out","entityType":"FEED","instanceTime":"2015-09-26T17:30:00+05:30","tags":"Missed SLA High"},
+            {"cluster":"local","entityName":"out","entityType":"FEED","instanceTime":"2015-09-26T17:34:00+05:30","tags":"Missed SLA High"},
+            {"cluster":"local","entityName":"out","entityType":"FEED","instanceTime":"2015-09-26T17:31:00+05:30","tags":"Missed SLA High"},
+            {"cluster":"local","entityName":"out","entityType":"FEED","instanceTime":"2015-09-26T17:32:00+05:30","tags":"Missed SLA High"}
+    ]
+}
+</verbatim>
+
+In case there are no pending instances which have missed sla the response will be like below:
+<verbatim>
+{
+    "status":"SUCCEEDED",
+    "message":"default/Success!\n",
+    "requestId":"default/979808239@qtp-1243851750-3 - 8c7396c0-efe2-43e9-9aea-7ae6afea5fd6\n"
+}
+</verbatim>

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/Graph.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/Graph.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/Graph.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/Graph.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,22 @@
+---++  GET api/metadata/lineage/serialize
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Dump the graph.
+
+---++ Parameters
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Serialize graph to a file configured using *.falcon.graph.serialize.path in Custom startup.properties.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/metadata/lineage/serialize?doAs=joe
+</verbatim>
+---+++ Result
+None.

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/InstanceDependencies.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/InstanceDependencies.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/InstanceDependencies.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/InstanceDependencies.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,49 @@
+---++ GET /api/instance/dependencies/:entity-type/:entity-name
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Get dependent instances for a particular instance.
+
+---++ Parameters
+   * :entity-type Valid options are feed or process.
+   * :entity-name Name of the entity
+   * instanceTime <mandatory param> time of the given instance
+   * colo <optional param> name of the colo
+
+
+---++ Results
+Dependent instances for the specified instance
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+GET http://localhost:15000/api/instance/dependencies/feed/myFeed?colo=*&instanceTime=2012-04-03T07:00Z
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    'status': 'SUCCEEDED',
+    'message': 'default/Success!\n',
+    'dependencies': [
+        {
+            'cluster': 'local',
+            'entityName': 'consumer-process',
+            'entityType': 'PROCESS',
+            'instanceTime': '2014-12-18T00:00Z',
+            'tags': 'Input'
+        },
+        {
+            'cluster': 'local',
+            'entityName': 'producer-process',
+            'entityType': 'PROCESS',
+            'instanceTime': '2014-12-18T00:00Z',
+            'tags': 'Output'
+        }
+    ],
+    'requestId': 'default/1405883107@qtp-1501726962-6-0c2e690f-546b-47b0-a5ee-0365d4522a31\n'
+}
+</verbatim>
+

Added: falcon/trunk/releases/0.9/src/site/twiki/restapi/InstanceKill.twiki
URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.9/src/site/twiki/restapi/InstanceKill.twiki?rev=1730446&view=auto
==============================================================================
--- falcon/trunk/releases/0.9/src/site/twiki/restapi/InstanceKill.twiki (added)
+++ falcon/trunk/releases/0.9/src/site/twiki/restapi/InstanceKill.twiki Mon Feb 15 05:08:31 2016
@@ -0,0 +1,44 @@
+---++  POST /api/instance/kill/:entity-type/:entity-name
+   * <a href="#Description">Description</a>
+   * <a href="#Parameters">Parameters</a>
+   * <a href="#Results">Results</a>
+   * <a href="#Examples">Examples</a>
+
+---++ Description
+Kill currently running instance(s) of an entity.
+
+---++ Parameters
+   * :entity-type can either be a feed or a process.
+   * :entity-name is name of the entity.
+   * start is the start time of the instance(s) that you want to refer to
+   * end is the end time of the instance(s) that you want to refer to
+   * lifecycle <optional param> can be Eviction/Replication(default) for feed and Execution(default) for process.
+   * doAs <optional query param> allows the current user to impersonate the user passed in doAs when interacting with the Falcon system.
+
+---++ Results
+Result of the kill operation.
+
+---++ Examples
+---+++ Rest Call
+<verbatim>
+POST http://localhost:15000/api/instance/kill/process/SampleProcess?colo=*&start=2012-04-03T07:00Z&end=2014-04-03T07:00Z&doAs=joe
+</verbatim>
+---+++ Result
+<verbatim>
+{
+    "instances": [
+        {
+            "details": "",
+            "endTime": "2013-10-21T15:26:59-07:00",
+            "startTime": "2013-10-21T15:19:57-07:00",
+            "cluster": "primary-cluster",
+            "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933395-oozie-rgau-W",
+            "status": "KILLED",
+            "instance": "2012-04-03T07:00Z"
+        }
+    ],
+    "requestId": "default\/23b3cfee-ee22-40c0-825d-39c322587d5f\n",
+    "message": "default\/KILL\n",
+    "status": "SUCCEEDED"
+}
+</verbatim>




Mime
View raw message