Return-Path: X-Original-To: apmail-falcon-commits-archive@minotaur.apache.org Delivered-To: apmail-falcon-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4C82317758 for ; Wed, 9 Sep 2015 13:50:22 +0000 (UTC) Received: (qmail 96817 invoked by uid 500); 9 Sep 2015 13:50:00 -0000 Delivered-To: apmail-falcon-commits-archive@falcon.apache.org Received: (qmail 96738 invoked by uid 500); 9 Sep 2015 13:50:00 -0000 Mailing-List: contact commits-help@falcon.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@falcon.apache.org Delivered-To: mailing list commits@falcon.apache.org Received: (qmail 96275 invoked by uid 99); 9 Sep 2015 13:49:59 -0000 Received: from eris.apache.org (HELO hades.apache.org) (140.211.11.105) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 09 Sep 2015 13:49:59 +0000 Received: from hades.apache.org (localhost [127.0.0.1]) by hades.apache.org (ASF Mail Server at hades.apache.org) with ESMTP id CF0E1AC022F for ; Wed, 9 Sep 2015 13:49:59 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1702009 [14/14] - in /falcon: site/ site/0.3-incubating/ site/0.4-incubating/ site/0.5-incubating/ site/0.6-incubating/ site/0.6.1/ site/0.7/ site/0.7/css/ site/0.7/images/ site/0.7/images/logos/ site/0.7/images/profiles/ site/0.7/img/ sit... Date: Wed, 09 Sep 2015 13:49:55 -0000 To: commits@falcon.apache.org From: ajayyadava@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20150909134959.CF0E1AC022F@hades.apache.org> Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/EntityValidate.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/EntityValidate.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/EntityValidate.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/EntityValidate.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,169 @@ +---++ POST api/entities/validate/entity-type?skipDryRun=false + * Description + * Parameters + * Results + * Examples + +---++ Description +Validates the submitted entity. + +---++ Parameters + * :entity-type can be cluster, feed or process. + * skipDryRun : Optional query param, Falcon skips oozie dryrun when value is set to true. + +---++ Results +Result of the validation. + +---++ Examples +---+++ Rest Call + +POST http://localhost:15000/api/entities/validate/cluster + + + + + + + + + + + + + + + + +---+++ Result + +{ + "requestId": "dd3f6c3a-a6f1-4c50-97fb-3f9a3f698e10", + "message": "Validated successfully (CLUSTER) primary-cluster", + "status": "SUCCEEDED" +} + + +---+++ Rest Call + +POST http://localhost:15000/api/entities/validate/feed?skipDryRun=true + + + + + group + hours(1) + + + + + + + + + + + + + + + + + + + + + +---+++ Result + +{ + "requestId": "c85b190e-e653-493a-a863-d62de9c2e3b0", + "message": "Validated successfully (FEED) SampleInput", + "status": "SUCCEEDED" +} + + +---+++ Rest Call + +POST http://localhost:15000/api/entities/validate/feed + + + + + group + hours(1) + + + + + + + + + + + + + + + + + +---+++ Result + +{ + "requestId": "60781732-460e-4c6c-ba86-a75fae574b05", + "message": "Validated successfully (FEED) SampleOutput", + "status": "SUCCEEDED" +} + + +---+++ Rest Call + +POST http://localhost:15000/api/entities/validate/process + + + + + + + + + + 1 + FIFO + hours(1) + + + + + + + + + + + + + + + + + + + + + + + + +---+++ Result + +{ + "requestId": "e4a965c6-c7a2-41d9-ba08-2e77f1c43f57", + "message": "Validated successfully (PROCESS) SampleProcess", + "status": "SUCCEEDED" +} + Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/FeedInstanceListing.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/FeedInstanceListing.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/FeedInstanceListing.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/FeedInstanceListing.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,45 @@ +---++ GET /api/instance/listing/feed/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Get falcon feed instance availability. + +---++ Parameters + * :entity-name Name of the entity. + * start Show instances from this date. Date format is yyyy-MM-dd'T'HH:mm'Z'. + * By default, it is set to (end - (10 * entityFrequency)). + * end Show instances up to this date. Date format is yyyy-MM-dd'T'HH:mm'Z'. + * Default is set to now. + * colo Colo on which the query should be run. + +---++ Results +Feed instance availability status + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/instance/listing/feed/SampleFeed?colo=*&start=2012-04-03T07:00Z + +---+++ Result + +{ + "instances": [ + { + "size": "450231212222", + "creationTime": "1236679827365", + "cluster": "primary-cluster", + "uri": "/data/SampleFeed/2012-04-03", + "status": "AVAILABLE", + "instance": "2012-04-03T07:00Z" + } + ], + "requestId": "default\/3527038e-8334-4e50-8173-76c4fa430d0b\n", + "message": "default\/STATUS\n", + "status": "SUCCEEDED" +} + + + Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/FeedLookup.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/FeedLookup.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/FeedLookup.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/FeedLookup.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,36 @@ +---++ GET api/entities/lookup/feed + * Description + * Parameters + * Results + * Examples + +---++ Description + +---++ Parameters + * path path of the instance for which you want to determine the feed. e.g. /data/project1/2014/10/10/23/ + Path has to be the complete path and can't be a part of it. + +---++ Results +Returns the name of the feed along with the location type(meta/data/stats) and cluster on which the given path belongs to this feed. + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/entities/lookup/feed?path=/data/project1/2014/10/10/23 + +---+++ Result +{ + "feeds": + [ + { + "feedName": "My-Feed1", + "locationType": "DATA", + "clusterName": "My-cluster1" + }, + { + "feedName": "My-Feed2", + "locationType": "DATA", + "clusterName": "My-cluster2" + } + ] +} \ No newline at end of file Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/Graph.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/Graph.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/Graph.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/Graph.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,22 @@ +---++ GET api/metadata/lineage/serialize + * Description + * Parameters + * Results + * Examples + +---++ Description +Dump the graph. + +---++ Parameters +None. + +---++ Results +Serialize graph to a file configured using *.falcon.graph.serialize.path in Custom startup.properties. + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/metadata/lineage/serialize + +---+++ Result +None. Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceDependency.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceDependency.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceDependency.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceDependency.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,49 @@ +---++ GET /api/instance/dependency/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Get dependent instances for a particular instance. + +---++ Parameters + * :entity-type Valid options are feed or process. + * :entity-name Name of the entity + * instanceTime time of the given instance + * colo name of the colo + + +---++ Results +Dependent instances for the specified instance + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/instance/dependency/feed/myFeed?colo=*&instanceTime=2012-04-03T07:00Z + +---+++ Result + +{ + 'status': 'SUCCEEDED', + 'message': 'default/Success!\n', + 'dependencies': [ + { + 'cluster': 'local', + 'entityName': 'consumer-process', + 'entityType': 'PROCESS', + 'instanceTime': '2014-12-18T00:00Z', + 'tags': 'Input' + }, + { + 'cluster': 'local', + 'entityName': 'producer-process', + 'entityType': 'PROCESS', + 'instanceTime': '2014-12-18T00:00Z', + 'tags': 'Output' + } + ], + 'requestId': 'default/1405883107@qtp-1501726962-6-0c2e690f-546b-47b0-a5ee-0365d4522a31\n' +} + + Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceKill.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceKill.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceKill.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceKill.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,43 @@ +---++ POST /api/instance/kill/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Kill currently running instance(s) of an entity. + +---++ Parameters + * :entity-type can either be a feed or a process. + * :entity-name is name of the entity. + * start is the start time of the instance(s) that you want to refer to + * end is the end time of the instance(s) that you want to refer to + * lifecycle can be Eviction/Replication(default) for feed and Execution(default) for process. + +---++ Results +Result of the kill operation. + +---++ Examples +---+++ Rest Call + +POST http://localhost:15000/api/instance/kill/process/SampleProcess?colo=*&start=2012-04-03T07:00Z&end=2014-04-03T07:00Z + +---+++ Result + +{ + "instances": [ + { + "details": "", + "endTime": "2013-10-21T15:26:59-07:00", + "startTime": "2013-10-21T15:19:57-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933395-oozie-rgau-W", + "status": "KILLED", + "instance": "2012-04-03T07:00Z" + } + ], + "requestId": "default\/23b3cfee-ee22-40c0-825d-39c322587d5f\n", + "message": "default\/KILL\n", + "status": "SUCCEEDED" +} + Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceList.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceList.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceList.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceList.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,89 @@ +---++ GET /api/instance/list/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Get list of all instances of a given entity. + +---++ Parameters + * :entity-type Valid options are cluster, feed or process. + * :entity-name Name of the entity. + * start Show instances from this date. Date format is yyyy-MM-dd'T'HH:mm'Z'. + * By default, it is set to (end - (10 * entityFrequency)). + * end Show instances up to this date. Date format is yyyy-MM-dd'T'HH:mm'Z'. + * Default is set to now. + * colo Colo on which the query should be run. + * lifecycle Valid lifecycles for feed are Eviction/Replication(default) and for process is Execution(default). + * filterBy Filter results by list of field:value pairs. Example: filterBy=STATUS:RUNNING,CLUSTER:primary-cluster + * Supported filter fields are STATUS, CLUSTER, SOURCECLUSTER, STARTEDAFTER. + * Query will do an AND among filterBy fields. + * orderBy Field by which results should be ordered. + * Supports ordering by "status","startTime","endTime","cluster". + * sortOrder Valid options are "asc" and "desc" + * offset Show results from the offset, used for pagination. Defaults to 0. + * numResults Number of results to show per request, used for pagination. Only integers > 0 are valid, Default is 10. + +---++ Results +List of instances of given entity. + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/instance/list/process/SampleProcess?colo=*&start=2012-04-03T07:00Z + +---+++ Result + +{ + "instances": [ + { + "details": "", + "endTime": "2013-10-21T14:40:26-07:00", + "startTime": "2013-10-21T14:39:56-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933395-oozie-rgau-W", + "status": "SUCCEEDED", + "instance": "2012-04-03T07:00Z" + } + ], + "requestId": "default\/e15bb378-d09f-4911-9df2-5334a45153d2\n", + "message": "default\/STATUS\n", + "status": "SUCCEEDED" +} + + + +---+++ Rest Call + +GET http://localhost:15000/api/instance/list/process/SampleProcess?colo=*&start=2012-04-03T07:00Z&filterBy=STATUS:SUCCEEDED,CLUSTER:primary-cluster&orderBy=startTime&offset=2&numResults=2 + +---+++ Result + +{ + "instances": [ + { + "details": "", + "endTime": "2013-10-21T14:40:26-07:00", + "startTime": "2013-10-21T14:39:56-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933395-oozie-rgau-W", + "status": "SUCCEEDED", + "instance": "2012-04-03T07:00Z" + }, + { + "details": "", + "endTime": "2013-10-21T14:42:26-07:00", + "startTime": "2013-10-21T14:41:56-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933397-oozie-rgau-W", + "status": "SUCCEEDED", + "instance": "2012-04-03T08:00Z" + }, + ], + + "requestId": "default\/e15bb378-d09f-4911-9df2-5334a45153d2\n", + "message": "default\/STATUS\n", + "status": "SUCCEEDED" +} + Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceLogs.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceLogs.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceLogs.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceLogs.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,112 @@ +---++ GET /api/instance/logs/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Get log of a specific instance of an entity. + +---++ Parameters + * :entity-type Valid options are cluster, feed or process. + * :entity-name Name of the entity. + * start Show instances from this date. Date format is yyyy-MM-dd'T'HH:mm'Z'. + * By default, it is set to (end - (10 * entityFrequency)). + * end Show instances up to this date. Date format is yyyy-MM-dd'T'HH:mm'Z'. + * Default is set to now. + * colo Colo on which the query should be run. + * runId Run Id. + * lifecycle Valid lifecycles for feed are Eviction/Replication(default) and for process is Execution(default). + * filterBy Filter results by list of field:value pairs. Example: filterBy=STATUS:RUNNING,CLUSTER:primary-cluster + * Supported filter fields are STATUS, CLUSTER, SOURCECLUSTER, STARTEDAFTER. + * Query will do an AND among filterBy fields. + * orderBy Field by which results should be ordered. + * Supports ordering by "status","startTime","endTime","cluster". + * sortOrder Valid options are "asc" and "desc" + * offset Show results from the offset, used for pagination. Defaults to 0. + * numResults Number of results to show per request, used for pagination. Only integers > 0 are valid, Default is 10. + +---++ Results +Log of specified instance. + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/instance/logs/process/SampleProcess?colo=*&start=2012-04-03T07:00Z + +---+++ Result + +{ + "instances": [ + { + "actions": [ + { + "logFile": "http:\/\/localhost:50070\/data\/apps\/falcon\/staging\/falcon\/workflows\/process\/SampleProcess\/logs\/job-2012-04-03-07-00\/000\/pig_SUCCEEDED.log", + "status": "SUCCEEDED", + "action": "pig" + } + ], + "details": "", + "endTime": "2013-10-21T14:40:26-07:00", + "startTime": "2013-10-21T14:39:56-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:50070\/data\/apps\/falcon\/staging\/falcon\/workflows\/process\/SampleProcess\/logs\/job-2012-04-03-07-00\/000\/oozie.log", + "status": "SUCCEEDED", + "instance": "2012-04-03T07:00Z" + } + ], + "requestId": "default\/3527038e-8334-4e50-8173-76c4fa430d0b\n", + "message": "default\/STATUS\n", + "status": "SUCCEEDED" +} + + +---+++ Rest Call + +GET http://localhost:15000/api/instance/logs/process/SampleProcess?colo=*&start=2012-04-03T07:00Z&filterBy=STATUS:SUCCEEDED,CLUSTER:primary-cluster&orderBy=startTime&offset=2&numResults=2 + +---+++ Result + +{ + "instances": [ + { + "actions": [ + { + "logFile": "http:\/\/localhost:50070\/data\/apps\/falcon\/staging\/falcon\/workflows\/process\/SampleProcess\/logs\/job-2012-04-03-07-00\/000\/pig_SUCCEEDED.log", + "status": "SUCCEEDED", + "action": "pig" + } + ], + "details": "", + "endTime": "2013-10-21T14:40:26-07:00", + "startTime": "2013-10-21T14:39:56-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:50070\/data\/apps\/falcon\/staging\/falcon\/workflows\/process\/SampleProcess\/logs\/job-2012-04-03-07-00\/000\/oozie.log", + "status": "SUCCEEDED", + "instance": "2012-04-03T07:00Z" + }, + { + "actions": [ + { + "logFile": "http:\/\/localhost:50070\/data\/apps\/falcon\/staging\/falcon\/workflows\/process\/SampleProcess\/logs\/job-2012-04-03-07-00\/001\/pig_SUCCEEDED.log", + "status": "SUCCEEDED", + "action": "pig" + } + ], + "details": "", + "endTime": "2013-10-21T14:42:27-07:00", + "startTime": "2013-10-21T14:41:57-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:50070\/data\/apps\/falcon\/staging\/falcon\/workflows\/process\/SampleProcess\/logs\/job-2012-04-03-07-00\/001\/oozie.log", + "status": "SUCCEEDED", + "instance": "2012-04-03T08:00Z" + } + ], + "requestId": "default\/3527038e-8334-4e50-8173-76c4fa430d0b\n", + "message": "default\/STATUS\n", + "status": "SUCCEEDED" +} + + + + Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceParams.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceParams.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceParams.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceParams.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,82 @@ +---++ GET /api/instance/params/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Get the params passed to the workflow for an instance of feed/process. + +---++ Parameters + * :entity-type Valid options are cluster, feed or process. + * :entity-name Name of the entity. + * start should be the nominal time of the instance for which you want the params to be returned + * colo Colo on which the query should be run. + * lifecycle Valid lifecycles for feed are Eviction/Replication(default) and for process is Execution(default). + + +---++ Results +List of instances currently running. + +---++ Examples +---+++ Rest Call + +http://userqa.user.com:16000/api/instance/params/process/Sample-Process?start=2014-10-01T11:00Z&colo=* + +---+++ Result +{ + "status": "SUCCEEDED", + "message": "ua1/PARAMS\n", + "requestId": "ua1/807e9fe6-ba60-490e-b720-f8dc8b92063e\n", + "instances": [ + { + "instance": "2014-10-01T11:00Z", + "status": "RUNNING", + "logFile": "http://spyke.user.com:11000/oozie?job=0000211-141117203201940-oozie-oozi-W", + "cluster": "sample-cluster", + "startTime": "2014-11-19T19:46:29+08:00", + "details": "", + "actions": [ + { + "action": "succeeded-post-processing", + "status": "RUNNING", + "logFile": "http://spyke.user.com:50030/jobdetails.jsp?jobid=job_201411071450_1052" + } + ], + "params": { + "entry": {"key": "jobTracker", "value": "10.16.114.113:8021"}, + "entry":{"key":"falconInputNames","value":"IGNORE"}, + "entry":{"key":"shouldRecord","value":"false"}, + "entry":{"key":"timeStamp","value":"2014-11-19-11-46"}, + "entry":{"key":"falconInPaths","value":"IGNORE"}, + "entry":{"key":"broker.url","value":"tcp://localhost:61616"}, + "entry":{"key":"feedNames","value":"NONE"}, + "entry":{"key":"falcon.libpath","value":"/path/falcon/sample/lib"}, + "entry":{"key":"ENTITY_PATH","value":"/path/falcon/staging/falcon/workflows/process/Sample-Process/9506be19980e0e6fdb709e1baffff_1416397585511/DEFAULT"}, + "entry":{"key":"entityType","value":"process"}, + "entry":{"key":"nominalTime","value":"2014-10-01-11-00"}, + "entry":{"key":"feedInstancePaths","value":"IGNORE"}, + "entry":{"key":"oozie.bundle.application.path","value":"hdfs://10.16.104.13:8020/path/falcon/staging/falcon/workflows/process/Sample-Process/9506be19980e0e669709e1baffff_1416397585511"}, + "entry":{"key":"logDir","value":"hdfs://10.16.104.13:8020/path/falcon/staging/falcon/workflows/process/Sample-Process/logs"}, + "entry":{"key":"userWorkflowEngine","value":"oozie"}, + "entry":{"key":"broker.ttlInMins","value":"4320"}, + "entry":{"key":"oozie.use.system.libpath","value":"true"}, + "entry":{"key":"queueName","value":"reports"}, + "entry":{"key":"falconDataOperation","value":"GENERATE"}, + "entry":{"key":"oozie.wf.external.id","value":"Sample-Process/DEFAULT/2014-10-01T11:00Z"}, + "entry":{"key":"workflowEngineUrl","value":"http://10.11.100.10:11000/oozie/"}, + "entry":{"key":"userBrokerImplClass","value":"org.apache.activemq.ActiveMQConnectionFactory"}, + "entry":{"key":"ENTITY_NAME","value":"FALCON_PROCESS_DEFAULT_Sample-Process"}, + "entry":{"key":"broker.impl.class","value":"org.apache.activemq.ActiveMQConnectionFactory"}, + "entry":{"key":"userWorkflowName","value":"Sample-workflow"}, + "entry":{"key":"entityName","value":"Sample-Process"}, + "entry":{"key":"srcClusterName","value":"NA"}, + "entry":{"key":"userBrokerUrl","value":"tcp://localhost:61616?daemon=true"}, + "entry":{"key":"falconInputFeeds","value":"NONE"}, + "entry":{"key":"user.name","value":"sampleuser"}, + "entry":{"key":"threedaysback","value":"2014-09-28"}, + "entry":{"key":"userWorkflowVersion","value":"1.0"} + } + } + ] +} \ No newline at end of file Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceRerun.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceRerun.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceRerun.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceRerun.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,65 @@ +---++ POST /api/instance/rerun/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Rerun instances of an entity. On issuing a rerun, by default the execution resumes from the last failed node in the workflow. + +---++ Parameters + * :entity-type can either be a feed or a process. + * :entity-name is name of the entity. + * start is the start time of the instance that you want to refer to + * end is the end time of the instance that you want to refer to + * lifecycle can be Eviction/Replication(default) for feed and Execution(default) for process. + * force can be used to forcefully rerun the entire instance. + +---++ Results +Results of the rerun command. + +---++ Examples +---+++ Rest Call + +POST http://localhost:15000/api/instance/rerun/process/SampleProcess?colo=*&start=2013-04-03T07:00Z&end=2014-04-03T07:00Z + +---+++ Result + +{ + "instances": [ + { + "details": "", + "startTime": "2013-10-21T15:10:47-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933395-oozie-rgau-W", + "status": "RUNNING", + "instance": "2012-04-03T07:00Z" + } + ], + "requestId": "default\/7a3582bd-608c-45a7-9b74-1837b51ba6d5\n", + "message": "default\/RERUN\n", + "status": "SUCCEEDED" +} + + + +POST http://localhost:15000/api/instance/rerun/process/SampleProcess?colo=*&start=2013-04-03T07:00Z&end=2014-04-03T07:00Z&force=true + +---+++ Result + +{ + "instances": [ + { + "details": "", + "startTime": "2013-10-21T15:10:47-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933395-oozie-rgau-W", + "status": "RUNNING", + "instance": "2012-04-03T07:00Z" + } + ], + "requestId": "default\/7a3582bd-608c-45a7-9b74-1837b51ba6d5\n", + "message": "default\/RERUN\n", + "status": "SUCCEEDED" +} + Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceResume.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceResume.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceResume.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceResume.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,41 @@ +---++ POST /api/instance/resume/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Resume suspended instances of an entity. + +---++ Parameters + * :entity-type can either be a feed or a process. + * :entity-name is name of the entity. + * start is the start time of the instance(s) that you want to refer to + * end is the end time of the instance(s) that you want to refer to + * lifecycle can be Eviction/Replication(default) for feed and Execution(default) for process. +---++ Results +Results of the resume command. + +---++ Examples +---+++ Rest Call + +POST http://localhost:15000/api/instance/resume/process/SampleProcess?colo=*&start=2012-04-03T07:00Z&end=2014-04-03T07:00Z + +---+++ Result + +{ + "instances": [ + { + "details": "", + "startTime": "2013-10-21T15:19:57-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933395-oozie-rgau-W", + "status": "RUNNING", + "instance": "2012-04-03T07:00Z" + } + ], + "requestId": "default\/e88ff2e0-2af7-4829-a360-f92e95be2981\n", + "message": "default\/RESUME\n", + "status": "SUCCEEDED" +} + Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceRunning.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceRunning.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceRunning.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceRunning.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,85 @@ +---++ GET /api/instance/running/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Get a list of instances currently running for a given entity. + +---++ Parameters + * :entity-type Valid options are cluster, feed or process. + * :entity-name Name of the entity. + * colo Colo on which the query should be run. + * lifecycle Valid lifecycles for feed are Eviction/Replication(default) and for process is Execution(default). + * filterBy Filter results by list of field:value pairs. Example: filterBy=CLUSTER:primary-cluster + * Supported filter fields are CLUSTER, SOURCECLUSTER, STARTEDAFTER. + * Query will do an AND among filterBy fields. + * orderBy Field by which results should be ordered. + * Supports ordering by "status","startTime","endTime","cluster". + * sortOrder Valid options are "asc" and "desc" + * offset Show results from the offset, used for pagination. Defaults to 0. + * numResults Number of results to show per request, used for pagination. Only integers > 0 are valid, Default is 10. + + +---++ Results +List of instances currently running. + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/instance/running/process/SampleProcess?colo=* + +---+++ Result + +{ + "instances": [ + { + "startTime": "2013-10-21T14:39:28-07:00", + "cluster": "primary-cluster", + "status": "RUNNING", + "instance": "2012-04-03T06:00Z" + } + ], + "requestId": "default\/12e9a7d4-3b4f-4a76-b471-c8f3786a62a0\n", + "message": "default\/Running Instances\n", + "status": "SUCCEEDED" +} + + + +---+++ Rest Call + +GET http://localhost:15000/api/instance/running/process/SampleProcess?colo=*&start=2012-04-03T07:00Z&filterBy=CLUSTER:primary-cluster&orderBy=startTime&offset=2&numResults=2 + +---+++ Result + +{ + "instances": [ + { + "details": "", + "endTime": "2013-10-21T14:40:26-07:00", + "startTime": "2013-10-21T14:39:56-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933395-oozie-rgau-W", + "status": "RUNNING", + "instance": "2012-04-03T07:00Z" + }, + { + "details": "", + "endTime": "2013-10-21T14:42:27-07:00", + "startTime": "2013-10-21T14:41:57-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933397-oozie-rgau-W", + "status": "RUNNING", + "instance": "2012-04-03T08:00Z" + }, + ], + + "requestId": "default\/e15bb378-d09f-4911-9df2-5334a45153d2\n", + "message": "default\/STATUS\n", + "status": "SUCCEEDED" +} + + + Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceStatus.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceStatus.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceStatus.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceStatus.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,97 @@ +---++ GET /api/instance/status/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Get status of a specific instance of an entity. + +---++ Parameters + * :entity-type Valid options are cluster, feed or process. + * :entity-name Name of the entity. + * start Show instances from this date. Date format is yyyy-MM-dd'T'HH:mm'Z'. + * By default, it is set to (end - (10 * entityFrequency)). + * end Show instances up to this date. Date format is yyyy-MM-dd'T'HH:mm'Z'. + * Default is set to now. + * colo Colo on which the query should be run. + * lifecycle Valid lifecycles for feed are Eviction/Replication(default) and for process is Execution(default). + * filterBy Filter results by list of field:value pairs. Example: filterBy=STATUS:RUNNING,CLUSTER:primary-cluster + * Supported filter fields are STATUS, CLUSTER, SOURCECLUSTER, STARTEDAFTER. + * Query will do an AND among filterBy fields. + * orderBy Field by which results should be ordered. + * Supports ordering by "status","startTime","endTime","cluster". + * sortOrder Valid options are "asc" and "desc" + * offset Show results from the offset, used for pagination. Defaults to 0. + * numResults Number of results to show per request, used for pagination. Only integers > 0 are valid, Default is 10. + + +---++ Results +Status of the specified instance along with job urls for all actions of user workflow and non-succeeded actions of the main-workflow. + +---++ Examples +---+++ Rest Call + +GET https://localhost:15443/api/instance/status/process/WordCount?start=2014-11-04T16:00Z&colo=* + +---+++ Result + +{ + "instances": [ + { + "details": "", + "endTime": "2014-11-05T16:08:10+05:30", + "startTime": "2014-11-05T16:07:29+05:30", + "cluster": "local", + "logFile": "http:\/\/localhost:11000\/oozie?job=0000011-141105155430303-oozie-oozi-W", + "status": "SUCCEEDED", + "instance": "2014-11-04T16:00Z", + "actions": [ + { + "action": "wordcount-mr", + "status": "SUCCEEDED", + "logFile": "http:\/\/localhost:50030\/jobdetails.jsp?jobid=job_201411051553_0005" + } + ] + } + ], + "requestId": "default\/b9fc3cba-1b46-4d1f-8196-52c795ea3580\n", + "message": "default\/STATUS\n", + "status": "SUCCEEDED" +} + + + +---+++ Rest Call + +GET http://localhost:15000/api/instance/status/process/SampleProcess?colo=*&start=2012-04-03T07:00Z&filterBy=STATUS:SUCCEEDED,CLUSTER:primary-cluster&orderBy=startTime&offset=2&numResults=2 + +---+++ Result + +{ + "instances": [ + { + "details": "", + "endTime": "2013-10-21T14:40:26-07:00", + "startTime": "2013-10-21T14:39:56-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933395-oozie-rgau-W", + "status": "SUCCEEDED", + "instance": "2012-04-03T07:00Z" + }, + { + "details": "", + "endTime": "2013-10-21T14:42:26-07:00", + "startTime": "2013-10-21T14:41:56-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933397-oozie-rgau-W", + "status": "SUCCEEDED", + "instance": "2012-04-03T08:00Z" + }, + ], + + "requestId": "default\/e15bb378-d09f-4911-9df2-5334a45153d2\n", + "message": "default\/STATUS\n", + "status": "SUCCEEDED" +} + Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceSummary.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceSummary.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceSummary.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceSummary.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,113 @@ +---++ GET /api/instance/summary/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Get summary of instance/instances of an entity. + +---++ Parameters + * :entity-type Valid options are feed or process. + * :entity-name Name of the entity. + * start Show instances from this date. Date format is yyyy-MM-dd'T'HH:mm'Z'. + * By default, it is set to (end - (10 * entityFrequency)). + * end Show instances up to this date. Date format is yyyy-MM-dd'T'HH:mm'Z'. + * Default is set to now. + * colo Colo on which the query should be run. + * lifecycle Valid lifecycles for feed are Eviction/Replication(default) and for process is Execution(default). + * filterBy Filter results by list of field:value pairs. + Example1: filterBy=STATUS:RUNNING,CLUSTER:primary-cluster + Example2: filterBy=Status:RUNNING,Status:KILLED + * Supported filter fields are STATUS, CLUSTER + * Query will do an AND among filterBy fields. + * orderBy Field by which results should be ordered. + * Supports ordering by "cluster". + * sortOrder Valid options are "asc" and "desc" + Example: orderBy=cluster sortOrder=asc + +---++ Results +Summary of the instances over the specified time range + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/instance/summary/process/WordCount?colo=*&start=2014-01-21T13:00Z&end=2014-01-21T16:00Z + +---+++ Result + +{ + "status":"SUCCEEDED", + "message":"default/SUMMARY\n", + "requestId":"default/c344567b-da73-44d5-bcd4-bf456524934c\n", + "instancesSummary": + { + "cluster":"local", + "map": + { + "entry": + { + "key":"SUCCEEDED", + "value":"value" + } + } + } + } +} + + +---+++ Rest Call + +GET https://localhost:16443/api/instance/summary/process/WordCount?filterBy=Status:KILLED,Status:RUNNING&start=2015-06-24T16:00Z&end=2015-06-24T23:00Z&colo=* + +---+++ Result + +{ + "status":"SUCCEEDED", + "message":"local/SUMMARY\n", + "requestId":"local/1246061948@qtp-1059149611-5 - 34d8c3bb-f461-4fd5-87cd-402c9c6b1ed2\n", + "instancesSummary":[ + { + "cluster":"local", + "map":{ + "entry":{ + "key":"RUNNING", + "value":"1" + }, + "entry":{ + "key":"KILLED", + "value":"1" + } + } + } + ] +} + + +---+++ Rest Call + +GET https://localhost:16443/api/instance/summary/process/WordCount?orderBy=cluster&sortOrder=asc&start=2015-06-24T16:00Z&end=2015-06-24T23:00Z&colo=* + +---+++ Result + +{ + "status":"SUCCEEDED", + "message":"local/SUMMARY\n", + "requestId":"local/1246061948@qtp-1059149611-5 - 42e2040d-6b6e-4bfd-a090-83db5ed1a429\n", + "instancesSummary":[ + { + "cluster":"local", + "map":{ + "entry":{ + "key":"SUCCEEDED", + "value":"6" + }, + "entry":{ + "key":"KILLED", + "value":"1" + } + } + } + ] +} + Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceSuspend.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceSuspend.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceSuspend.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/InstanceSuspend.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,43 @@ +---++ POST /api/instance/suspend/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Suspend instances of an entity. + +---++ Parameters + * :entity-type can either be a feed or a process. + * :entity-name is name of the entity. + * start is the start time of the instance(s) that you want to refer to + * end is the end time of the instance(s) that you want to refer to + * lifecycle can be Eviction/Replication(default) for feed and Execution(default) for process. + +---++ Results +Results of the suspend command. + +---++ Examples +---+++ Rest Call + +POST http://localhost:15000/api/instance/suspend/process/SampleProcess?colo=*&start=2012-04-03T07:00Z&end=2014-04-03T07:00Z + +---+++ Result + +{ + "instances": [ + { + "details": "", + "endTime": "2013-10-21T15:15:01-07:00", + "startTime": "2013-10-21T15:14:32-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933395-oozie-rgau-W", + "status": "SUCCEEDED", + "instance": "2012-04-03T07:00Z" + } + ], + "requestId": "default\/ff07e45b-b6da-4f47-ae96-9182bd8a7e53\n", + "message": "default\/SUSPEND\n", + "status": "SUCCEEDED" +} + Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/MetadataList.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/MetadataList.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/MetadataList.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/MetadataList.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,30 @@ +---++ GET api/metadata/discovery/:type/list + * Description + * Parameters + * Results + * Examples + +---++ Description +Get all dimensions of specified type. + +---++ Parameters + * :type Valid dimension types are cluster_entity,feed_entity, process_entity, user, colo, tags, groups, pipelines + * cluster Show dimensions related to this cluster. + + +---++ Results +List of dimensions that match requested type [and cluster]. + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/metadata/discovery/process_entity/list?cluster=primary-cluster + +---+++ Result + +{ + "results": ["sampleIngestProcess","testProcess","anotherProcess"], + "totalSize": 3 +} + + Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/MetadataRelations.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/MetadataRelations.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/MetadataRelations.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/MetadataRelations.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,45 @@ +---++ GET api/metadata/discovery/:dimension-type/:dimension-name/relations + * Description + * Parameters + * Results + * Examples + +---++ Description +Get all relations of a specific dimension. + +---++ Parameters + * :type Valid dimension types are cluster_entity,feed_entity, process_entity, user, colo, tags, groups, pipelines + * :name Name of the dimension. + +---++ Results +Get all relations of a specific dimension. + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/metadata/discovery/process_entity/sample-process/relations + +---+++ Result + +{ + "timestamp":"2014-09-09T01:31Z", + "userWorkflowEngine":"pig", + "name":"sample-process", + "type":"PROCESS_ENTITY", + "userWorkflowName":"imp-click-join-workflow", + "version":"1.0.9", + "inVertices":[ + {"name":"clicks-feed","type":"FEED_ENTITY","label":"input"}, + {"name":"impression-feed","type":"FEED_ENTITY","label":"input"}, + {"name":"sample-process\/2014-01-01T01:00Z","type":"PROCESS_INSTANCE","label":"instance-of"} + ], + "outVertices":[ + {"name":"Critical","type":"TAGS","label":"classified-as"}, + {"name":"testPipeline","type":"PIPELINES","label":"pipeline"}, + {"name":"primary-cluster","type":"CLUSTER_ENTITY","label":"runs-on"}, + {"name":"imp-click-join2","type":"FEED_ENTITY","label":"output"}, + {"name":"imp-click-join1","type":"FEED_ENTITY","label":"output"}, + {"name":"falcon-user","type":"USER","label":"owned-by"} + ] +} + Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/ResourceList.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/ResourceList.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/ResourceList.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/ResourceList.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,90 @@ +---+ RESTful Resources + +---++ Resource List + * REST Call on Entity Resource + * REST Call on Feed/Process Instances + * REST Call on Admin Resource + * REST Call on Lineage Graph Resource + * REST Call on Metadata Resource + +---++ Authentication + +When security is off (Pseudo/Simple), the authenticated user is the username specified in the user.name query +parameter. If the user.name parameter is not set, the server may either set the authenticated user to a default web +user, if there is any, or return an error response. + +When security is on (kerberos), authentication is performed by Kerberos SPNEGO. + +Below are examples using the curl command tool. + +Authentication when security is off (Pseudo/Simple): + +curl -i "http://:/?[user.name=&]=..." + + +Authentication using Kerberos SPNEGO when security is on: + +curl -i --negotiate -u : "http://:/?=..." + + +See also: [[../Security.twiki][Security in Falcon]] + +---++ REST Call on Admin Resource + +| *Call Type* | *Resource* | *Description* | +| GET | [[AdminStack][api/admin/stack]] | Get stack of the server | +| GET | [[AdminVersion][api/admin/version]] | Get version of the server | +| GET | [[AdminConfig][api/admin/config/:config-type]] | Get configuration information of the server | + +---++ REST Call on Entity Resource + +| *Call Type* | *Resource* | *Description* | +| POST | [[EntityValidate][api/entities/validate/:entity-type]] | Validate the entity | +| POST | [[EntitySubmit][api/entities/submit/:entity-type]] | Submit the entity | +| POST | [[EntityUpdate][api/entities/update/:entity-type/:entity-name]] | Update the entity | +| POST | [[EntitySubmitAndSchedule][api/entities/submitAndSchedule/:entity-type]] | Submit & Schedule the entity | +| POST | [[EntitySchedule][api/entities/schedule/:entity-type/:entity-name]] | Schedule the entity | +| POST | [[EntitySuspend][api/entities/suspend/:entity-type/:entity-name]] | Suspend the entity | +| POST | [[EntityResume][api/entities/resume/:entity-type/:entity-name]] | Resume the entity | +| DELETE | [[EntityDelete][api/entities/delete/:entity-type/:entity-name]] | Delete the entity | +| GET | [[EntityStatus][api/entities/status/:entity-type/:entity-name]] | Get the status of the entity | +| GET | [[EntityDefinition][api/entities/definition/:entity-type/:entity-name]] | Get the definition of the entity | +| GET | [[EntityList][api/entities/list/:entity-type]] | Get the list of entities | +| GET | [[EntitySummary][api/entities/summary/:entity-type/:cluster]] | Get instance summary of all entities | +| GET | [[EntityDependencies][api/entities/dependencies/:entity-type/:entity-name]] | Get the dependencies of the entity | +| GET | [[FeedLookup][api/entities/lookup/feed/]] | Get feed for given path | + +---++ REST Call on Feed and Process Instances + +| *Call Type* | *Resource* | *Description* | +| GET | [[InstanceRunning][api/instance/running/:entity-type/:entity-name]] | List of running instances. | +| GET | [[InstanceParams][api/instance/params/:entity-type/:entity-name]] | List of entity instances along with their workflow params. | +| GET | [[InstanceList][api/instance/list/:entity-type/:entity-name]] | List of instances | +| GET | [[InstanceStatus][api/instance/status/:entity-type/:entity-name]] | Status of a given instance | +| POST | [[InstanceKill][api/instance/kill/:entity-type/:entity-name]] | Kill a given instance | +| POST | [[InstanceSuspend][api/instance/suspend/:entity-type/:entity-name]] | Suspend a running instance | +| POST | [[InstanceResume][api/instance/resume/:entity-type/:entity-name]] | Resume a given instance | +| POST | [[InstanceRerun][api/instance/rerun/:entity-type/:entity-name]] | Rerun a given instance | +| GET | [[InstanceLogs][api/instance/logs/:entity-type/:entity-name]] | Get logs of a given instance | +| GET | [[Triage][api/instance/triage/:entity-type/:entity-name]] | Triage an instance to see it's stuck lineage | +| GET | [[InstanceSummary][api/instance/summary/:entity-type/:entity-name]] | Return summary of instances for an entity | +| GET | [[InstanceDependency][api/instance/dependency/:entity-type/:entity-name]] | Return dependent instances for a given instance | + +---++ REST Call on Metadata Lineage Resource + +| *Call Type* | *Resource* | *Description* | +| GET | [[Graph][api/metadata/lineage/serialize]] | dump the graph | +| GET | [[AllVertices][api/metadata/lineage/vertices/all]] | get all vertices | +| GET | [[Vertices][api/metadata/lineage/vertices?key=:key&value=:value]] | get all vertices for a key index | +| GET | [[Vertex][api/metadata/lineage/vertices/:id]] | get the vertex with the specified id | +| GET | [[VertexProperties][api/metadata/lineage/vertices/properties/:id?relationships=:true]] | get the properties of the vertex with the specified id | +| GET | [[AdjacentVertices][api/metadata/lineage/vertices/:id/:direction]] | get the adjacent vertices or edges of the vertex with the specified direction | +| GET | [[AllEdges][api/metadata/lineage/edges/all]] | get all edges | +| GET | [[Edge][api/metadata/lineage/edges/:id]] | get the edge with the specified id | +| GET | [[EntityLineage][api/metadata/lineage/entities?pipeline=:name]] | Get lineage graph for processes and feeds in the specified pipeline | + +---++ REST Call on Metadata Discovery Resource + +| *Call Type* | *Resource* | *Description* | +| GET | [[MetadataList][api/metadata/discovery/:dimension-type/list]] | list of dimensions | +| GET | [MetadataRelations][api/metadata/discovery/:dimension-type/:dimension-name/relations]] | Return all relations of a dimension | Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/Triage.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/Triage.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/Triage.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/Triage.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,44 @@ +---++ GET api/instance/triage/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Given a feed/process instance this command traces it's ancestors to find what all ancestors have failed. It's useful if +lot of instances are failing in a pipeline as it then finds out the root cause of the pipeline being stuck. + + +---++ Parameters + * :entity-type type of entity(feed/process). + * :entity-name name of the feed/process. + * :start instance time of the entity instance. + * :colo name of the colo on which you want to triage + +---++ Results +It returns a json graph + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/instance/triage/feed/my-feed?start=2015-03-02T00:00Z&colo=local + +---+++ Result + +{ + "vertices": ["(FEED) my-feed (2015-03-02T00:00Z) [Unavailable]", "(PROCESS) producer-process (2015-03-01T10:00Z) [TIMEDOUT]", "(FEED) input-feed-for-producer (2015-03-01T00:00Z) [Available]"], + "edges": + [ + { + "from" : "(PROCESS) producer-process (2015-03-01T10:00Z) [TIMEDOUT]", + "to" : "(FEED) my-feed (2015-03-02T00:00Z) [Unavailable]", + "label" : "produces" + }, + { + "from" : "(FEED) input-feed-for-producer (2015-03-01T00:00Z) [Available]", + "to" : "(PROCESS) producer-process (2015-03-01T10:00Z) [TIMEDOUT]", + "label" : "consumed by" + } + ] +} + Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/Vertex.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/Vertex.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/Vertex.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/Vertex.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,35 @@ +---++ GET api/metadata/lineage/vertices/:id + * Description + * Parameters + * Results + * Examples + +---++ Description +Gets the vertex with specified id. + +---++ Parameters + * :id is the unique id of the vertex. + +---++ Results +Vertex with the specified id. + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/metadata/lineage/vertices/4 + +---+++ Result + +{ + "results": [ + { + "timestamp":"2014-04-21T20:55Z", + "name":"sampleIngestProcess", + "type":"process-instance", + "version":"2.0.0", + "_id":4, + "_type":"vertex" + } + ] +} + Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/VertexProperties.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/VertexProperties.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/VertexProperties.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/VertexProperties.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,33 @@ +---++ GET api/metadata/lineage/vertices/properties/:id?relationships=:true + * Description + * Parameters + * Results + * Examples + +---++ Description +Gets the properties of the vertex with specified id. + +---++ Parameters + * :id is the unique id of the vertex. + * :relationships has default value of false. Pass true if relationships should be fetched. + +---++ Results + Properties associated with the specified vertex. + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/metadata/lineage/vertices/properties/40004?relationships=true + +---+++ Result + +{ + "results": + { + "timestamp":"2014-04-25T22:20Z", + "name":"local", + "type":"cluster-entity" + }, + "totalSize":3 +} + Added: falcon/trunk/releases/0.7/src/site/twiki/restapi/Vertices.twiki URL: http://svn.apache.org/viewvc/falcon/trunk/releases/0.7/src/site/twiki/restapi/Vertices.twiki?rev=1702009&view=auto ============================================================================== --- falcon/trunk/releases/0.7/src/site/twiki/restapi/Vertices.twiki (added) +++ falcon/trunk/releases/0.7/src/site/twiki/restapi/Vertices.twiki Wed Sep 9 13:49:51 2015 @@ -0,0 +1,37 @@ +---++ GET api/metadata/lineage/vertices?key=:key&value=:value + * Description + * Parameters + * Results + * Examples + +---++ Description +Get all vertices for a key index given the specified value. + +---++ Parameters + * :key is the key to be matched. + * :value is the associated value of the key. + +---++ Results +All vertices matching given property key and a value. + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/metadata/lineage/vertices?key=name&value=sampleIngestProcess + +---+++ Result + +{ + "results": [ + { + "timestamp":"2014-04-21T20:55Z", + "name":"sampleIngestProcess", + "type":"process-instance", + "version":"2.0.0", + "_id":4, + "_type":"vertex" + } + ], + "totalSize": 1 +} + Modified: falcon/trunk/releases/pom.xml URL: http://svn.apache.org/viewvc/falcon/trunk/releases/pom.xml?rev=1702009&r1=1702008&r2=1702009&view=diff ============================================================================== --- falcon/trunk/releases/pom.xml (original) +++ falcon/trunk/releases/pom.xml Wed Sep 9 13:49:51 2015 @@ -36,6 +36,7 @@ 0.5-incubating 0.6-incubating 0.6.1 + 0.7