Return-Path: X-Original-To: apmail-falcon-commits-archive@minotaur.apache.org Delivered-To: apmail-falcon-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4A67511A3A for ; Thu, 25 Sep 2014 06:04:47 +0000 (UTC) Received: (qmail 86018 invoked by uid 500); 25 Sep 2014 06:04:47 -0000 Delivered-To: apmail-falcon-commits-archive@falcon.apache.org Received: (qmail 85981 invoked by uid 500); 25 Sep 2014 06:04:47 -0000 Mailing-List: contact commits-help@falcon.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@falcon.incubator.apache.org Delivered-To: mailing list commits@falcon.incubator.apache.org Received: (qmail 85972 invoked by uid 99); 25 Sep 2014 06:04:47 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Sep 2014 06:04:47 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Sep 2014 06:04:22 +0000 Received: from eris.apache.org (localhost [127.0.0.1]) by eris.apache.org (Postfix) with ESMTP id E2F032388B45; Thu, 25 Sep 2014 06:04:19 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1627461 [5/5] - in /incubator/falcon: site/ site/0.3-incubating/ site/0.4-incubating/ site/0.5-incubating/ site/restapi/ trunk/ trunk/general/ trunk/general/src/site/ trunk/releases/ trunk/releases/0.5-incubating/ trunk/releases/0.5-incuba... Date: Thu, 25 Sep 2014 06:04:14 -0000 To: commits@falcon.incubator.apache.org From: sriksun@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20140925060419.E2F032388B45@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Added: incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceLogs.twiki URL: http://svn.apache.org/viewvc/incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceLogs.twiki?rev=1627461&view=auto ============================================================================== --- incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceLogs.twiki (added) +++ incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceLogs.twiki Thu Sep 25 06:04:11 2014 @@ -0,0 +1,49 @@ +---++ GET /api/instance/logs/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Get log of a specific instance of an entity. + +---++ Parameters + * :entity-type can either be a feed or a process. + * :entity-name is name of the entity. + * start is the start time of the instance that you want to refer to + * end is the end time of the instance that you want to refer to + +---++ Results +Log of specified instance. + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/instance/logs/process/SampleProcess?colo=*&start=2012-04-03T07:00Z + +---+++ Result + +{ + "instances": [ + { + "actions": [ + { + "logFile": "http:\/\/localhost:50070\/data\/apps\/falcon\/staging\/falcon\/workflows\/process\/SampleProcess\/logs\/job-2012-04-03-07-00\/000\/pig_SUCCEEDED.log", + "status": "SUCCEEDED", + "action": "pig" + } + ], + "details": "", + "endTime": "2013-10-21T14:40:26-07:00", + "startTime": "2013-10-21T14:39:56-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:50070\/data\/apps\/falcon\/staging\/falcon\/workflows\/process\/SampleProcess\/logs\/job-2012-04-03-07-00\/000\/oozie.log", + "status": "SUCCEEDED", + "instance": "2012-04-03T07:00Z" + } + ], + "requestId": "default\/3527038e-8334-4e50-8173-76c4fa430d0b\n", + "message": "default\/STATUS\n", + "status": "SUCCEEDED" +} + Added: incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceRerun.twiki URL: http://svn.apache.org/viewvc/incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceRerun.twiki?rev=1627461&view=auto ============================================================================== --- incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceRerun.twiki (added) +++ incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceRerun.twiki Thu Sep 25 06:04:11 2014 @@ -0,0 +1,40 @@ +---++ POST /api/instance/rerun/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Rerun a specific instance of an entity. + +---++ Parameters + * :entity-type can either be a feed or a process. + * :entity-name is name of the entity. + * start is the start time of the instance that you want to refer to + +---++ Results +Results of the rerun command. + +---++ Examples +---+++ Rest Call + +POST http://localhost:15000/api/instance/rerun/process/SampleProcess?colo=*&start=2012-04-03T07:00Z + +---+++ Result + +{ + "instances": [ + { + "details": "", + "startTime": "2013-10-21T15:10:47-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933395-oozie-rgau-W", + "status": "RUNNING", + "instance": "2012-04-03T07:00Z" + } + ], + "requestId": "default\/7a3582bd-608c-45a7-9b74-1837b51ba6d5\n", + "message": "default\/RERUN\n", + "status": "SUCCEEDED" +} + Added: incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceResume.twiki URL: http://svn.apache.org/viewvc/incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceResume.twiki?rev=1627461&view=auto ============================================================================== --- incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceResume.twiki (added) +++ incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceResume.twiki Thu Sep 25 06:04:11 2014 @@ -0,0 +1,40 @@ +---++ POST /api/instance/resume/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Resume a specific instance of an entity. + +---++ Parameters + * :entity-type can either be a feed or a process. + * :entity-name is name of the entity. + * start is the start time of the instance that you want to refer to + +---++ Results +Results of the resume command. + +---++ Examples +---+++ Rest Call + +POST http://localhost:15000/api/instance/resume/process/SampleProcess?colo=*&start=2012-04-03T07:00Z + +---+++ Result + +{ + "instances": [ + { + "details": "", + "startTime": "2013-10-21T15:19:57-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933395-oozie-rgau-W", + "status": "RUNNING", + "instance": "2012-04-03T07:00Z" + } + ], + "requestId": "default\/e88ff2e0-2af7-4829-a360-f92e95be2981\n", + "message": "default\/RESUME\n", + "status": "SUCCEEDED" +} + Added: incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceRunning.twiki URL: http://svn.apache.org/viewvc/incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceRunning.twiki?rev=1627461&view=auto ============================================================================== --- incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceRunning.twiki (added) +++ incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceRunning.twiki Thu Sep 25 06:04:11 2014 @@ -0,0 +1,37 @@ +---++ GET /api/instance/running/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Get a list of instances currently running for a given entity. + +---++ Parameters + * :entity-type can either be a feed or a process. + * :entity-name is name of the entity. + +---++ Results +List of instances currently running. + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/instance/running/process/SampleProcess?colo=* + +---+++ Result + +{ + "instances": [ + { + "startTime": "2013-10-21T14:39:28-07:00", + "cluster": "primary-cluster", + "status": "RUNNING", + "instance": "2012-04-03T06:00Z" + } + ], + "requestId": "default\/12e9a7d4-3b4f-4a76-b471-c8f3786a62a0\n", + "message": "default\/Running Instances\n", + "status": "SUCCEEDED" +} + Added: incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceStatus.twiki URL: http://svn.apache.org/viewvc/incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceStatus.twiki?rev=1627461&view=auto ============================================================================== --- incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceStatus.twiki (added) +++ incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceStatus.twiki Thu Sep 25 06:04:11 2014 @@ -0,0 +1,42 @@ +---++ GET /api/instance/status/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Get status of a specific instance of an entity. + +---++ Parameters + * :entity-type can either be a feed or a process. + * :entity-name is name of the entity. + * start is the start time of the instance that you want to refer to + * end is the end time of the instance that you want to refer to + +---++ Results +Status of the specified instance. + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/instance/status/process/SampleProcess?colo=*&start=2012-04-03T07:00Z + +---+++ Result + +{ + "instances": [ + { + "details": "", + "endTime": "2013-10-21T14:40:26-07:00", + "startTime": "2013-10-21T14:39:56-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933395-oozie-rgau-W", + "status": "SUCCEEDED", + "instance": "2012-04-03T07:00Z" + } + ], + "requestId": "default\/e15bb378-d09f-4911-9df2-5334a45153d2\n", + "message": "default\/STATUS\n", + "status": "SUCCEEDED" +} + Added: incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceSummary.twiki URL: http://svn.apache.org/viewvc/incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceSummary.twiki?rev=1627461&view=auto ============================================================================== --- incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceSummary.twiki (added) +++ incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceSummary.twiki Thu Sep 25 06:04:11 2014 @@ -0,0 +1,45 @@ +---++ GET /api/instance/summary/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Get summary of instance/instances of an entity. + +---++ Parameters + * :entity-type can either be a feed or a process. + * :entity-name is name of the entity. + * start is the start time of the instance that you want to refer to + * end is the end time of the instance that you want to refer to + +---++ Results +Summary of the instances over the specified time range + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/instance/summary/process/WordCount?colo=*&start=2014-01-21T13:00Z&end=2014-01-21T16:00Z +Remote-User: suhas + +---+++ Result + +{ + "status":"SUCCEEDED", + "message":"default/SUMMARY\n", + "requestId":"default/c344567b-da73-44d5-bcd4-bf456524934c\n", + "instancesSummary": + { + "cluster":"local", + "map": + { + "entry": + { + "key":"SUCCEEDED", + "value":"3" + } + } + } + } +} + Added: incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceSuspend.twiki URL: http://svn.apache.org/viewvc/incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceSuspend.twiki?rev=1627461&view=auto ============================================================================== --- incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceSuspend.twiki (added) +++ incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/InstanceSuspend.twiki Thu Sep 25 06:04:11 2014 @@ -0,0 +1,41 @@ +---++ POST /api/instance/suspend/:entity-type/:entity-name + * Description + * Parameters + * Results + * Examples + +---++ Description +Suspend a specific instance of an entity. + +---++ Parameters + * :entity-type can either be a feed or a process. + * :entity-name is name of the entity. + * start is the start time of the instance that you want to refer to + +---++ Results +Results of the suspend command. + +---++ Examples +---+++ Rest Call + +POST http://localhost:15000/api/instance/suspend/process/SampleProcess?colo=*&start=2012-04-03T07:00Z + +---+++ Result + +{ + "instances": [ + { + "details": "", + "endTime": "2013-10-21T15:15:01-07:00", + "startTime": "2013-10-21T15:14:32-07:00", + "cluster": "primary-cluster", + "logFile": "http:\/\/localhost:11000\/oozie?job=0000070-131021115933395-oozie-rgau-W", + "status": "SUCCEEDED", + "instance": "2012-04-03T07:00Z" + } + ], + "requestId": "default\/ff07e45b-b6da-4f47-ae96-9182bd8a7e53\n", + "message": "default\/SUSPEND\n", + "status": "SUCCEEDED" +} + Added: incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/ResourceList.twiki URL: http://svn.apache.org/viewvc/incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/ResourceList.twiki?rev=1627461&view=auto ============================================================================== --- incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/ResourceList.twiki (added) +++ incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/ResourceList.twiki Thu Sep 25 06:04:11 2014 @@ -0,0 +1,75 @@ +---+ RESTful Resources + +---++ Resource List + * REST Call on Entity Resource + * REST Call on Feed/Process Instances + * REST Call on Admin Resource + * REST Call on Lineage Graph Resource + +---++ Authentication + +When security is off (Pseudo/Simple), the authenticated user is the username specified in the user.name query +parameter. If the user.name parameter is not set, the server may either set the authenticated user to a default web +user, if there is any, or return an error response. + +When security is on (kerberos), authentication is performed by Kerberos SPNEGO. + +Below are examples using the curl command tool. + +Authentication when security is off (Pseudo/Simple): + +curl -i "http://:/?[user.name=&]=..." + + +Authentication using Kerberos SPNEGO when security is on: + +curl -i --negotiate -u : "http://:/?=..." + + +See also: [[../Security.twiki][Security in Falcon]] + +---++ REST Call on Admin Resource + +| *Call Type* | *Resource* | *Description* | +| GET | [[AdminStack][api/admin/stack]] | Get stack of the server | +| GET | [[AdminVersion][api/admin/version]] | Get version of the server | +| GET | [[AdminConfig][api/admin/config/:config-type]] | Get configuration information of the server | + +---++ REST Call on Entity Resource + +| *Call Type* | *Resource* | *Description* | +| POST | [[EntityValidate][api/entities/validate/:entity-type]] | Validate the entity | +| POST | [[EntitySubmit][api/entities/submit/:entity-type]] | Submit the entity | +| POST | [[EntityUpdate][api/entities/update/:entity-type/:entity-name]] | Update the entity | +| POST | [[EntitySubmitAndSchedule][api/entities/submitAndSchedule/:entity-type]] | Submit & Schedule the entity | +| POST | [[EntitySchedule][api/entities/schedule/:entity-type/:entity-name]] | Schedule the entity | +| POST | [[EntitySuspend][api/entities/suspend/:entity-type/:entity-name]] | Suspend the entity | +| POST | [[EntityResume][api/entities/resume/:entity-type/:entity-name]] | Resume the entity | +| DELETE | [[EntityDelete][api/entities/delete/:entity-type/:entity-name]] | Delete the entity | +| GET | [[EntityStatus][api/entities/status/:entity-type/:entity-name]] | Get the status of the entity | +| GET | [[EntityDefinition][api/entities/definition/:entity-type/:entity-name]] | Get the definition of the entity | +| GET | [[EntityList][api/entities/list/:entity-type?fields=:fields]] | Get the list of entities | +| GET | [[EntityDependencies][api/entities/dependencies/:entity-type/:entity-name]] | Get the dependencies of the entity | + +---++ REST Call on Feed and Process Instances + +| *Call Type* | *Resource* | *Description* | +| GET | [[InstanceRunning][api/instance/running/:entity-type/:entity-name]] | List of running instances. | +| GET | [[InstanceStatus][api/instance/status/:entity-type/:entity-name]] | Status of a given instance | +| POST | [[InstanceKill][api/instance/kill/:entity-type/:entity-name]] | Kill a given instance | +| POST | [[InstanceSuspend][api/instance/suspend/:entity-type/:entity-name]] | Suspend a running instance | +| POST | [[InstanceResume][api/instance/resume/:entity-type/:entity-name]] | Resume a given instance | +| POST | [[InstanceRerun][api/instance/rerun/:entity-type/:entity-name]] | Rerun a given instance | +| GET | [[InstanceLogs][api/instance/logs/:entity-type/:entity-name]] | Get logs of a given instance | + +---++ REST Call on Lineage Graph + +| *Call Type* | *Resource* | *Description* | +| GET | [[Graph][api/graphs/lineage/serialize]] | dump the graph | +| GET | [[AllVertices][api/graphs/lineage/vertices/all]] | get all vertices | +| GET | [[Vertices][api/graphs/lineage/vertices?key=:key&value=:value]] | get all vertices for a key index | +| GET | [[Vertex][api/graphs/lineage/vertices/:id]] | get the vertex with the specified id | +| GET | [[VertexProperties][api/graphs/lineage/vertices/properties/:id?relationships=:true]] | get the properties of the vertex with the specified id | +| GET | [[AdjacentVertices][api/graphs/lineage/vertices/:id/:direction]] | get the adjacent vertices or edges of the vertex with the specified direction | +| GET | [[AllEdges][api/graphs/lineage//edges/all]] | get all edges | +| GET | [[Edge][api/graphs/lineage/edges/:id]] | get the edge with the specified id | Added: incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/Vertex.twiki URL: http://svn.apache.org/viewvc/incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/Vertex.twiki?rev=1627461&view=auto ============================================================================== --- incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/Vertex.twiki (added) +++ incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/Vertex.twiki Thu Sep 25 06:04:11 2014 @@ -0,0 +1,35 @@ +---++ GET api/graphs/lineage/vertices/:id + * Description + * Parameters + * Results + * Examples + +---++ Description +Gets the vertex with specified id. + +---++ Parameters + * :id is the unique id of the vertex. + +---++ Results +Vertex with the specified id. + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/graphs/lineage/vertices/4 + +---+++ Result + +{ + "results": [ + { + "timestamp":"2014-04-21T20:55Z", + "name":"sampleIngestProcess", + "type":"process-instance", + "version":"2.0.0", + "_id":4, + "_type":"vertex" + } + ] +} + Added: incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/VertexProperties.twiki URL: http://svn.apache.org/viewvc/incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/VertexProperties.twiki?rev=1627461&view=auto ============================================================================== --- incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/VertexProperties.twiki (added) +++ incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/VertexProperties.twiki Thu Sep 25 06:04:11 2014 @@ -0,0 +1,33 @@ +---++ GET api/graphs/lineage/vertices/properties/:id?relationships=:true + * Description + * Parameters + * Results + * Examples + +---++ Description +Gets the properties of the vertex with specified id. + +---++ Parameters + * :id is the unique id of the vertex. + * :relationships has default value of false. Pass true if relationships should be fetched. + +---++ Results + Properties associated with the specified vertex. + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/graphs/lineage/vertices/properties/40004?relationships=true + +---+++ Result + +{ + "results": + { + "timestamp":"2014-04-25T22:20Z", + "name":"local", + "type":"cluster-entity" + }, + "totalSize":3 +} + Added: incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/Vertices.twiki URL: http://svn.apache.org/viewvc/incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/Vertices.twiki?rev=1627461&view=auto ============================================================================== --- incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/Vertices.twiki (added) +++ incubator/falcon/trunk/releases/0.5-incubating/src/site/twiki/restapi/Vertices.twiki Thu Sep 25 06:04:11 2014 @@ -0,0 +1,37 @@ +---++ GET api/graphs/lineage/vertices?key=:key&value=:value + * Description + * Parameters + * Results + * Examples + +---++ Description +Get all vertices for a key index given the specified value. + +---++ Parameters + * :key is the key to be matched. + * :value is the associated value of the key. + +---++ Results +All vertices matching given property key and a value. + +---++ Examples +---+++ Rest Call + +GET http://localhost:15000/api/graphs/lineage/vertices?key=name&value=sampleIngestProcess + +---+++ Result + +{ + "results": [ + { + "timestamp":"2014-04-21T20:55Z", + "name":"sampleIngestProcess", + "type":"process-instance", + "version":"2.0.0", + "_id":4, + "_type":"vertex" + } + ], + "totalSize": 1 +} + Modified: incubator/falcon/trunk/releases/pom.xml URL: http://svn.apache.org/viewvc/incubator/falcon/trunk/releases/pom.xml?rev=1627461&r1=1627460&r2=1627461&view=diff ============================================================================== --- incubator/falcon/trunk/releases/pom.xml (original) +++ incubator/falcon/trunk/releases/pom.xml Thu Sep 25 06:04:11 2014 @@ -22,7 +22,7 @@ org.apache.falcon falcon-website - 0.5-SNAPSHOT + 0.6-SNAPSHOT falcon-website-releases 0.2 @@ -33,6 +33,7 @@ 0.3-incubating 0.4-incubating + 0.5-incubating