openwhisk-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From berst...@apache.org
Subject [incubator-openwhisk] branch master updated: Limit length of HTTP body displayed when debugging. (#2491)
Date Fri, 04 Aug 2017 14:34:47 GMT
This is an automated email from the ASF dual-hosted git repository.

berstler pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk.git


The following commit(s) were added to refs/heads/master by this push:
     new 7b5671d  Limit length of HTTP body displayed when debugging. (#2491)
7b5671d is described below

commit 7b5671d7d0f3920e69ca54fce3322887ce4e380d
Author: Benjamin Poole <Benjamin.D.Poole@ibm.com>
AuthorDate: Fri Aug 4 10:34:44 2017 -0400

    Limit length of HTTP body displayed when debugging. (#2491)
    
    * Limited http req body in debug
    * Added error checking
    * Removed largeFile, added asset-cleaner, removed --debug from create and refined code
    * Removed temp file creation
    * Now only limits HTTP Req/Resp Body with flag --verbose
    * Now uses json to parse and find code field
    * Update based test location, JSON parsing and comments
---
 docs/rest_api.md                                   | 30 +++++++------
 .../whisk/core/cli/test/WskBasicUsageTests.scala   | 16 +++++++
 .../whisk/core/limits/ActionLimitsTests.scala      |  1 -
 tools/cli/go-whisk/whisk/client.go                 | 52 ++++++++++++++++++++--
 tools/cli/go-whisk/whisk/trace.go                  |  5 ++-
 5 files changed, 83 insertions(+), 21 deletions(-)

diff --git a/docs/rest_api.md b/docs/rest_api.md
index 29f8584..aa6ae5d 100644
--- a/docs/rest_api.md
+++ b/docs/rest_api.md
@@ -29,11 +29,11 @@ There are entity endpoints for each type of entity:
 - `https://{APIHOST}/api/v1/namespaces/{namespace}/packages/{packageName}`
 - `https://{APIHOST}/api/v1/namespaces/{namespace}/activations/{activationName}`
 
-The namespace and activation endpoints support only GET requests. The actions, triggers,
rules, and packages endpoints support GET, PUT, and DELETE requests. The endpoints of actions,
triggers, and rules also support POST requests, which are used to invoke actions and triggers
and enable or disable rules. 
+The namespace and activation endpoints support only GET requests. The actions, triggers,
rules, and packages endpoints support GET, PUT, and DELETE requests. The endpoints of actions,
triggers, and rules also support POST requests, which are used to invoke actions and triggers
and enable or disable rules.
 
-All APIs are protected with HTTP Basic authentication. 
+All APIs are protected with HTTP Basic authentication.
 You can use the [wskadmin](../tools/admin/wskadmin) tool to generate a new namespace and
authentication.
-The Basic authentication credentials are in the `AUTH` property in your `~/.wskprops` file,
delimited by a colon. 
+The Basic authentication credentials are in the `AUTH` property in your `~/.wskprops` file,
delimited by a colon.
 You can also retrieve these credentials using the CLI running `wsk property get --auth`.
 
 
@@ -70,7 +70,9 @@ The OpenWhisk API supports request-response calls from web clients. OpenWhisk
re
 ## Using the CLI verbose mode
 
 The OpenWhisk CLI is an interface to the OpenWhisk REST API.
-You can run the CLI in verbose mode with the flag `-v`, this will print all the information
about the HTTP request and response.
+You can run the CLI in verbose mode with the flag `-v`, this will print truncated information
about the HTTP request and response. To print all information use the flag `-d` for debug.
+
+**Note** HTTP request and response bodies will only be truncated if they exceed 1000 bytes.
 
 Let's try getting the namespace value for the current user.
 ```
@@ -97,7 +99,7 @@ Response body received:
 ["john@example.com_dev"]
 ```
 
-As you can see you the printed information provides the properties of the HTTP request, it
performs a HTTP method `GET` on the URL `https://openwhisk.ng.bluemix.net/api/v1/namespaces`
using a Basic Authorization header `Basic XXXYYYY`. 
+As you can see you the printed information provides the properties of the HTTP request, it
performs a HTTP method `GET` on the URL `https://openwhisk.ng.bluemix.net/api/v1/namespaces`
using a Basic Authorization header `Basic XXXYYYY`.
 Notice that the authorization value is your base64-encoded OpenWhisk authorization string.
 The response is of content type `application/json`.
 
@@ -105,7 +107,7 @@ The response is of content type `application/json`.
 
 To create or update an action send a HTTP request with method `PUT` on the the actions collection.
For example, to create a `nodejs:6` action with the name `hello` using a single file content
use the following:
 ```bash
-curl -u $AUTH -d '{"namespace":"_","name":"hello","exec":{"kind":"nodejs:6","code":"function
main(params) { return {payload:\"Hello \"+params.name}}"}}' -X PUT -H "Content-Type: application/json"
https://openwhisk.ng.bluemix.net/api/v1/namespaces/_/actions/hello?overwrite=true 
+curl -u $AUTH -d '{"namespace":"_","name":"hello","exec":{"kind":"nodejs:6","code":"function
main(params) { return {payload:\"Hello \"+params.name}}"}}' -X PUT -H "Content-Type: application/json"
https://openwhisk.ng.bluemix.net/api/v1/namespaces/_/actions/hello?overwrite=true
 ```
 
 To perform a blocking invocation on an action, send a HTTP request with a method `POST` and
body containing the input parameter `name` use the following:
@@ -151,7 +153,7 @@ If you just want to get the `response.result`, run the command again with
the qu
 ```bash
 curl -u $AUTH "https://openwhisk.ng.bluemix.net/api/v1/namespaces/_/actions/hello?blocking=true&result=true"
\
 -X POST -H "Content-Type: application/json" \
--d '{"name":"John"}' 
+-d '{"name":"John"}'
 ```
 You get the following response:
 ```json
@@ -191,7 +193,7 @@ Create a sequence with the actions `/whisk.system/utils/split` and `/whisk.syste
 ```bash
 curl -u $AUTH https://openwhisk.ng.bluemix.net/api/v1/namespaces/_/actions/sequenceAction?overwrite=true
\
 -X PUT -H "Content-Type: application/json" \
--d '{"namespace":"_","name":"sequenceAction","exec":{"kind":"sequence","components":["/whisk.system/utils/split","/whisk.system/utils/sort"]},"annotations":[{"key":"web-export","value":true},{"key":"raw-http","value":false},{"key":"final","value":true}]}'

+-d '{"namespace":"_","name":"sequenceAction","exec":{"kind":"sequence","components":["/whisk.system/utils/split","/whisk.system/utils/sort"]},"annotations":[{"key":"web-export","value":true},{"key":"raw-http","value":false},{"key":"final","value":true}]}'
 ```
 
 Take into account when specifying the names of the actions, they have to be full qualified.
@@ -204,7 +206,7 @@ Create a trigger with name `events` with a default parameter `type` with
value `
 ```bash
 curl -u $AUTH https://openwhisk.ng.bluemix.net/api/v1/namespaces/_/triggers/events?overwrite=true
\
 -X PUT -H "Content-Type: application/json" \
--d '{"name":"events","parameters":[{"key":"type","value":"webhook"}]}' 
+-d '{"name":"events","parameters":[{"key":"type","value":"webhook"}]}'
 ```
 
 Now whenever you have an event that needs to fire this trigger it just takes an HTTP request
with a method `POST` using the OpenWhisk Authorization key.
@@ -214,7 +216,7 @@ To fire the trigger `events` with a parameter `temperature`, send the
following
 ```bash
 curl -u $AUTH https://openwhisk.ng.bluemix.net/api/v1/namespaces/_/triggers/events \
 -X POST -H "Content-Type: application/json" \
--d '{"temperature":60}' 
+-d '{"temperature":60}'
 ```
 
 ### Triggers with Feed Actions
@@ -245,7 +247,7 @@ Now that the trigger is created, lets invoke the feed action
 ```bash
 curl -u $AUTH "https://openwhisk.ng.bluemix.net/api/v1/namespaces/whisk.system/actions/alarms/alarm?blocking=true&result=false"
\
 -X POST -H "Content-Type: application/json" \
--d "{\"authKey\":\"$AUTH\",\"cron\":\"0 */2 * * *\",\"lifecycleEvent\":\"CREATE\",\"triggerName\":\"/_/periodic\"}"

+-d "{\"authKey\":\"$AUTH\",\"cron\":\"0 */2 * * *\",\"lifecycleEvent\":\"CREATE\",\"triggerName\":\"/_/periodic\"}"
 ```
 
 Deleting the trigger is a similar to creating the trigger, this time deleting the trigger
and also using the feed action to configure the feed provider to delete the handler for the
trigger.
@@ -260,7 +262,7 @@ curl -u $AUTH "https://openwhisk.ng.bluemix.net/api/v1/namespaces/whisk.system/a
 Now delete the trigger with a HTTP request using `DELETE` method
 ```bash
 curl -u $AUTH https://openwhisk.ng.bluemix.net/api/v1/namespaces/_/triggers/periodic \
--X DELETE -H "Content-Type: application/json" 
+-X DELETE -H "Content-Type: application/json"
 ```
 
 ## Rules
@@ -269,7 +271,7 @@ To create a rule that associates a trigger with an action, send a HTTP
request w
 ```bash
 curl -u $AUTH https://openwhisk.ng.bluemix.net/api/v1/namespaces/_/rules/t2a?overwrite=true
\
 -X PUT -H "Content-Type: application/json" \
--d '{"name":"t2a","status":"","trigger":"/_/events","action":"/_/hello"}' 
+-d '{"name":"t2a","status":"","trigger":"/_/events","action":"/_/hello"}'
 ```
 
 Rules can be enabled or disabled, and you can change the status of the rule by updating its
status property. For example, to disable the rule `t2a` send in the body of the request `status:
"inactive"` with a `POST` method.
@@ -286,7 +288,7 @@ To create an action in a package you have to create a package first, to
create a
 ```bash
 curl -u $AUTH https://openwhisk.ng.bluemix.net/api/v1/namespaces/_/packages/iot?overwrite=true
\
 -X PUT -H "Content-Type: application/json" \
--d '{"namespace":"_","name":"iot"}' 
+-d '{"namespace":"_","name":"iot"}'
 ```
 
 ## Activations
diff --git a/tests/src/test/scala/whisk/core/cli/test/WskBasicUsageTests.scala b/tests/src/test/scala/whisk/core/cli/test/WskBasicUsageTests.scala
index a4109e7..af1297b 100644
--- a/tests/src/test/scala/whisk/core/cli/test/WskBasicUsageTests.scala
+++ b/tests/src/test/scala/whisk/core/cli/test/WskBasicUsageTests.scala
@@ -633,6 +633,22 @@ class WskBasicUsageTests
 
             wsk.action.get(nonExistentActionName, url = Some(true), expectedExitCode = NOT_FOUND)
     }
+    it should "limit length of HTTP request and response bodies for --verbose" in withAssetCleaner(wskprops)
{
+        (wp, assetHelper) =>
+            val name = "limitVerbose"
+            val msg = "will be truncated"
+            val params = Seq("-p", "bigValue", "a" * 1000)
+
+            assetHelper.withCleaner(wsk.action, name) {
+                (action, _) => action.create(name, Some(TestUtils.getTestActionFilename("echo.js")))
+            }
+
+            val truncated = wsk.cli(Seq("action", "invoke", name, "-b", "-v", "--auth", wskprops.authKey)
++ params ++ wskprops.overrides).stdout
+            msg.r.findAllIn(truncated).length shouldBe 2
+
+            val notTruncated = wsk.cli(Seq("action", "invoke", name, "-b", "-d", "--auth",
wskprops.authKey) ++ params ++ wskprops.overrides).stdout
+            msg.r.findAllIn(notTruncated).length shouldBe 0
+    }
 
     behavior of "Wsk packages"
 
diff --git a/tests/src/test/scala/whisk/core/limits/ActionLimitsTests.scala b/tests/src/test/scala/whisk/core/limits/ActionLimitsTests.scala
index 3983bfb..e1219eb 100644
--- a/tests/src/test/scala/whisk/core/limits/ActionLimitsTests.scala
+++ b/tests/src/test/scala/whisk/core/limits/ActionLimitsTests.scala
@@ -268,5 +268,4 @@ class ActionLimitsTests extends TestHelpers with WskTestHelpers {
                 }
             }
     }
-
 }
diff --git a/tools/cli/go-whisk/whisk/client.go b/tools/cli/go-whisk/whisk/client.go
index dbe8dd3..0c1095f 100644
--- a/tools/cli/go-whisk/whisk/client.go
+++ b/tools/cli/go-whisk/whisk/client.go
@@ -223,6 +223,29 @@ func (c *Client) addAuthHeader(req *http.Request, authRequired bool)
error {
     return nil
 }
 
+// bodyTruncator limits the size of Req/Resp Body for --verbose ONLY.
+// It returns truncated Req/Resp Body, reloaded io.ReadCloser and any errors.
+func bodyTruncator(body io.ReadCloser) (string, io.ReadCloser, error) {
+    limit := 1000    // 1000 byte limit, anything over is truncated
+
+    data, err := ioutil.ReadAll(body)
+    if err != nil {
+        Verbose("ioutil.ReadAll(req.Body) error: %s\n", err)
+        werr := MakeWskError(err, EXITCODE_ERR_NETWORK, DISPLAY_MSG, NO_DISPLAY_USAGE)
+        return "", body, werr
+    }
+
+    reload := ioutil.NopCloser(bytes.NewBuffer(data))
+
+    if len(data) > limit {
+        Verbose("Body exceeds %d bytes and will be truncated\n", limit)
+        newData := string(data)[:limit] + "..."
+        return string(newData), reload, nil
+    }
+
+    return string(data), reload, nil
+}
+
 // Do sends an API request and returns the API response.  The API response is
 // JSON decoded and stored in the value pointed to by v, or returned as an
 // error if an API error has occurred.  If v implements the io.Writer
@@ -230,17 +253,27 @@ func (c *Client) addAuthHeader(req *http.Request, authRequired bool)
error {
 // first decode it.
 func (c *Client) Do(req *http.Request, v interface{}, ExitWithErrorOnTimeout bool) (*http.Response,
error) {
     var err error
+    var truncatedBody string
 
     if IsVerbose() {
         fmt.Println("REQUEST:")
         fmt.Printf("[%s]\t%s\n", req.Method, req.URL)
+
         if len(req.Header) > 0 {
             fmt.Println("Req Headers")
             PrintJSON(req.Header)
         }
+
         if req.Body != nil {
             fmt.Println("Req Body")
-            fmt.Println(req.Body)
+            if !IsDebug() {
+                if truncatedBody, req.Body, err = bodyTruncator(req.Body); err != nil {
+                    return nil, err
+                }
+                fmt.Println(truncatedBody)
+            } else {
+                fmt.Println(req.Body)
+            }
             Debug(DbgInfo, "Req Body (ASCII quoted string):\n%+q\n", req.Body)
         }
     }
@@ -252,10 +285,12 @@ func (c *Client) Do(req *http.Request, v interface{}, ExitWithErrorOnTimeout
boo
         werr := MakeWskError(err, EXITCODE_ERR_NETWORK, DISPLAY_MSG, NO_DISPLAY_USAGE)
         return nil, werr
     }
+
     // Don't "defer resp.Body.Close()" here because the body is reloaded to allow caller
to
     // do custom body parsing, such as handling per-route error responses.
     Verbose("RESPONSE:")
     Verbose("Got response with code %d\n", resp.StatusCode)
+
     if (IsVerbose() && len(resp.Header) > 0) {
         fmt.Println("Resp Headers")
         PrintJSON(resp.Header)
@@ -268,14 +303,23 @@ func (c *Client) Do(req *http.Request, v interface{}, ExitWithErrorOnTimeout
boo
         werr := MakeWskError(err, EXITCODE_ERR_NETWORK, DISPLAY_MSG, NO_DISPLAY_USAGE)
         return resp, werr
     }
-    Verbose("Response body size is %d bytes\n", len(data))
-    Verbose("Response body received:\n%s\n", string(data))
-    Debug(DbgInfo, "Response body received (ASCII quoted string):\n%+q\n", string(data))
 
     // Reload the response body to allow caller access to the body; otherwise,
     // the caller will have any empty body to read
     resp.Body = ioutil.NopCloser(bytes.NewBuffer(data))
 
+    Verbose("Response body size is %d bytes\n", len(data))
+
+    if !IsDebug() {
+        if truncatedBody, resp.Body, err = bodyTruncator(resp.Body); err != nil {
+            return nil, err
+        }
+        Verbose("Response body received:\n%s\n", truncatedBody)
+    } else {
+        Verbose("Response body received:\n%s\n", string(data))
+        Debug(DbgInfo, "Response body received (ASCII quoted string):\n%+q\n", string(data))
+    }
+
     // With the HTTP response status code and the HTTP body contents,
     // the possible response scenarios are:
     //
diff --git a/tools/cli/go-whisk/whisk/trace.go b/tools/cli/go-whisk/whisk/trace.go
index 16b4523..26f7c80 100644
--- a/tools/cli/go-whisk/whisk/trace.go
+++ b/tools/cli/go-whisk/whisk/trace.go
@@ -54,6 +54,9 @@ func SetVerbose (b bool) {
 func IsVerbose() bool {
     return isVerbose || isDebug
 }
+func IsDebug() bool {
+    return isDebug
+}
 
 /* Function for tracing debug level messages to stdout
    Output format:
@@ -90,5 +93,3 @@ func Verbose(msgFormat string, args ...interface{}) {
         fmt.Printf("%v", msg)
     }
 }
-
-

-- 
To stop receiving notification emails like this one, please contact
['"commits@openwhisk.apache.org" <commits@openwhisk.apache.org>'].

Mime
View raw message