From commits-return-3135-archive-asf-public=cust-asf.ponee.io@openwhisk.apache.org Thu Jan 11 10:06:06 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 63838180656 for ; Thu, 11 Jan 2018 10:06:06 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 53F68160C20; Thu, 11 Jan 2018 09:06:06 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id C85A6160C1F for ; Thu, 11 Jan 2018 10:06:04 +0100 (CET) Received: (qmail 54576 invoked by uid 500); 11 Jan 2018 09:06:04 -0000 Mailing-List: contact commits-help@openwhisk.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@openwhisk.apache.org Delivered-To: mailing list commits@openwhisk.apache.org Received: (qmail 54566 invoked by uid 99); 11 Jan 2018 09:06:03 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 11 Jan 2018 09:06:03 +0000 Received: by gitbox.apache.org (ASF Mail Server at gitbox.apache.org, from userid 33) id 17DFB8211B; Thu, 11 Jan 2018 09:06:00 +0000 (UTC) Date: Thu, 11 Jan 2018 09:06:00 +0000 To: "commits@openwhisk.apache.org" Subject: [incubator-openwhisk] branch master updated: Enrich entity views, and add count reducer (#3152) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Message-ID: <151566156062.16369.12315194670055794256@gitbox.apache.org> From: cbickel@apache.org X-Git-Host: gitbox.apache.org X-Git-Repo: incubator-openwhisk X-Git-Refname: refs/heads/master X-Git-Reftype: branch X-Git-Oldrev: 25c8e3c5c109f59093e3bcd8f102a295fea69869 X-Git-Newrev: c7ef96a0a09eb29ec9116bead011d0657196d2ec X-Git-Rev: c7ef96a0a09eb29ec9116bead011d0657196d2ec X-Git-NotificationType: ref_changed_plus_diff X-Git-Multimail-Version: 1.5.dev Auto-Submitted: auto-generated This is an automated email from the ASF dual-hosted git repository. cbickel pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk.git The following commit(s) were added to refs/heads/master by this push: new c7ef96a Enrich entity views, and add count reducer (#3152) c7ef96a is described below commit c7ef96a0a09eb29ec9116bead011d0657196d2ec Author: rodric rabbah AuthorDate: Thu Jan 11 04:05:55 2018 -0500 Enrich entity views, and add count reducer (#3152) * Remove reduce_limit setting for couchdb as it is no longer necessary. * Upate design docs to provide more asset summaries and support counts. Enrich action list view with limits and "binary" property of exec. Enrich rule list view to include action/trigger names. Enrich package list view to include binding information. Add package list view for public packages. Add count reducers. Factor out all view, easier to then drop it without affecting rest of the views. Update tests for new design docs. --- ansible/README.md | 5 +--- ...document_for_activations_db_filters_v2.1.0.json | 10 +++++++ ..._design_document_for_activations_db_v2.1.0.json | 10 +++++++ ...design_document_for_all_entities_db_v2.1.0.json | 10 +++++++ ...sks_design_document_for_entities_db_v2.1.0.json | 26 +++++++++++++++++ ansible/roles/couchdb/tasks/deploy.yml | 12 -------- .../main/scala/whisk/core/entity/WhiskAction.scala | 24 ++++++++++++++-- .../scala/whisk/core/entity/WhiskActivation.scala | 9 ++++-- .../main/scala/whisk/core/entity/WhiskEntity.scala | 15 +++++++--- .../scala/whisk/core/entity/WhiskPackage.scala | 21 ++++++++------ .../main/scala/whisk/core/controller/Actions.scala | 16 +++-------- .../scala/whisk/core/controller/Packages.scala | 33 +++++++++------------- .../main/scala/whisk/core/controller/Rules.scala | 8 ++---- .../scala/whisk/core/controller/Triggers.scala | 12 +++----- .../scala/whisk/core/entity/test/ViewTests.scala | 31 ++++++-------------- tools/db/README.md | 1 - 16 files changed, 140 insertions(+), 103 deletions(-) diff --git a/ansible/README.md b/ansible/README.md index 95bbcf9..7651e63 100644 --- a/ansible/README.md +++ b/ansible/README.md @@ -136,10 +136,7 @@ ansible-playbook -i environments/ prereq.yml **Hint:** During playbook execution the `TASK [prereq : check for pip]` can show as failed. This is normal if no pip is installed. The playbook will then move on and install pip on the target machines. ### Deploying Using CouchDB -- Make sure your `db_local.ini` file is set up for CouchDB. See [Setup](#setup) -- If you deploy CouchDB manually (i.e., without using the deploy CouchDB playbook), you must set the `reduce_limit` property on views to `false`. -This may be done via the REST API, as in: `curl -X PUT ${OW_DB_PROTOCOL}://${OW_DB_HOST}:${OW_DB_PORT}/_config/query_server_config/reduce_limit -d '"false"' -u ${OW_DB_USERNAME}:${OW_DB_PASSWORD}`. -- Then execute +- Make sure your `db_local.ini` file is [setup for](#setup) CouchDB then execute: ``` cd diff --git a/ansible/files/whisks_design_document_for_activations_db_filters_v2.1.0.json b/ansible/files/whisks_design_document_for_activations_db_filters_v2.1.0.json new file mode 100644 index 0000000..9e2b1f0 --- /dev/null +++ b/ansible/files/whisks_design_document_for_activations_db_filters_v2.1.0.json @@ -0,0 +1,10 @@ +{ + "_id": "_design/whisks-filters.v2.1.0", + "language": "javascript", + "views": { + "activations": { + "map": "function (doc) {\n var PATHSEP = \"/\";\n var isActivation = function (doc) { return (doc.activationId !== undefined) };\n var summarize = function (doc) {\n var endtime = doc.end !== 0 ? doc.end : undefined;\n return {\n namespace: doc.namespace,\n name: doc.name,\n version: doc.version,\n publish: doc.publish,\n annotations: doc.annotations,\n activationId: doc.activationId,\n start: doc.start,\n end: endtim [...] + "reduce": "_count" + } + } +} diff --git a/ansible/files/whisks_design_document_for_activations_db_v2.1.0.json b/ansible/files/whisks_design_document_for_activations_db_v2.1.0.json new file mode 100644 index 0000000..87da6ed --- /dev/null +++ b/ansible/files/whisks_design_document_for_activations_db_v2.1.0.json @@ -0,0 +1,10 @@ +{ + "_id": "_design/whisks.v2.1.0", + "language": "javascript", + "views": { + "activations": { + "map": "function (doc) {\n var PATHSEP = \"/\";\n var isActivation = function (doc) { return (doc.activationId !== undefined) };\n var summarize = function (doc) {\n var endtime = doc.end !== 0 ? doc.end : undefined;\n return {\n namespace: doc.namespace,\n name: doc.name,\n version: doc.version,\n publish: doc.publish,\n annotations: doc.annotations,\n activationId: doc.activationId,\n start: doc.start,\n end: endtim [...] + "reduce": "_count" + } + } +} diff --git a/ansible/files/whisks_design_document_for_all_entities_db_v2.1.0.json b/ansible/files/whisks_design_document_for_all_entities_db_v2.1.0.json new file mode 100644 index 0000000..ae8f475 --- /dev/null +++ b/ansible/files/whisks_design_document_for_all_entities_db_v2.1.0.json @@ -0,0 +1,10 @@ +{ + "_id": "_design/all-whisks.v2.1.0", + "language": "javascript", + "views": { + "all": { + "map": "function (doc) {\n var PATHSEP = \"/\";\n\n var isPackage = function (doc) { return (doc.binding !== undefined) };\n var isAction = function (doc) { return (doc.exec !== undefined) };\n var isTrigger = function (doc) { return (doc.exec === undefined && doc.binding === undefined && doc.parameters !== undefined) };\n var isRule = function (doc) { return (doc.trigger !== undefined) };\n \n var collection = function (doc) {\n if (isPackage(doc)) return \"packages\"; [...] + "reduce": "_count" + } + } +} \ No newline at end of file diff --git a/ansible/files/whisks_design_document_for_entities_db_v2.1.0.json b/ansible/files/whisks_design_document_for_entities_db_v2.1.0.json new file mode 100644 index 0000000..34d52f8 --- /dev/null +++ b/ansible/files/whisks_design_document_for_entities_db_v2.1.0.json @@ -0,0 +1,26 @@ +{ + "_id": "_design/whisks.v2.1.0", + "language": "javascript", + "views": { + "rules": { + "map": "function (doc) {\n var PATHSEP = \"/\";\n var isRule = function (doc) { return (doc.trigger !== undefined) };\n if (isRule(doc)) try {\n var ns = doc.namespace.split(PATHSEP);\n var root = ns[0];\n emit([doc.namespace, doc.updated], 1);\n if (root !== doc.namespace) {\n emit([root, doc.updated], 1);\n }\n } catch (e) {}\n}", + "reduce": "_count" + }, + "packages-public": { + "map": "function (doc) {\n var isPublicPackage = function(doc) { \n return Object.keys(doc.binding).length == 0 && doc.publish;\n }\n if (isPublicPackage(doc)) try {\n var value = {\n namespace: doc.namespace,\n name: doc.name,\n version: doc.version,\n publish: doc.publish,\n annotations: doc.annotations,\n updated: doc.updated,\n binding: false\n };\n emit([doc.namespace, doc.updated], value);\n } catch (e) {}\n}", + "reduce": "_count" + }, + "packages": { + "map": "function (doc) {\n var isPackage = function (doc) { return (doc.binding !== undefined) };\n if (isPackage(doc)) try {\n var value = {\n namespace: doc.namespace,\n name: doc.name,\n version: doc.version,\n publish: doc.publish,\n annotations: doc.annotations,\n updated: doc.updated\n };\n if (Object.keys(doc.binding).length > 0) {\n value.binding = doc.binding;\n } else {\n value.binding = false;\n }\n emit([doc.n [...] + "reduce": "_count" + }, + "actions": { + "map": "function (doc) {\n var PATHSEP = \"/\";\n var isAction = function (doc) { return (doc.exec !== undefined) };\n if (isAction(doc)) try {\n var ns = doc.namespace.split(PATHSEP);\n var root = ns[0];\n var value = {\n namespace: doc.namespace,\n name: doc.name,\n version: doc.version,\n publish: doc.publish,\n annotations: doc.annotations,\n limits: doc.limits,\n exec: { binary: doc.exec.binary || false},\n updated: doc.updat [...] + "reduce": "_count" + }, + "triggers": { + "map": "function (doc) {\n var PATHSEP = \"/\";\n var isTrigger = function (doc) { return (doc.exec === undefined && doc.binding === undefined && doc.parameters !== undefined) };\n if (isTrigger(doc)) try {\n var ns = doc.namespace.split(PATHSEP);\n var root = ns[0];\n var value = {\n namespace: doc.namespace,\n name: doc.name,\n version: doc.version,\n publish: doc.publish,\n annotations: doc.annotations,\n updated: doc.updated\n };\n [...] + "reduce": "_count" + } + } +} \ No newline at end of file diff --git a/ansible/roles/couchdb/tasks/deploy.yml b/ansible/roles/couchdb/tasks/deploy.yml index 542111a..2111aae 100644 --- a/ansible/roles/couchdb/tasks/deploy.yml +++ b/ansible/roles/couchdb/tasks/deploy.yml @@ -89,15 +89,3 @@ password: "{{ db_password }}" force_basic_auth: yes when: (inventory_hostname == coordinator) and (db.instances|int >= 2) - -- name: disable reduce limit on views - uri: - url: "{{ db_protocol }}://{{ ansible_host }}:{{ db_port }}/_node/couchdb@{{ ansible_host }}/_config/query_server_config/reduce_limit" - method: PUT - body: > - "false" - body_format: json - status_code: 200 - user: "{{ db_username }}" - password: "{{ db_password }}" - force_basic_auth: yes diff --git a/common/scala/src/main/scala/whisk/core/entity/WhiskAction.scala b/common/scala/src/main/scala/whisk/core/entity/WhiskAction.scala index 4aaddbd..194f774 100644 --- a/common/scala/src/main/scala/whisk/core/entity/WhiskAction.scala +++ b/common/scala/src/main/scala/whisk/core/entity/WhiskAction.scala @@ -24,7 +24,6 @@ import java.util.Base64 import scala.concurrent.ExecutionContext import scala.concurrent.Future import scala.util.{Failure, Success, Try} - import spray.json._ import spray.json.DefaultJsonProtocol._ import whisk.common.TransactionId @@ -160,8 +159,27 @@ case class WhiskAction(namespace: EntityPath, Some( ExecutableWhiskAction(namespace, name, codeExec, parameters, limits, version, publish, annotations) .revision[ExecutableWhiskAction](rev)) - case _ => - None + case _ => None + } + + /** + * This the action summary as computed by the database view. + * Strictly used in view testing to enforce alignment. + */ + override def summaryAsJson = { + if (WhiskEntityQueries.designDoc.endsWith("v2")) { + super.summaryAsJson + } else { + val binary = exec match { + case c: CodeExec[_] => c.binary + case _ => false + } + + JsObject( + super.summaryAsJson.fields + + ("limits" -> limits.toJson) + + ("exec" -> JsObject("binary" -> JsBoolean(binary)))) + } } } diff --git a/common/scala/src/main/scala/whisk/core/entity/WhiskActivation.scala b/common/scala/src/main/scala/whisk/core/entity/WhiskActivation.scala index 0525f50..11caf3e 100644 --- a/common/scala/src/main/scala/whisk/core/entity/WhiskActivation.scala +++ b/common/scala/src/main/scala/whisk/core/entity/WhiskActivation.scala @@ -77,7 +77,10 @@ case class WhiskActivation(namespace: EntityPath, def toJson = WhiskActivation.serdes.write(this).asJsObject - /** This the activation summary as computed by the database view. Strictly used for testing. */ + /** + * This the activation summary as computed by the database view. + * Strictly used in view testing to enforce alignment. + */ override def summaryAsJson = { import WhiskActivation.instantSerdes @@ -91,7 +94,7 @@ case class WhiskActivation(namespace: EntityPath, } JsObject( - super.summaryAsJson.fields + + super.summaryAsJson.fields - "updated" + ("activationId" -> activationId.toJson) + ("start" -> start.toJson) ++ cause.map(("cause" -> _.toJson)) ++ @@ -133,7 +136,7 @@ object WhiskActivation val initTimeAnnotation = "initTime" val waitTimeAnnotation = "waitTime" - private implicit val instantSerdes = new RootJsonFormat[Instant] { + protected[entity] implicit val instantSerdes = new RootJsonFormat[Instant] { def write(t: Instant) = t.toEpochMilli.toJson def read(value: JsValue) = diff --git a/common/scala/src/main/scala/whisk/core/entity/WhiskEntity.scala b/common/scala/src/main/scala/whisk/core/entity/WhiskEntity.scala index eafb7d5..8bafd94 100644 --- a/common/scala/src/main/scala/whisk/core/entity/WhiskEntity.scala +++ b/common/scala/src/main/scala/whisk/core/entity/WhiskEntity.scala @@ -81,15 +81,22 @@ abstract class WhiskEntity protected[entity] (en: EntityName) extends WhiskDocum /** * A JSON view of the entity, that should match the result returned in a list operation. - * This should be synchronized with the views computed in wipeTransientDBs.sh. + * This should be synchronized with the views computed in the databse. + * Strictly used in view testing to enforce alignment. */ - def summaryAsJson = - JsObject( + def summaryAsJson = { + import WhiskActivation.instantSerdes + val base = Map( "namespace" -> namespace.toJson, "name" -> name.toJson, "version" -> version.toJson, WhiskEntity.sharedFieldName -> JsBoolean(publish), - "annotations" -> annotations.toJsArray) + "annotations" -> annotations.toJsArray, + "updated" -> updated.toJson) + if (WhiskEntityQueries.designDoc.endsWith("v2")) { + JsObject(base - "updated") + } else JsObject(base) + } } object WhiskEntity { diff --git a/common/scala/src/main/scala/whisk/core/entity/WhiskPackage.scala b/common/scala/src/main/scala/whisk/core/entity/WhiskPackage.scala index b9c1c46..d730d33 100644 --- a/common/scala/src/main/scala/whisk/core/entity/WhiskPackage.scala +++ b/common/scala/src/main/scala/whisk/core/entity/WhiskPackage.scala @@ -96,13 +96,7 @@ case class WhiskPackage(namespace: EntityPath, /** * Gets binding for package iff this is not already a package reference. */ - def bind: Option[Binding] = { - if (binding.isDefined) { - None - } else { - Some(Binding(namespace.root, name)) - } - } + def bind: Option[Binding] = if (binding.isEmpty) Some(Binding(namespace.root, name)) else None /** * Adds actions to package. The actions list is filtered so that only actions that @@ -137,9 +131,18 @@ case class WhiskPackage(namespace: EntityPath, def toJson = WhiskPackage.serdes.write(this).asJsObject + /** + * This the package summary as computed by the database view. + * Strictly used in view testing to enforce alignment. + */ override def summaryAsJson = { - val JsObject(fields) = super.summaryAsJson - JsObject(fields + (WhiskPackage.bindingFieldName -> binding.isDefined.toJson)) + if (WhiskEntityQueries.designDoc.endsWith("v2")) { + JsObject(super.summaryAsJson.fields + (WhiskPackage.bindingFieldName -> binding.isDefined.toJson)) + } else { + JsObject( + super.summaryAsJson.fields + + (WhiskPackage.bindingFieldName -> binding.map(Binding.serdes.write(_)).getOrElse(JsBoolean(false)))) + } } } diff --git a/core/controller/src/main/scala/whisk/core/controller/Actions.scala b/core/controller/src/main/scala/whisk/core/controller/Actions.scala index 2dd0bbc..0f30d37 100644 --- a/core/controller/src/main/scala/whisk/core/controller/Actions.scala +++ b/core/controller/src/main/scala/whisk/core/controller/Actions.scala @@ -319,21 +319,13 @@ trait WhiskActionsApi extends WhiskCollectionAPI with PostActionActivation with * - 500 Internal Server Error */ override def list(user: Identity, namespace: EntityPath, excludePrivate: Boolean)(implicit transid: TransactionId) = { - // for consistency, all the collections should support the same list API - // but because supporting docs on actions is difficult, the API does not - // offer an option to fetch entities with full docs yet. - // - // the complication with actions is that providing docs on actions in - // package bindings is cannot be do readily done with a couchdb view - // and would require finding all bindings in namespace and - // joining the actions explicitly here. - val docs = false parameter('skip ? 0, 'limit.as[ListLimit] ? ListLimit(collection.defaultListLimit), 'count ? false) { (skip, limit, count) => listEntities { - WhiskAction.listCollectionInNamespace(entityStore, namespace, skip, limit.n, docs) map { list => - val actions = list.fold((js) => js, (as) => as.map(WhiskAction.serdes.write(_))) - FilterEntityList.filter(actions, excludePrivate) + WhiskAction.listCollectionInNamespace(entityStore, namespace, skip, limit.n, includeDocs = false) map { + list => + val actions = list.fold((js) => js, (as) => as.map(WhiskAction.serdes.write(_))) + FilterEntityList.filter(actions, excludePrivate) } } } diff --git a/core/controller/src/main/scala/whisk/core/controller/Packages.scala b/core/controller/src/main/scala/whisk/core/controller/Packages.scala index a6388b1..d469b0e 100644 --- a/core/controller/src/main/scala/whisk/core/controller/Packages.scala +++ b/core/controller/src/main/scala/whisk/core/controller/Packages.scala @@ -174,28 +174,23 @@ trait WhiskPackagesApi extends WhiskCollectionAPI with ReferencedEntities { * - 500 Internal Server Error */ override def list(user: Identity, namespace: EntityPath, excludePrivate: Boolean)(implicit transid: TransactionId) = { - // for consistency, all the collections should support the same list API - // but because supporting docs on actions is difficult, the API does not - // offer an option to fetch entities with full docs yet; see comment in - // Actions API for more. - val docs = false - parameter('skip ? 0, 'limit.as[ListLimit] ? ListLimit(collection.defaultListLimit), 'count ? false) { (skip, limit, count) => listEntities { - WhiskPackage.listCollectionInNamespace(entityStore, namespace, skip, limit.n, docs) map { list => - // any subject is entitled to list packages in any namespace - // however, they shall only observe public packages if the packages - // are not in one of the namespaces the subject is entitled to - val packages = list.fold((js) => js, (ps) => ps.map(WhiskPackage.serdes.write(_))) - - FilterEntityList.filter(packages, excludePrivate, additionalFilter = { - // additionally exclude bindings - _.fields.get(WhiskPackage.bindingFieldName) match { - case Some(JsBoolean(isbinding)) => !isbinding - case _ => false // exclude anything that does not conform - } - }) + WhiskPackage.listCollectionInNamespace(entityStore, namespace, skip, limit.n, includeDocs = false) map { + list => + // any subject is entitled to list packages in any namespace + // however, they shall only observe public packages if the packages + // are not in one of the namespaces the subject is entitled to + val packages = list.fold((js) => js, (ps) => ps.map(WhiskPackage.serdes.write(_))) + + FilterEntityList.filter(packages, excludePrivate, additionalFilter = { + // additionally exclude bindings + _.fields.get(WhiskPackage.bindingFieldName) match { + case Some(JsBoolean(isbinding)) => !isbinding + case _ => false // exclude anything that does not conform + } + }) } } } diff --git a/core/controller/src/main/scala/whisk/core/controller/Rules.scala b/core/controller/src/main/scala/whisk/core/controller/Rules.scala index 9f4be64..0540e7d 100644 --- a/core/controller/src/main/scala/whisk/core/controller/Rules.scala +++ b/core/controller/src/main/scala/whisk/core/controller/Rules.scala @@ -235,15 +235,11 @@ trait WhiskRulesApi extends WhiskCollectionAPI with ReferencedEntities { * - 500 Internal Server Error */ override def list(user: Identity, namespace: EntityPath, excludePrivate: Boolean)(implicit transid: TransactionId) = { - // for consistency, all the collections should support the same list API - // but because supporting docs on actions is difficult, the API does not - // offer an option to fetch entities with full docs yet; see comment in - // Actions API for more. - val docs = false parameter('skip ? 0, 'limit.as[ListLimit] ? ListLimit(collection.defaultListLimit), 'count ? false) { (skip, limit, count) => + val includeDocs = WhiskEntityQueries.designDoc.endsWith("v2.1.0") listEntities { - WhiskRule.listCollectionInNamespace(entityStore, namespace, skip, limit.n, docs) map { list => + WhiskRule.listCollectionInNamespace(entityStore, namespace, skip, limit.n, includeDocs) map { list => val rules = list.fold((js) => js, (rls) => rls.map(WhiskRule.serdes.write(_))) FilterEntityList.filter(rules, excludePrivate) } diff --git a/core/controller/src/main/scala/whisk/core/controller/Triggers.scala b/core/controller/src/main/scala/whisk/core/controller/Triggers.scala index 7cc292d..8c71fce 100644 --- a/core/controller/src/main/scala/whisk/core/controller/Triggers.scala +++ b/core/controller/src/main/scala/whisk/core/controller/Triggers.scala @@ -236,17 +236,13 @@ trait WhiskTriggersApi extends WhiskCollectionAPI { * - 500 Internal Server Error */ override def list(user: Identity, namespace: EntityPath, excludePrivate: Boolean)(implicit transid: TransactionId) = { - // for consistency, all the collections should support the same list API - // but because supporting docs on actions is difficult, the API does not - // offer an option to fetch entities with full docs yet; see comment in - // Actions API for more. - val docs = false parameter('skip ? 0, 'limit.as[ListLimit] ? ListLimit(collection.defaultListLimit), 'count ? false) { (skip, limit, count) => listEntities { - WhiskTrigger.listCollectionInNamespace(entityStore, namespace, skip, limit.n, docs) map { list => - val triggers = list.fold((js) => js, (ts) => ts.map(WhiskTrigger.serdes.write(_))) - FilterEntityList.filter(triggers, excludePrivate) + WhiskTrigger.listCollectionInNamespace(entityStore, namespace, skip, limit.n, includeDocs = false) map { + list => + val triggers = list.fold((js) => js, (ts) => ts.map(WhiskTrigger.serdes.write(_))) + FilterEntityList.filter(triggers, excludePrivate) } } } diff --git a/tests/src/test/scala/whisk/core/entity/test/ViewTests.scala b/tests/src/test/scala/whisk/core/entity/test/ViewTests.scala index 24c861f..63f720d 100644 --- a/tests/src/test/scala/whisk/core/entity/test/ViewTests.scala +++ b/tests/src/test/scala/whisk/core/entity/test/ViewTests.scala @@ -83,12 +83,13 @@ class ViewTests behavior of "Datastore View" - def getAllInNamespace[Au <: WhiskEntity](store: ArtifactStore[Au], ns: EntityPath)( + def getEntitiesInNamespace[Au <: WhiskEntity](entityStore: ArtifactStore[Au], ns: EntityPath)( implicit entities: Seq[WhiskEntity]) = { implicit val tid = transid() - val result = - Await.result(listAllInNamespace(store, ns.root, false, StaleParameter.No), dbOpTimeout).values.toList.flatten - val expected = entities filter { _.namespace.root.toPath == ns } + val map = Await.result(listAllInNamespace(entityStore, ns.root, false, StaleParameter.No), dbOpTimeout) + val result = map.values.toList.flatten + val expected = entities filter (_.namespace.root.toPath == ns) + map.get(WhiskActivation.collectionName) should be(None) result should have length expected.length result should contain theSameElementsAs expected.map(_.summaryAsJson) } @@ -107,18 +108,6 @@ class ViewTests result should contain theSameElementsAs expected.map(_.summaryAsJson) } - def getEntitiesInNamespace(ns: EntityPath)(implicit entities: Seq[WhiskEntity]) = { - implicit val tid = transid() - val map = Await.result(listAllInNamespace(entityStore, ns.root, false), dbOpTimeout) - val result = map.values.toList flatMap { t => - t - } - val expected = entities filter { !_.isInstanceOf[WhiskActivation] } filter { _.namespace.root.toPath == ns } - map.get(WhiskActivation.collectionName) should be(None) - result should have length expected.length - result should contain theSameElementsAs expected.map(_.summaryAsJson) - } - def resolveListMethodForKind(kind: String) = kind match { case "actions" => WhiskAction case "packages" => WhiskPackage @@ -283,7 +272,7 @@ class ViewTests waitOnView(entityStore, namespace1.root, 15, WhiskEntityQueries.viewAll) waitOnView(entityStore, namespace2.root, 14, WhiskEntityQueries.viewAll) - getAllInNamespace(entityStore, namespace1) + getEntitiesInNamespace(entityStore, namespace1) getKindInNamespace(entityStore, namespace1, "actions", { case (e: WhiskAction) => true case (_) => false @@ -292,7 +281,7 @@ class ViewTests case (e: WhiskTrigger) => true case (_) => false }) - getKindInNamespace(entityStore, namespace1, "rules", { + getKindInNamespaceWithDoc[WhiskRule](namespace1, "rules", { case (e: WhiskRule) => true case (_) => false }) @@ -308,9 +297,8 @@ class ViewTests case (e: WhiskAction) => (e.name == actionName) case (_) => false }) - getEntitiesInNamespace(namespace1) - getAllInNamespace(entityStore, namespace2) + getEntitiesInNamespace(entityStore, namespace2) getKindInNamespace(entityStore, namespace2, "actions", { case (e: WhiskAction) => true case (_) => false @@ -319,7 +307,7 @@ class ViewTests case (e: WhiskTrigger) => true case (_) => false }) - getKindInNamespace(entityStore, namespace2, "rules", { + getKindInNamespaceWithDoc[WhiskRule](namespace2, "rules", { case (e: WhiskRule) => true case (_) => false }) @@ -331,7 +319,6 @@ class ViewTests case (e: WhiskAction) => true case (_) => false }) - getEntitiesInNamespace(namespace2) } it should "query whisk view by namespace, collection and entity name in whisk activations db" in { diff --git a/tools/db/README.md b/tools/db/README.md index 9de0ac4..10ad2cf 100644 --- a/tools/db/README.md +++ b/tools/db/README.md @@ -8,7 +8,6 @@ If you are using your own installation of CouchDB, make a note of the host, port * the username must have administrative rights * the CouchDB instance must be accessible over `http` or `https` (the latter requires a valid certificate) - * the CouchDB instance must set `reduce_limit` on views to `false` (see [this](../../ansible/README.md#persistent-couchdb) for how to do this via REST) ### Using an ephemeral CouchDB container -- To stop receiving notification emails like this one, please contact ['"commits@openwhisk.apache.org" '].