Return-Path: X-Original-To: apmail-accumulo-commits-archive@www.apache.org Delivered-To: apmail-accumulo-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A6B0119C32 for ; Tue, 12 Apr 2016 13:58:19 +0000 (UTC) Received: (qmail 39092 invoked by uid 500); 12 Apr 2016 13:58:19 -0000 Delivered-To: apmail-accumulo-commits-archive@accumulo.apache.org Received: (qmail 39015 invoked by uid 500); 12 Apr 2016 13:58:19 -0000 Mailing-List: contact commits-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@accumulo.apache.org Delivered-To: mailing list commits@accumulo.apache.org Received: (qmail 37963 invoked by uid 99); 12 Apr 2016 13:58:18 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 12 Apr 2016 13:58:18 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id BF926E01E2; Tue, 12 Apr 2016 13:58:18 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: dlmarion@apache.org To: commits@accumulo.apache.org Date: Tue, 12 Apr 2016 13:58:42 -0000 Message-Id: <3e2da487f5294531ba2727dc04395ee2@git.apache.org> In-Reply-To: <28ce56b801f84481baae935b546b14cd@git.apache.org> References: <28ce56b801f84481baae935b546b14cd@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [25/39] accumulo git commit: ACCUMULO-4147 Add Accumulo Namespace Operations to Proxy ACCUMULO-4147 Add Accumulo Namespace Operations to Proxy Squashed commit of the following: commit a97782a97bddd0495fc493c7484907ff29b9e023 Author: Tristen Georgiou Date: Tue Mar 29 22:29:55 2016 -0700 Added in missing calls to Assert.fail() to ensure that expected exceptions are thrown and removed a function call that doesn't actually throw the exception. commit 3b9cff9219cecc93392e32741177531ed7d1dbe5 Merge: 9d2dc05 1d49030 Author: Tristen Georgiou Date: Fri Mar 25 14:03:48 2016 -0700 Merge remote-tracking branch 'upstream/master' commit 9d2dc05dde1a3896af1ac64e0e2bea67d784ad89 Author: Tristen Georgiou Date: Fri Mar 25 14:01:52 2016 -0700 Added in integration tests for the proxy namespace operations. commit 9cb8bc88fbc205ca841bf59159c5060fede43990 Merge: eacedaa c4d6eee Author: Tristen Georgiou Date: Thu Feb 25 17:05:55 2016 -0800 Merge remote-tracking branch 'upstream/master' commit eacedaaccd9d79d8a1f2303a2aef15599acfcb10 Author: Tristen Georgiou Date: Thu Feb 25 17:03:17 2016 -0800 Added in IT tests for namespace operations and added in a namespace example file under the python examples . commit 8dc5ab95d97f5c825222ab2ed52afaf2553e05a2 Merge: e149233 2651ba8 Author: Tristen Georgiou Date: Thu Feb 25 11:50:34 2016 -0800 Merge remote-tracking branch 'upstream/master' commit e149233c5340329410d83270a523fdb6a8a57cb9 Author: Tristen Georgiou Date: Sat Feb 20 12:13:58 2016 -0800 Formatting update. commit cc254d7857adc696413afd6ff0d945f955af5853 Author: Tristen Georgiou Date: Sat Feb 20 10:55:21 2016 -0800 Added in namespace security operation permission functions. commit 7e3c4d40c7ec6df43730e142b5dc87fa498caebe Author: Tristen Georgiou Date: Fri Feb 19 16:10:21 2016 -0800 Initial implementation for proxy namespace operations. Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/8839fe51 Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/8839fe51 Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/8839fe51 Branch: refs/heads/ACCUMULO-4173 Commit: 8839fe5116337bbbf1e9b81109d0e9127fef7715 Parents: ada31a9 Author: Tristen Georgiou Authored: Mon Apr 4 12:19:41 2016 -0400 Committer: Keith Turner Committed: Mon Apr 4 13:47:10 2016 -0400 ---------------------------------------------------------------------- proxy/examples/python/TestNamespace.py | 172 + proxy/src/main/cpp/AccumuloProxy.cpp | 33610 ++++--- proxy/src/main/cpp/AccumuloProxy.h | 5162 +- .../main/cpp/AccumuloProxy_server.skeleton.cpp | 115 + proxy/src/main/cpp/proxy_types.cpp | 315 + proxy/src/main/cpp/proxy_types.h | 166 + .../org/apache/accumulo/proxy/ProxyServer.java | 289 + .../accumulo/proxy/thrift/AccumuloProxy.java | 88369 +++++++++++------ .../proxy/thrift/NamespaceExistsException.java | 414 + .../thrift/NamespaceNotEmptyException.java | 414 + .../thrift/NamespaceNotFoundException.java | 414 + .../proxy/thrift/NamespacePermission.java | 82 + proxy/src/main/python/AccumuloProxy-remote | 161 + proxy/src/main/python/AccumuloProxy.py | 15767 ++- proxy/src/main/python/ttypes.py | 239 + proxy/src/main/ruby/accumulo_proxy.rb | 1740 +- proxy/src/main/ruby/proxy_types.rb | 83 + proxy/src/main/thrift/proxy.thrift | 61 +- .../accumulo/test/proxy/SimpleProxyBase.java | 407 +- .../proxy/TestProxyNamespaceOperations.java | 165 + .../test/proxy/TestProxySecurityOperations.java | 13 + 21 files changed, 102216 insertions(+), 45942 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/accumulo/blob/8839fe51/proxy/examples/python/TestNamespace.py ---------------------------------------------------------------------- diff --git a/proxy/examples/python/TestNamespace.py b/proxy/examples/python/TestNamespace.py new file mode 100644 index 0000000..e7d2377 --- /dev/null +++ b/proxy/examples/python/TestNamespace.py @@ -0,0 +1,172 @@ +#! /usr/bin/env python + +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from thrift.protocol import TCompactProtocol +from thrift.transport import TSocket, TTransport + +from proxy import AccumuloProxy +from proxy.ttypes import NamespacePermission, IteratorSetting, IteratorScope, AccumuloException + + +def main(): + transport = TSocket.TSocket('localhost', 42424) + transport = TTransport.TFramedTransport(transport) + protocol = TCompactProtocol.TCompactProtocol(transport) + client = AccumuloProxy.Client(protocol) + transport.open() + login = client.login('root', {'password': 'password'}) + + client.createLocalUser(login, 'user1', 'password1') + + print client.listNamespaces(login) + + # create a namespace and give the user1 all permissions + print 'creating namespace testing' + client.createNamespace(login, 'testing') + assert client.namespaceExists(login, 'testing') + print client.listNamespaces(login) + + print 'testing namespace renaming' + client.renameNamespace(login, 'testing', 'testing2') + assert not client.namespaceExists(login, 'testing') + assert client.namespaceExists(login, 'testing2') + client.renameNamespace(login, 'testing2', 'testing') + assert not client.namespaceExists(login, 'testing2') + assert client.namespaceExists(login, 'testing') + + print 'granting all namespace permissions to user1' + for k, v in NamespacePermission._VALUES_TO_NAMES.iteritems(): + client.grantNamespacePermission(login, 'user1', 'testing', k) + + # make sure the last operation worked + for k, v in NamespacePermission._VALUES_TO_NAMES.iteritems(): + assert client.hasNamespacePermission(login, 'user1', 'testing', k), \ + 'user1 does\'nt have namespace permission %s' % v + + print 'default namespace: ' + client.defaultNamespace() + print 'system namespace: ' + client.systemNamespace() + + # grab the namespace properties + print 'retrieving namespace properties' + props = client.getNamespaceProperties(login, 'testing') + assert props and props['table.compaction.major.ratio'] == '3' + + # update a property and verify it is good + print 'setting namespace property table.compaction.major.ratio = 4' + client.setNamespaceProperty(login, 'testing', 'table.compaction.major.ratio', '4') + props = client.getNamespaceProperties(login, 'testing') + assert props and props['table.compaction.major.ratio'] == '4' + + print 'retrieving namespace ID map' + nsids = client.namespaceIdMap(login) + assert nsids and 'accumulo' in nsids + + print 'attaching debug iterator to namespace testing' + setting = IteratorSetting(priority=40, name='DebugTheThings', + iteratorClass='org.apache.accumulo.core.iterators.DebugIterator', properties={}) + client.attachNamespaceIterator(login, 'testing', setting, [IteratorScope.SCAN]) + setting = client.getNamespaceIteratorSetting(login, 'testing', 'DebugTheThings', IteratorScope.SCAN) + assert setting and setting.name == 'DebugTheThings' + + # make sure the iterator is in the list + iters = client.listNamespaceIterators(login, 'testing') + found = False + for name, scopes in iters.iteritems(): + if name == 'DebugTheThings': + found = True + break + assert found + + print 'checking for iterator conflicts' + + # this next statment should be fine since we are on a different scope + client.checkNamespaceIteratorConflicts(login, 'testing', setting, [IteratorScope.MINC]) + + # this time it should throw an exception since we have already added the iterator with this scope + try: + client.checkNamespaceIteratorConflicts(login, 'testing', setting, [IteratorScope.SCAN, IteratorScope.MINC]) + except AccumuloException: + pass + else: + assert False, 'There should have been a namespace iterator conflict!' + + print 'removing debug iterator from namespace testing' + client.removeNamespaceIterator(login, 'testing', 'DebugTheThings', [IteratorScope.SCAN]) + + # make sure the iterator is NOT in the list anymore + iters = client.listNamespaceIterators(login, 'testing') + found = False + for name, scopes in iters.iteritems(): + if name == 'DebugTheThings': + found = True + break + assert not found + + print 'adding max mutation size namespace constraint' + constraintid = client.addNamespaceConstraint(login, 'testing', + 'org.apache.accumulo.examples.simple.constraints.MaxMutationSize') + + print 'make sure constraint was added' + constraints = client.listNamespaceConstraints(login, 'testing') + found = False + for name, cid in constraints.iteritems(): + if cid == constraintid and name == 'org.apache.accumulo.examples.simple.constraints.MaxMutationSize': + found = True + break + assert found + + print 'remove max mutation size namespace constraint' + client.removeNamespaceConstraint(login, 'testing', constraintid) + + print 'make sure constraint was removed' + constraints = client.listNamespaceConstraints(login, 'testing') + found = False + for name, cid in constraints.iteritems(): + if cid == constraintid and name == 'org.apache.accumulo.examples.simple.constraints.MaxMutationSize': + found = True + break + assert not found + + print 'test a namespace class load of the VersioningIterator' + res = client.testNamespaceClassLoad(login, 'testing', 'org.apache.accumulo.core.iterators.user.VersioningIterator', + 'org.apache.accumulo.core.iterators.SortedKeyValueIterator') + assert res + + print 'test a bad namespace class load of the VersioningIterator' + res = client.testNamespaceClassLoad(login, 'testing', 'org.apache.accumulo.core.iterators.user.VersioningIterator', + 'dummy') + assert not res + + # revoke the permissions + print 'revoking namespace permissions for user1' + for k, v in NamespacePermission._VALUES_TO_NAMES.iteritems(): + client.revokeNamespacePermission(login, 'user1', 'testing', k) + + # make sure the last operation worked + for k, v in NamespacePermission._VALUES_TO_NAMES.iteritems(): + assert not client.hasNamespacePermission(login, 'user1', 'testing', k), \ + 'user1 does\'nt have namespace permission %s' % v + + print 'deleting namespace testing' + client.deleteNamespace(login, 'testing') + assert not client.namespaceExists(login, 'testing') + + print 'deleting user1' + client.dropLocalUser(login, 'user1') + +if __name__ == "__main__": + main()