Return-Path: X-Original-To: apmail-ambari-dev-archive@www.apache.org Delivered-To: apmail-ambari-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C196717A55 for ; Wed, 29 Apr 2015 16:00:36 +0000 (UTC) Received: (qmail 53504 invoked by uid 500); 29 Apr 2015 16:00:36 -0000 Delivered-To: apmail-ambari-dev-archive@ambari.apache.org Received: (qmail 53471 invoked by uid 500); 29 Apr 2015 16:00:36 -0000 Mailing-List: contact dev-help@ambari.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ambari.apache.org Delivered-To: mailing list dev@ambari.apache.org Received: (qmail 53450 invoked by uid 99); 29 Apr 2015 16:00:36 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Apr 2015 16:00:36 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id D51AE1DBA95; Wed, 29 Apr 2015 16:00:36 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============6133974331105390740==" MIME-Version: 1.0 Subject: Re: Review Request 33642: Add the ability to obtain details about required Kerberos identities From: "Robert Levas" To: "Emil Anca" , "John Speidel" , "Robert Nettleton" , "Tom Beerbower" Cc: "Robert Levas" , "Ambari" Date: Wed, 29 Apr 2015 16:00:36 -0000 Message-ID: <20150429160036.17829.40543@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org/ Auto-Submitted: auto-generated Sender: "Robert Levas" X-ReviewGroup: Ambari X-ReviewRequest-URL: https://reviews.apache.org/r/33642/ X-Sender: "Robert Levas" References: <20150428224459.26756.44472@reviews.apache.org> In-Reply-To: <20150428224459.26756.44472@reviews.apache.org> Reply-To: "Robert Levas" X-ReviewRequest-Repository: ambari --===============6133974331105390740== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33642/ ----------------------------------------------------------- (Updated April 29, 2015, noon) Review request for Ambari, Emil Anca, John Speidel, Robert Nettleton, and Tom Beerbower. Changes ------- Addressed reviewer concerns Bugs: AMBARI-10576 https://issues.apache.org/jira/browse/AMBARI-10576 Repository: ambari Description ------- Add the ability to obtain details about required Kerberos identities for the cluster. These details should be obtained using a REST API call formatted as a JSON structure. Resulting JSON block per Kerberos identity: ``` "KerberosIdentity" : { "cluster_name" : "c1", "description" : "/spnego", "host_name" : "host1", "keytab_file_group" : "hadoop", "keytab_file_group_access" : "r", "keytab_file_installed" : "true", "keytab_file_mode" : "440", "keytab_file_owner" : "root", "keytab_file_owner_access" : "r", "keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab", "principal_local_username" : null, "principal_name" : "HTTP/host1@EXAMPLE.COM", "principal_type" : "SERVICE" } ``` The data will be converted into CSV-formatted data similar to the file exported from Ambari 1.7. **Solution** The following API calls are to be used to obtain the data: *GET /api/v1/clusters/c1/hosts?fields=kerberos_identities/** ``` { "href" : "http://ambari:8080/api/v1/clusters/c1/hosts?fields=kerberos_identities/*", "items" : [ { "href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1", "Hosts" : { "cluster_name" : "c1", "host_name" : "host1" }, "kerberos_identities" : [ { "href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1/kerberos_identities/HTTP%2Fhost1%40EXAMPLE.COM", "KerberosIdentity" : { "cluster_name" : "c1", "description" : "/spnego", "host_name" : "host1", "keytab_file_group" : "hadoop", "keytab_file_group_access" : "r", "keytab_file_installed" : "true", "keytab_file_mode" : "440", "keytab_file_owner" : "root", "keytab_file_owner_access" : "r", "keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab", "principal_local_username" : null, "principal_name" : "HTTP/host1@EXAMPLE.COM", "principal_type" : "SERVICE" } }, { "href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1/kerberos_identities/ambari-qa%40EXAMPLE.COM", "KerberosIdentity" : { "cluster_name" : "c1", "description" : "/smokeuser", "host_name" : "host1", "keytab_file_group" : "hadoop", "keytab_file_group_access" : "r", "keytab_file_installed" : "true", "keytab_file_mode" : "440", "keytab_file_owner" : "ambari-qa", "keytab_file_owner_access" : "r", "keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab", "principal_local_username" : "ambari-qa", "principal_name" : "ambari-qa@EXAMPLE.COM", "principal_type" : "USER" } }, ... ] }, ... ] } ``` *GET /api/v1/clusters/c1/hosts?fields=kerberos_identities/*&format=csv* ``` host,description,principal name,principal type,local username,keytab file path,keytab file owner,keytab file owner access,keytab file group,keytab file group access,keytab file mode,keytab file installed host1,/spnego,HTTP/host1@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true host1,/smokeuser,ambari-qa@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true host1,datanode_dn,dn/host1@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true host1,/hdfs,hdfs@EXAMPLE.COM,USER,hdfs,/etc/security/keytabs/hdfs.headless.keytab,hdfs,r,hadoop,r,440,true host1,nodemanager_nm,nm/host1@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/nm.service.keytab,yarn,r,hadoop,,400,true host1,namenode_nn,nn/host1@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true host1,zookeeper_zk,zookeeper/host1@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true host2,/spnego,HTTP/host2@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true host2,/smokeuser,ambari-qa@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true host2,datanode_dn,dn/host2@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true host2,/hdfs,hdfs@EXAMPLE.COM,USER,hdfs,/etc/security/keytabs/hdfs.headless.keytab,hdfs,r,hadoop,r,440,true host2,history_server_jhs,jhs/host2@EXAMPLE.COM,SERVICE,mapred,/etc/security/keytabs/jhs.service.keytab,mapred,r,hadoop,,400,true host2,nodemanager_nm,nm/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/nm.service.keytab,yarn,r,hadoop,,400,true host2,secondary_namenode_nn,nn/host2@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true host2,resource_manager_rm,rm/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/rm.service.keytab,yarn,r,hadoop,,400,true host2,app_timeline_server_yarn,yarn/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/yarn.service.keytab,yarn,r,hadoop,,400,true host2,zookeeper_zk,zookeeper/host2@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true ... ``` *GET /api/v1/clusters/c1/kerberos_identities?fields=** ``` { "href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities?fields=*", "items" : [ { "href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities/HTTP%2Fhost1%40EXAMPLE.COM", "KerberosIdentity" : { "cluster_name" : "c1", "description" : "/spnego", "host_name" : "host1", "keytab_file_group" : "hadoop", "keytab_file_group_access" : "r", "keytab_file_installed" : "true", "keytab_file_mode" : "440", "keytab_file_owner" : "root", "keytab_file_owner_access" : "r", "keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab", "principal_local_username" : null, "principal_name" : "HTTP/host1@EXAMPLE.COM", "principal_type" : "SERVICE" } }, { "href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities/ambari-qa%40EXAMPLE.COM", "KerberosIdentity" : { "cluster_name" : "c1", "description" : "/smokeuser", "host_name" : "host1", "keytab_file_group" : "hadoop", "keytab_file_group_access" : "r", "keytab_file_installed" : "true", "keytab_file_mode" : "440", "keytab_file_owner" : "ambari-qa", "keytab_file_owner_access" : "r", "keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab", "principal_local_username" : "ambari-qa", "principal_name" : "ambari-qa@EXAMPLE.COM", "principal_type" : "USER" } }, ... ] } ``` Diffs (updated) ----- ambari-server/src/main/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRenderer.java 5c84d4c ambari-server/src/main/java/org/apache/ambari/server/api/query/render/HostKerberosIdentityCsvRenderer.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/api/resources/BaseResourceDefinition.java 02342a8 ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostComponentResourceDefinition.java 6dc9e2d ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostKerberosIdentityResourceDefinition.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostResourceDefinition.java 380e751 ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java 776f1f4 ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseService.java 3afc23d ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java be40bc4 ambari-server/src/main/java/org/apache/ambari/server/api/services/HostKerberosIdentityService.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/api/services/HostService.java aaf3007 ambari-server/src/main/java/org/apache/ambari/server/api/services/ResultPostProcessorImpl.java 61afee2 ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/CsvSerializer.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/JsonSerializer.java 7f57f7f ambari-server/src/main/java/org/apache/ambari/server/api/util/TreeNode.java 796d64f ambari-server/src/main/java/org/apache/ambari/server/api/util/TreeNodeImpl.java 1739b88 ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java d6da1eb ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java bdf94a5 ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java f7eb2d9 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java 210227e ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProvider.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java 30bac9e ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java 2e5a27d ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java 79537d4 ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java 2c0c90a ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java d33adcd ambari-server/src/test/java/org/apache/ambari/server/api/query/render/MinimalRendererTest.java 37bf33c ambari-server/src/test/java/org/apache/ambari/server/api/resources/BaseResourceDefinitionTest.java 3f64d9a ambari-server/src/test/java/org/apache/ambari/server/api/resources/HostKerberosIdentityResourceDefinitionTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/api/resources/HostResourceDefinitionTest.java b3851eb ambari-server/src/test/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImplTest.java 0daffce ambari-server/src/test/java/org/apache/ambari/server/api/services/HostKerberosIdentityServiceTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/api/services/serializers/CsvSerializerTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java 47f051d ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProviderTest.java PRE-CREATION Diff: https://reviews.apache.org/r/33642/diff/ Testing ------- Manually tested in test cluster **Local unit test results:** [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 44:39.244s [INFO] Finished at: Tue Apr 28 18:29:01 EDT 2015 [INFO] Final Memory: 60M/1186M [INFO] ------------------------------------------------------------------------ **Jenkins unit test results: PENDING** Thanks, Robert Levas --===============6133974331105390740==--