Return-Path: X-Original-To: apmail-ambari-user-archive@www.apache.org Delivered-To: apmail-ambari-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 77D2E187F0 for ; Fri, 31 Jul 2015 05:40:06 +0000 (UTC) Received: (qmail 81216 invoked by uid 500); 31 Jul 2015 05:40:06 -0000 Delivered-To: apmail-ambari-user-archive@ambari.apache.org Received: (qmail 81185 invoked by uid 500); 31 Jul 2015 05:40:06 -0000 Mailing-List: contact user-help@ambari.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ambari.apache.org Delivered-To: mailing list user@ambari.apache.org Received: (qmail 81174 invoked by uid 99); 31 Jul 2015 05:40:06 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 31 Jul 2015 05:40:06 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id CD7ABC0925 for ; Fri, 31 Jul 2015 05:40:05 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.42 X-Spam-Level: *** X-Spam-Status: No, score=3.42 tagged_above=-999 required=6.31 tests=[AC_DIV_BONANZA=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HK_RANDOM_ENVFROM=0.626, HK_RANDOM_FROM=0.999, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H2=-1.108, URIBL_BLOCKED=0.001, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 1Lmh65lCdFS6 for ; Fri, 31 Jul 2015 05:39:56 +0000 (UTC) Received: from mail-io0-f173.google.com (mail-io0-f173.google.com [209.85.223.173]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id 4988F20F20 for ; Fri, 31 Jul 2015 05:39:56 +0000 (UTC) Received: by iodd187 with SMTP id d187so75203324iod.2 for ; Thu, 30 Jul 2015 22:39:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Abdn9r78MXhBllKmR8RTFsu2DEFvDQs9Ykrwd5V80Us=; b=nXudN/GkyJKUi2J5jqsKR1kykOMHXCnVmTRKm8qHz8aBKrsewXszQN+E1xZMFxlw6l mAkcciqrKaG+V/aiqWclkw0qwUKQmflGT6C/1GN0FQ+TumM5GmdPLB+wIVXGsDT1if3s 3OlE5Kd7YUR6pFbigNnS+xtX2bwVmNpzj9Ao938ykYUyMoIvt7WvdEgBIkeVEGUsZI8N KKXtrvE3Iz8fLJkHk4KhqE2KqaSDlKO672IyfsfSQ0msSPuXs4xbgp4fotg40Dv4l/4f WzraUbkvxMNPfpGZ30Qjr5bsQ3X6LeZ8wCSZROtStwPwGEoHvSInbya1rylw65QAJjwh /TjQ== MIME-Version: 1.0 X-Received: by 10.107.128.28 with SMTP id b28mr1585987iod.84.1438321150747; Thu, 30 Jul 2015 22:39:10 -0700 (PDT) Received: by 10.36.94.1 with HTTP; Thu, 30 Jul 2015 22:39:10 -0700 (PDT) In-Reply-To: References: Date: Fri, 31 Jul 2015 07:39:10 +0200 Message-ID: Subject: Re: Ambari 2.1 / HDP 2.3 & dfs.http.policy = HTTPS_ONLY issue From: philippe lanckvrind To: Alejandro Fernandez Cc: "user@ambari.apache.org" , Jing Zhao Content-Type: multipart/alternative; boundary=001a113f9a62436b46051c253d11 --001a113f9a62436b46051c253d11 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable thank you for your answer Alejandro. I'll give more detail about my concern because even with the SASL activated, it remains the same. Also, I strongly suspect that a part of the issue is comming from abari-server. Conrete situation: All the component are stop through ambari ui I just add the parameter dfs.http.policy to HTTPS_ONLY, I save the configuration and then I directly receive an error message from Ambari UI error 400 related to ressource component. The same goes with YARN when I set dfs.http.policy to HTTPS_ONLY. And I repeat, Saving the configuration from the ambari ui before restarting the HDFS through the ui. If you wish, I can create a youtube video and show the steps. Also, when I set dfs.http.policy to HTTP_AND_HTTPS, every thing goes perfect, no error from ambari ui and https namenode is accessible. Hope it helps. Best 2015-07-30 19:32 GMT+02:00 Alejandro Fernandez = : > +Jing > > Hi Philippe, > > When setting dfs.hdfs.policy to HTTPS_ONLY, you typically have to enable > SSL on your cluster. > > http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_hdfs_admin_too= ls/content/configuring_datanode_sasl.html > > > This question is better suited for the HDFS team. > > Thanks, > Alejandro > > From: philippe lanckvrind > Reply-To: "user@ambari.apache.org" > Date: Wednesday, July 29, 2015 at 11:25 PM > To: "user@ambari.apache.org" > Subject: Fwd: Ambari 2.1 / HDP 2.3 & dfs.http.policy =3D HTTPS_ONLY issue > > Dear all, > > I've noticed a strange issue with ambari 2.1 when I set the parameter > dfs.hdfs.policy to HTTPS_ONLY. > > issue 1: > When the parameter is set, the web-ui popup error codes : > 500 status code received on GET method for API: > /api/v1/clusters/HDP_CLUSTER/services/HDFS/components/NAMENODE?fields=3Dm= etrics/dfs/FSNamesystem/CorruptBlocks,metrics/dfs/FSNamesystem/UnderReplica= tedBlocks&format=3Dnull_padding > > > When i continu, short after I can't access the the dashboard anymore or > any other services on it with the related error: > 500 status code received on GET method for API: > /api/v1/clusters/HDP_CLUSTER/components/?ServiceComponentInfo/component_n= ame=3DFLUME_HANDLER|ServiceComponentInfo/component_name=3DAPP_TIMELINE_SERV= ER|ServiceComponentInfo/category=3DMASTER&fields=3DServiceComponentInfo/ser= vice_name,host_components/HostRoles/host_name,host_components/HostRoles/sta= te,host_components/HostRoles/maintenance_state,host_components/HostRoles/st= ale_configs,host_components/HostRoles/ha_state,host_components/HostRoles/de= sired_admin_state,host_components/metrics/jvm/memHeapUsedM,host_components/= metrics/jvm/HeapMemoryMax,host_components/metrics/jvm/HeapMemoryUsed,host_c= omponents/metrics/jvm/memHeapCommittedM,host_components/metrics/mapred/jobt= racker/trackers_decommissioned,host_components/metrics/cpu/cpu_wio,host_com= ponents/metrics/rpc/RpcQueueTime_avg_time,host_components/metrics/dfs/FSNam= esystem/*,host_components/metrics/dfs/namenode/Version,host_components/metr= ics/dfs/namenode/LiveNodes,host_components/metrics/dfs/namenode/DeadNodes,h= ost_components/metrics/dfs/namenode/DecomNodes,host_components/metrics/dfs/= namenode/TotalFiles,host_components/metrics/dfs/namenode/UpgradeFinalized,h= ost_components/metrics/dfs/namenode/Safemode,host_components/metrics/runtim= e/StartTime,host_components/processes/HostComponentProcess,host_components/= metrics/hbase/master/IsActiveMaster,host_components/metrics/hbase/master/Ma= sterStartTime,host_components/metrics/hbase/master/MasterActiveTime,host_co= mponents/metrics/hbase/master/AverageLoad,host_components/metrics/master/As= signmentManger/ritCount,metrics/api/v1/cluster/summary,metrics/api/v1/topol= ogy/summary,host_components/metrics/yarn/Queue,host_components/metrics/yarn= /ClusterMetrics/NumActiveNMs,host_components/metrics/yarn/ClusterMetrics/Nu= mLostNMs,host_components/metrics/yarn/ClusterMetrics/NumUnhealthyNMs,host_c= omponents/metrics/yarn/ClusterMetrics/NumRebootedNMs,host_components/metric= s/yarn/ClusterMetrics/NumDecommissionedNMs&minimal_response=3Dtrue > > > Issue 2 : > before losing the control of Ambari, after setting dfs.hdfs.policy to > HTTPS_ONLY, When I try to start HDFS, I receveive the folowing error: > Connection failed to http://*******:50090 (Execution of 'curl -k > --negotiate -u : -b > /var/lib/ambari-agent/data/tmp/cookies/275cbc46-ffae-4524-bc29-6896c0b565= e5 > -c > /var/lib/ambari-agent/data/tmp/cookies/275cbc46-ffae-4524-bc29-6896c0b565= e5 > -w '%{http_code}' http://*******t:50090 --connect-timeout 5 --max-time 7 > -o /dev/null' returned 7. curl: (7) couldn't connect to host > 000) > > > > Configuration testing: > Configuration 1 > > - Docker v1.7 > - HP 2.3 > - Ambari 2.1 Hortonworks repo > - centos 6.6 > > Configuration 2 > > - virtual box v 4.3.10 > - HDP 2.3 > - Ambari 2.1 Hortonworks repo > - Centos 6.6 server > > I also noticed tat I can without a problem manually start the hdfs > component with the SSL activated. > > > In advance, tank you for your feedback > > > --001a113f9a62436b46051c253d11 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
tha= nk you for your answer Alejandro.

I'll give more detail ab= out my concern because even with the SASL activated, it remains the same. <= br>
Also, I strongly suspect that a part of the issue is comming from = abari-server.
Conrete situation:
All the component are st= op through ambari ui
I just add the parameter dfs.http.policy to HTTPS_ONLY, I save the=20 configuration and then I directly receive an error message from Ambari=20 UI error 400 related to ressource component. The same goes with YARN=20 when I set dfs.http.policy to HTTPS_ONLY.
And I repeat, Saving the= configuration from the ambari ui before restarting the HDFS through the ui= .

If you wish, I can create a youtube video and show the= steps.

Also, when I set dfs.http.policy to HTTP_AND_HTTPS, every thing goes perfect, no error from ambari ui and https namenode is accessible.

Ho= pe it helps.

Best

2015-07-30 19:32 GMT+02:00 Alejandro Fernandez <afernandez@hortonworks.com>:
+Jing

Hi Philippe,

When setting dfs.hdfs.policy to HTTPS_ONLY, you typically have to enab= le SSL on your cluster.

This question is better suited for the HDFS team.

Thanks,
Alejandro

From: philippe lanckvrind <lanckvrind.ph= ilippe@gmail.com>
Reply-To: "user@ambari.apache.org" &= lt;user@ambari.= apache.org>
Date: Wednesday, July 29, 2015 at 1= 1:25 PM
To: "user@ambari.apache.org" <user@ambari.apache= .org>
Subject: Fwd: Ambari 2.1 / HDP 2.3 = & dfs.http.policy =3D HTTPS_ONLY issue

Dear all,

I've noticed a strange issue with ambari 2.1 when I set the parameter d= fs.hdfs.policy to HTTPS_ONLY.

issue 1:
When the parameter is set, the web-ui popup error codes :
500 status code received on GET method for API: /api/v1/clusters/HDP_CLUSTE= R/services/HDFS/components/NAMENODE?fields=3Dmetrics/dfs/FSNamesystem/Corru= ptBlocks,metrics/dfs/FSNamesystem/UnderReplicatedBlocks&format=3Dnull_p= adding


When i continu, short after I can't access the the dashboard anymo= re or any other services on it with the related error:
500 status code received on GET method for API: /api/v1/clusters/HDP_CLUSTE= R/components/?ServiceComponentInfo/component_name=3DFLUME_HANDLER|ServiceCo= mponentInfo/component_name=3DAPP_TIMELINE_SERVER|ServiceComponentInfo/categ= ory=3DMASTER&fields=3DServiceComponentInfo/service_name,host_components= /HostRoles/host_name,host_components/HostRoles/state,host_components/HostRo= les/maintenance_state,host_components/HostRoles/stale_configs,host_componen= ts/HostRoles/ha_state,host_components/HostRoles/desired_admin_state,host_co= mponents/metrics/jvm/memHeapUsedM,host_components/metrics/jvm/HeapMemoryMax= ,host_components/metrics/jvm/HeapMemoryUsed,host_components/metrics/jvm/mem= HeapCommittedM,host_components/metrics/mapred/jobtracker/trackers_decommiss= ioned,host_components/metrics/cpu/cpu_wio,host_components/metrics/rpc/RpcQu= eueTime_avg_time,host_components/metrics/dfs/FSNamesystem/*,host_components= /metrics/dfs/namenode/Version,host_components/metrics/dfs/namenode/LiveNode= s,host_components/metrics/dfs/namenode/DeadNodes,host_components/metrics/df= s/namenode/DecomNodes,host_components/metrics/dfs/namenode/TotalFiles,host_= components/metrics/dfs/namenode/UpgradeFinalized,host_components/metrics/df= s/namenode/Safemode,host_components/metrics/runtime/StartTime,host_componen= ts/processes/HostComponentProcess,host_components/metrics/hbase/master/IsAc= tiveMaster,host_components/metrics/hbase/master/MasterStartTime,host_compon= ents/metrics/hbase/master/MasterActiveTime,host_components/metrics/hbase/ma= ster/AverageLoad,host_components/metrics/master/AssignmentManger/ritCount,m= etrics/api/v1/cluster/summary,metrics/api/v1/topology/summary,host_componen= ts/metrics/yarn/Queue,host_components/metrics/yarn/ClusterMetrics/NumActive= NMs,host_components/metrics/yarn/ClusterMetrics/NumLostNMs,host_components/= metrics/yarn/ClusterMetrics/NumUnhealthyNMs,host_components/metrics/yarn/Cl= usterMetrics/NumRebootedNMs,host_components/metrics/yarn/ClusterMetrics/Num= DecommissionedNMs&minimal_response=3Dtrue


Issue 2 :
before losing the control of Ambari, after setting dfs.hdfs.policy to = HTTPS_ONLY, When I try to start HDFS, I receveive the folowing error:
Connection failed to htt= p://*******:50090 (Execution of 'curl -k --negotiate -u : -b /var/l= ib/ambari-agent/data/tmp/cookies/275cbc46-ffae-4524-bc29-6896c0b565e5 -c /v= ar/lib/ambari-agent/data/tmp/cookies/275cbc46-ffae-4524-bc29-6896c0b565e5 -w '%{http_code}' http://*******t:50090 --connect-timeout 5 --max-time 7 -o /dev/null= ' returned 7. curl: (7) couldn't connect to host
000)



Configuration testing:
Configuration 1
  • Docker v1.7
  • HP 2.3
  • Ambari 2.1 Hortonworks repo
    • centos 6.6

Configuration 2

  • virtual box v 4.3.10
  • HDP 2.3
  • Ambari 2.1 Hortonworks rep= o
  • Centos 6.6 server

I also noticed tat I can without a problem manually start the hdfs compo= nent with the SSL activated.


In advance, tank you for your feedback

--001a113f9a62436b46051c253d11--