Return-Path: X-Original-To: apmail-ambari-user-archive@www.apache.org Delivered-To: apmail-ambari-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 160A818DA6 for ; Mon, 18 May 2015 15:50:14 +0000 (UTC) Received: (qmail 69988 invoked by uid 500); 18 May 2015 15:50:13 -0000 Delivered-To: apmail-ambari-user-archive@ambari.apache.org Received: (qmail 69955 invoked by uid 500); 18 May 2015 15:50:13 -0000 Mailing-List: contact user-help@ambari.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ambari.apache.org Delivered-To: mailing list user@ambari.apache.org Received: (qmail 69944 invoked by uid 99); 18 May 2015 15:50:13 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 18 May 2015 15:50:13 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 4880DC5954 for ; Mon, 18 May 2015 15:50:13 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 4.003 X-Spam-Level: **** X-Spam-Status: No, score=4.003 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=3, KAM_LAZY_DOMAIN_SECURITY=1, UNPARSEABLE_RELAY=0.001, URIBL_BLOCKED=0.001, WEIRD_PORT=0.001] autolearn=disabled Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id stmDXduuwr7p for ; Mon, 18 May 2015 15:50:00 +0000 (UTC) Received: from smtppost.atos.net (smtppost.atos.net [193.56.114.164]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id 2421E43DD5 for ; Mon, 18 May 2015 15:49:57 +0000 (UTC) Received: from mail.awl.fr.atosorigin.com (serv-smtp-wse02.fr.atosworldline.com) by smarthost3.atos.net with smtp (TLS: TLSv1/SSLv3,128bits,RC4-MD5) id 517f_f287_bba0da90_ddb6_4a66_82cd_a9751f80d27f; Mon, 18 May 2015 17:48:50 +0200 Received: from frspx302.fr01.awl.atosorigin.net (10.24.253.187) by frspx402.priv.atos.fr (10.24.220.8) with Microsoft SMTP Server (TLS) id 8.3.348.2; Mon, 18 May 2015 17:44:45 +0200 Received: from frvdx202.fr01.awl.atosorigin.net (10.16.253.38) by frspx302.fr01.awl.atosorigin.net (10.24.253.187) with Microsoft SMTP Server (TLS) id 8.3.348.2; Mon, 18 May 2015 17:44:45 +0200 Received: from FRVDX100.fr01.awl.atosorigin.net ([10.16.253.46]) by frvdx202.fr01.awl.atosorigin.net ([10.16.253.38]) with mapi; Mon, 18 May 2015 17:44:45 +0200 From: =?iso-8859-1?Q?Chanel_Lo=EFc?= To: "user@ambari.apache.org" Date: Mon, 18 May 2015 17:44:43 +0200 Subject: RE: Restarting nodes to avoid HTTP 403 Thread-Topic: Restarting nodes to avoid HTTP 403 Thread-Index: AQHQjcVEuoT+mANYkEKRlS00ZFRw4Z2BusXQ Message-ID: <3D14BFE16DE13B4A941B2ECAC978CA8A7C09C89466@FRVDX100.fr01.awl.atosorigin.net> References: <3D14BFE16DE13B4A941B2ECAC978CA8A7C09C88A20@FRVDX100.fr01.awl.atosorigin.net> <73C9A048-13E0-432D-A08B-769A5C19E96C@hortonworks.com> <3D14BFE16DE13B4A941B2ECAC978CA8A7C09C88A45@FRVDX100.fr01.awl.atosorigin.net> <3D14BFE16DE13B4A941B2ECAC978CA8A7C09C88AAF@FRVDX100.fr01.awl.atosorigin.net> <1A5D4BA4-01A0-417A-B3E2-76A942AC6153@hortonworks.com> <3D14BFE16DE13B4A941B2ECAC978CA8A7C09C88BBD@FRVDX100.fr01.awl.atosorigin.net> <8B588726-79F0-42B5-8B61-8532B59160A9@hortonworks.com> <3D14BFE16DE13B4A941B2ECAC978CA8A7C09C88CA9@FRVDX100.fr01.awl.atosorigin.net> <35412791-C706-45B7-8489-DA7EF33507A1@hortonworks.com> <3D14BFE16DE13B4A941B2ECAC978CA8A7C09C89176@FRVDX100.fr01.awl.atosorigin.net> <1AA608CD-AC4E-474A-A2E3-0E59BBD2EE64@hortonworks.com> In-Reply-To: <1AA608CD-AC4E-474A-A2E3-0E59BBD2EE64@hortonworks.com> Accept-Language: fr-FR, en-US Content-Language: fr-FR X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: fr-FR, en-US Content-Type: multipart/alternative; boundary="_000_3D14BFE16DE13B4A941B2ECAC978CA8A7C09C89466FRVDX100fr01a_" MIME-Version: 1.0 --_000_3D14BFE16DE13B4A941B2ECAC978CA8A7C09C89466FRVDX100fr01a_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Jonathan, First of all sorry for my (very) late answer. I confirm that restarting the agents before installing the cluster fixes th= e issue. But what is slightly more complicated is the fact that there is no= t network trace. The command sudo tcpdump -i lo -l -s0 -w - tcp dst port 8042 | strings= & executed on the host returning 403 returns absolutely nothing. Does that help you ? If you want any additional information, feel free to ask. Regards, Lo=EFc De : Jonathan Hurley [mailto:jhurley@hortonworks.com] Envoy=E9 : mercredi 13 mai 2015 23:39 =C0 : user@ambari.apache.org Objet : Re: Restarting nodes to avoid HTTP 403 Well, I'm really baffled by this. Restarting the agents before installing a= cluster fixes the issue as well? So it seems like after agents are install= ed they are not able to make connections from python without getting a 403 = forbidden until they are restarted at least once. Is it possible to get a n= etwork trace of the agents when they encounter the 403 forbidden? That way = we can see the communication path between the agent and a particular endpoi= nt, like NameNode WebUI. On May 13, 2015, at 8:31 AM, Chanel Lo=EFc > wrote: Hi Jonathan, Additional information : the restart does not have to be necessarily AFTER = the cluster have been deployed using a Blueprint. Restarting the ambari-age= nt before using a blueprint to deploy the cluster make a perfectly clear cl= uster, without the 11 Warnings/Errors I mentioned in my previous emails. Hope this will help understand where the problem comes from. Have a nice day, Lo=EFc De : Jonathan Hurley [mailto:jhurley@hortonworks.com] Envoy=E9 : jeudi 7 mai 2015 18:53 =C0 : user@ambari.apache.org Objet : Re: Restarting nodes to avoid HTTP 403 All of this really seems to point to some kind of firewall/proxy issue on t= he agent hosts. From your agent named "vm-03cfbc97-f027-46fe-8e65-cb8c54edf= 377.frida.priv.atos.fr", could you try the following python code: >>> import urllib2 >>> response =3D urllib2.urlopen("http://vm-03cfbc97-f027-46fe-8e65-cb8c54e= df377.frida.priv.atos.fr:8042/ws/v1/node/info", timeout=3D10.0) >>> print(response.code) I'm really curious if executing that from your agent host results in a 200 = or a 403. On May 7, 2015, at 11:45 AM, Chanel Lo=EFc > wrote: Hi Jonathan, Here are the answers to your questions : 1) Which operating system are you running the agents on? --> CentOS6 2) Is there a linux proxy setup, such as "export http_proxy=3Dfoo" - curl d= oesn't respect this proxy setting but python's urllib2 does, which has caus= ed some issues before --> There is a proxy, but there is no such setup (I mean "echo $http_proxy"= returns nothing, and I do not remember setting Ambari properly in order to= access to the Internet via the proxy) 3) Which stack are you deploying? --> I am deploying HDP-2.2.4.2-2 Have a nice weekend, Lo=EFc De : Jonathan Hurley [mailto:jhurley@hortonworks.com] Envoy=E9 : jeudi 7 mai 2015 17:34 =C0 : user@ambari.apache.org Objet : Re: Restarting nodes to avoid HTTP 403 The logs indicate that the alerts are running correctly and are simply hitt= ing a 403. Normally, you might encounter this kind of problem from Python w= hen making web connections without specifying a known user agent header. Py= thon's default header sometimes causes issues since it's not standard. Howe= ver, that problem would continue to happen after you restarted the agents. = The fact that a simple agent restart completely fixes the issue is baffling= . I've certainly never seen this type of behavior before. I'd like to know a = few more details on your environment: 1) Which operating system are you running the agents on? 2) Is there a linux proxy setup, such as "export http_proxy=3Dfoo" - curl d= oesn't respect this proxy setting but python's urllib2 does, which has caus= ed some issues before 3) Which stack are you deploying? On May 7, 2015, at 4:05 AM, Chanel Lo=EFc > wrote: Hi Jonathan, You will find in attachment what I get from the API you gave me the URL. As far as the log file is concerned, here is an extract from the one on the= NameNode : WARNING 2015-05-07 10:01:47,221 base_alert.py:365 - [Alert][namenode_direct= ory_status] HA nameservice value is present but there are no aliases for {{= hdfs-site/dfs.ha.namenodes.{{ha-nameservice}}}} WARNING 2015-05-07 10:01:47,222 base_alert.py:365 - [Alert][datanode_health= _summary] HA nameservice value is present but there are no aliases for {{hd= fs-site/dfs.ha.namenodes.{{ha-nameservice}}}} INFO 2015-05-07 10:01:47,228 scheduler.py:509 - Running job "e89e2c29-1f2c-= 4bf5-a37c-7a6c5b43433a (trigger: interval[0:01:00], next run at: 2015-05-07= 10:02:47.211937)" (scheduled at 2015-05-07 10:01:47.211937) INFO 2015-05-07 10:01:47,229 scheduler.py:509 - Running job "ad0e0fd3-d5f3-= 471f-955e-ec10f53cdd5b (trigger: interval[0:01:00], next run at: 2015-05-07= 10:01:47.212773)" (scheduled at 2015-05-07 10:01:47.212773) WARNING 2015-05-07 10:01:47,230 base_alert.py:365 - [Alert][namenode_webui]= HA nameservice value is present but there are no aliases for {{hdfs-site/d= fs.ha.namenodes.{{ha-nameservice}}}} INFO 2015-05-07 10:01:47,230 scheduler.py:527 - Job "71988556-19e3-4871-92e= 0-e3c0a838df13 (trigger: interval[0:01:00], next run at: 2015-05-07 10:02:4= 7.206667)" executed successfully INFO 2015-05-07 10:01:47,238 scheduler.py:509 - Running job "2de4dc85-8993-= 4c68-9915-db25c4313d6e (trigger: interval[0:01:00], next run at: 2015-05-07= 10:02:47.213529)" (scheduled at 2015-05-07 10:01:47.213529) WARNING 2015-05-07 10:01:47,240 base_alert.py:140 - [Alert][datanode_health= _summary] Unable to execute alert. HTTP Error 403: Forbidden INFO 2015-05-07 10:01:47,242 scheduler.py:527 - Job "5bcf7b5d-73e7-4d59-bc0= a-f4772d4e3166 (trigger: interval[0:01:00], next run at: 2015-05-07 10:02:4= 7.208176)" executed successfully INFO 2015-05-07 10:01:47,241 scheduler.py:509 - Running job "ff1fc102-bdec-= 4858-baf5-8b60de4488e4 (trigger: interval[0:01:00], next run at: 2015-05-07= 10:01:47.214826)" (scheduled at 2015-05-07 10:01:47.214826) WARNING 2015-05-07 10:01:47,244 base_alert.py:365 - [Alert][yarn_resourcema= nager_webui] HA nameservice value is present but there are no aliases for {= {yarn-site/yarn.resourcemanager.ha.rm-ids}} INFO 2015-05-07 10:01:47,240 scheduler.py:527 - Job "72e5031e-4c2a-4236-b09= e-6a749100bc9a (trigger: interval[0:01:00], next run at: 2015-05-07 10:02:4= 7.203349)" executed successfully INFO 2015-05-07 10:01:47,250 scheduler.py:509 - Running job "a083e3c2-d4b2-= 430b-9b64-b60c783a06a5 (trigger: interval[0:01:00], next run at: 2015-05-07= 10:01:47.218706)" (scheduled at 2015-05-07 10:01:47.218706) INFO 2015-05-07 10:01:47,250 scheduler.py:509 - Running job "7e9470cb-043b-= 48a8-990a-0050f2c63311 (trigger: interval[0:01:00], next run at: 2015-05-07= 10:02:47.217519)" (scheduled at 2015-05-07 10:01:47.217519) WARNING 2015-05-07 10:01:47,253 base_alert.py:140 - [Alert][namenode_direct= ory_status] Unable to execute alert. HTTP Error 403: Forbidden So yes, I can see some output in the logs corresponding to the alerts I get= on the Ambari web app. Please tell me if you need any complementary information about my problem, Thanks, Lo=EFc De : Jonathan Hurley [mailto:jhurley@hortonworks.com] Envoy=E9 : mercredi 6 mai 2015 16:24 =C0 : user@ambari.apache.org Objet : Re: Restarting nodes to avoid HTTP 403 OK, so I think I have a clear picture of how you get to this situation. I'd= still like to know a few things: 1) When you have the warnings present in the web client, can you try the al= erts URL I posted below to see the actual alerts coming back from the API. = I'm mostly interesting in whether any come back in the first place, and wha= t the most recent timestamp was (indicating they are actually running and s= till reporting a warning status) 2) When the alerts are present in the web client, do you see any output in = the agent log file that I mentioned for alerts that start with [Alert]. On May 6, 2015, at 5:14 AM, Chanel Lo=EFc > wrote: I encounter the warnings and critical alerts when deploying the cluster. I = install Ambari server 2.0 on a VM and Ambari agent 2.0 on 4 others VMs. The= n, I give the Ambari server a blueprint coming from a functioning cluster t= o instantiate my new cluster and have a quickstart configuration. Then, when logging into the Ambari web application to ensure everything is = running properly, I have these alerts concerning the HTTP 403 errors return= ed by all host VMs but the one which only handles Ambari metrics. As I am not sure my explanations are quite understandable, do not hesitate = to tell me if something remains unclear. Thanks, De : Jonathan Hurley [mailto:jhurley@hortonworks.com] Envoy=E9 : mardi 5 mai 2015 20:38 =C0 : user@ambari.apache.org Objet : Re: Restarting nodes to avoid HTTP 403 If restarting the agents fixes everything, can you explain when you first e= ncounter the warnings? Is this only after a cluster deployment? You can als= o check to see if this is some sort of web client issue by issuing the foll= owing GET: http://server/api/v1/clusters//alerts?fields=3D*&Alert/state.in(CR= ITICAL,WARNING) This will show you alerts which are actually being returned from the agents= in a warning or critical state. For reference, you can also look in /var/log/ambari-agent/ambari-agent.log = to see if you see any alert issues. Most important messages are prefixed wi= th "[Alert]". On May 5, 2015, at 11:43 AM, Chanel Lo=EFc > wrote: Indeed, I did not gave so much information about my problem, sorry about th= at. Here is your answers : 1) Version of Ambari --> I'm using Ambari 2.0 2) Whether the environment is kerberized --> Not it's not. Kerberos security is not enabled on this cluster. 3) Are you running the Ambari agent as root, or another user --> I am running it as root 4) Any information from the ambari-agent.log file that might seem to indica= te a problem --> Here is another weird thing I did not mentioned : I could not find logs= referring to the problem. 5) You said that restarting the agents resolves the issue. Does it continue= to happen after restarting? If so, how long before new warnings start to s= how up --> Restarting the agent totally resolves the problem. It does not happen a= nymore, and everything run quite normally. De : Jonathan Hurley [mailto:jhurley@hortonworks.com] Envoy=E9 : mardi 5 mai 2015 17:36 =C0 : user@ambari.apache.org Objet : Re: Restarting nodes to avoid HTTP 403 Can you provide some more information on your environment, such as: 1) Version of Ambari 2) Whether the environment is kerberized 3) Are you running the Ambari agent as root, or another user. 4) Any information from the ambari-agent.log file that might seen to indica= te a problem 5) You said that restarting the agents resolves the issue. Does it continue= to happen after restarting? If so, how long before new warnings start to s= how up. I'm guessing you have a kerberized environment running Ambari 2.0. Ambari w= ill use curl in this case to attempt to make a connection to the web endpoi= nts. It uses the keytabs and principals defined on the alert definition. Fo= r NameNode, as an example, it would use: hdfs-site/dfs.web.authentication.kerberos.keytab hdfs-site/dfs.web.authentication.kerberos.principal You'll want to verify that these properties are correctly set and that the = keytab file is accessible to the agent user. It could also be a cache problem as the agents cache the kerberos credentia= ls in the agent's temp directory. How long it takes after alerts start prod= ucing warnings would in determining if it's a caching issue. On May 5, 2015, at 10:49 AM, Chanel Lo=EFc > wrote: Hi, I have currently some issues with cluster nodes. According to the Ambari we= b User Interface, I have 11 alerts linked to the fact that all nodes (excep= t the monitoring one) return HTTP 403 response for services such as NameNod= e web UI, DataNode web UI, or NodeManager Health. The first weird thing is the fact that I can easily see that the so-called = Forbidden ports are actually quite available (via cURL for example) and ind= icate that the cluster is totally ok. The second is the fact that the 11 alerts magically disappear when rebootin= g the agent on the hosts related to the errors. Does anyone know where these errors might come from ? Thanks, Lo=EFc ________________________________ Ce message et les pi=E8ces jointes sont confidentiels et r=E9serv=E9s =E0 l= 'usage exclusif de ses destinataires. Il peut =E9galement =EAtre prot=E9g= =E9 par le secret professionnel. Si vous recevez ce message par erreur, mer= ci d'en avertir imm=E9diatement l'exp=E9diteur et de le d=E9truire. L'int= =E9grit=E9 du message ne pouvant =EAtre assur=E9e sur Internet, la responsa= bilit=E9 de Worldline ne pourra =EAtre recherch=E9e quant au contenu de ce = message. Bien que les meilleurs efforts soient faits pour maintenir cette t= ransmission exempte de tout virus, l'exp=E9diteur ne donne aucune garantie = =E0 cet =E9gard et sa responsabilit=E9 ne saurait =EAtre recherch=E9e pour = tout dommage r=E9sultant d'un virus transmis. This e-mail and the documents attached are confidential and intended solely= for the addressee; it may also be privileged. If you receive this e-mail i= n error, please notify the sender immediately and destroy it. As its integr= ity cannot be secured on the Internet, the Worldline liability cannot be tr= iggered for the message content. Although the sender endeavours to maintain= a computer virus-free network, the sender does not warrant that this trans= mission is virus-free and will not be liable for any damages resulting from= any virus transmitted. ________________________________ Ce message et les pi=E8ces jointes sont confidentiels et r=E9serv=E9s =E0 l= 'usage exclusif de ses destinataires. Il peut =E9galement =EAtre prot=E9g= =E9 par le secret professionnel. Si vous recevez ce message par erreur, mer= ci d'en avertir imm=E9diatement l'exp=E9diteur et de le d=E9truire. L'int= =E9grit=E9 du message ne pouvant =EAtre assur=E9e sur Internet, la responsa= bilit=E9 de Worldline ne pourra =EAtre recherch=E9e quant au contenu de ce = message. Bien que les meilleurs efforts soient faits pour maintenir cette t= ransmission exempte de tout virus, l'exp=E9diteur ne donne aucune garantie = =E0 cet =E9gard et sa responsabilit=E9 ne saurait =EAtre recherch=E9e pour = tout dommage r=E9sultant d'un virus transmis. This e-mail and the documents attached are confidential and intended solely= for the addressee; it may also be privileged. If you receive this e-mail i= n error, please notify the sender immediately and destroy it. As its integr= ity cannot be secured on the Internet, the Worldline liability cannot be tr= iggered for the message content. Although the sender endeavours to maintain= a computer virus-free network, the sender does not warrant that this trans= mission is virus-free and will not be liable for any damages resulting from= any virus transmitted. ________________________________ Ce message et les pi=E8ces jointes sont confidentiels et r=E9serv=E9s =E0 l= 'usage exclusif de ses destinataires. Il peut =E9galement =EAtre prot=E9g= =E9 par le secret professionnel. Si vous recevez ce message par erreur, mer= ci d'en avertir imm=E9diatement l'exp=E9diteur et de le d=E9truire. L'int= =E9grit=E9 du message ne pouvant =EAtre assur=E9e sur Internet, la responsa= bilit=E9 de Worldline ne pourra =EAtre recherch=E9e quant au contenu de ce = message. Bien que les meilleurs efforts soient faits pour maintenir cette t= ransmission exempte de tout virus, l'exp=E9diteur ne donne aucune garantie = =E0 cet =E9gard et sa responsabilit=E9 ne saurait =EAtre recherch=E9e pour = tout dommage r=E9sultant d'un virus transmis. This e-mail and the documents attached are confidential and intended solely= for the addressee; it may also be privileged. If you receive this e-mail i= n error, please notify the sender immediately and destroy it. As its integr= ity cannot be secured on the Internet, the Worldline liability cannot be tr= iggered for the message content. Although the sender endeavours to maintain= a computer virus-free network, the sender does not warrant that this trans= mission is virus-free and will not be liable for any damages resulting from= any virus transmitted. ________________________________ Ce message et les pi=E8ces jointes sont confidentiels et r=E9serv=E9s =E0 l= 'usage exclusif de ses destinataires. Il peut =E9galement =EAtre prot=E9g= =E9 par le secret professionnel. Si vous recevez ce message par erreur, mer= ci d'en avertir imm=E9diatement l'exp=E9diteur et de le d=E9truire. L'int= =E9grit=E9 du message ne pouvant =EAtre assur=E9e sur Internet, la responsa= bilit=E9 de Worldline ne pourra =EAtre recherch=E9e quant au contenu de ce = message. Bien que les meilleurs efforts soient faits pour maintenir cette t= ransmission exempte de tout virus, l'exp=E9diteur ne donne aucune garantie = =E0 cet =E9gard et sa responsabilit=E9 ne saurait =EAtre recherch=E9e pour = tout dommage r=E9sultant d'un virus transmis. This e-mail and the documents attached are confidential and intended solely= for the addressee; it may also be privileged. If you receive this e-mail i= n error, please notify the sender immediately and destroy it. As its integr= ity cannot be secured on the Internet, the Worldline liability cannot be tr= iggered for the message content. Although the sender endeavours to maintain= a computer virus-free network, the sender does not warrant that this trans= mission is virus-free and will not be liable for any damages resulting from= any virus transmitted. ________________________________ Ce message et les pi=E8ces jointes sont confidentiels et r=E9serv=E9s =E0 l= 'usage exclusif de ses destinataires. Il peut =E9galement =EAtre prot=E9g= =E9 par le secret professionnel. Si vous recevez ce message par erreur, mer= ci d'en avertir imm=E9diatement l'exp=E9diteur et de le d=E9truire. L'int= =E9grit=E9 du message ne pouvant =EAtre assur=E9e sur Internet, la responsa= bilit=E9 de Worldline ne pourra =EAtre recherch=E9e quant au contenu de ce = message. Bien que les meilleurs efforts soient faits pour maintenir cette t= ransmission exempte de tout virus, l'exp=E9diteur ne donne aucune garantie = =E0 cet =E9gard et sa responsabilit=E9 ne saurait =EAtre recherch=E9e pour = tout dommage r=E9sultant d'un virus transmis. This e-mail and the documents attached are confidential and intended solely= for the addressee; it may also be privileged. If you receive this e-mail i= n error, please notify the sender immediately and destroy it. As its integr= ity cannot be secured on the Internet, the Worldline liability cannot be tr= iggered for the message content. Although the sender endeavours to maintain= a computer virus-free network, the sender does not warrant that this trans= mission is virus-free and will not be liable for any damages resulting from= any virus transmitted. ________________________________ Ce message et les pi=E8ces jointes sont confidentiels et r=E9serv=E9s =E0 l= 'usage exclusif de ses destinataires. Il peut =E9galement =EAtre prot=E9g= =E9 par le secret professionnel. Si vous recevez ce message par erreur, mer= ci d'en avertir imm=E9diatement l'exp=E9diteur et de le d=E9truire. L'int= =E9grit=E9 du message ne pouvant =EAtre assur=E9e sur Internet, la responsa= bilit=E9 de Worldline ne pourra =EAtre recherch=E9e quant au contenu de ce = message. Bien que les meilleurs efforts soient faits pour maintenir cette t= ransmission exempte de tout virus, l'exp=E9diteur ne donne aucune garantie = =E0 cet =E9gard et sa responsabilit=E9 ne saurait =EAtre recherch=E9e pour = tout dommage r=E9sultant d'un virus transmis. This e-mail and the documents attached are confidential and intended solely= for the addressee; it may also be privileged. If you receive this e-mail i= n error, please notify the sender immediately and destroy it. As its integr= ity cannot be secured on the Internet, the Worldline liability cannot be tr= iggered for the message content. Although the sender endeavours to maintain= a computer virus-free network, the sender does not warrant that this trans= mission is virus-free and will not be liable for any damages resulting from= any virus transmitted. ________________________________ Ce message et les pi=E8ces jointes sont confidentiels et r=E9serv=E9s =E0 l= 'usage exclusif de ses destinataires. Il peut =E9galement =EAtre prot=E9g= =E9 par le secret professionnel. Si vous recevez ce message par erreur, mer= ci d'en avertir imm=E9diatement l'exp=E9diteur et de le d=E9truire. L'int= =E9grit=E9 du message ne pouvant =EAtre assur=E9e sur Internet, la responsa= bilit=E9 de Worldline ne pourra =EAtre recherch=E9e quant au contenu de ce = message. Bien que les meilleurs efforts soient faits pour maintenir cette t= ransmission exempte de tout virus, l'exp=E9diteur ne donne aucune garantie = =E0 cet =E9gard et sa responsabilit=E9 ne saurait =EAtre recherch=E9e pour = tout dommage r=E9sultant d'un virus transmis. This e-mail and the documents attached are confidential and intended solely= for the addressee; it may also be privileged. If you receive this e-mail i= n error, please notify the sender immediately and destroy it. As its integr= ity cannot be secured on the Internet, the Worldline liability cannot be tr= iggered for the message content. Although the sender endeavours to maintain= a computer virus-free network, the sender does not warrant that this trans= mission is virus-free and will not be liable for any damages resulting from= any virus transmitted. --_000_3D14BFE16DE13B4A941B2ECAC978CA8A7C09C89466FRVDX100fr01a_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi Jonatha= n,

 = ;

First of a= ll sorry for my (very) late answer.

I confirm = that restarting the agents before installing the cluster fixes the issue. B= ut what is slightly more complicated is the fact that there is not network trace.

The comman= d      sudo tcp= dump -i lo -l -s0 -w - tcp dst port 8042 | strings &       executed on the host returning 403 returns absolutely nothing.=

 = ;

Does that = help you ?

If you wan= t any additional information, feel free to ask.

 = ;

Regards,

 = ;

 = ;

Lo=EFc

 = ;

De : Jona= than Hurley [mailto:jhurley@hortonworks.com]
Envoy=E9 : mercredi 13 mai 2015 23:39
=C0 : user@ambari.apache.org
Objet : Re: Restarting nodes to avoid HTTP 403

 

Well, I’m really baffled by this. Restarting t= he agents before installing a cluster fixes the issue as well? So it seems = like after agents are installed they are not able to make connections from = python without getting a 403 forbidden until they are restarted at least once. Is it possible to get a network trace of= the agents when they encounter the 403 forbidden? That way we can see the = communication path between the agent and a particular endpoint, like NameNo= de WebUI.

 

On May 13, 2015, at 8:31 AM, Chanel Lo=EFc <loic.chanel@worldline.com> w= rote:

 

Hi Jonathan,<= /o:p>

 <= /p>

Additional= information : the restart does not have to be necessarily AFTER the c= luster have been deployed using a Blueprint. Restarting the ambari-agent before using a blueprint to deploy the cluster make a perfectly clear clus= ter, without the 11 Warnings/Errors I mentioned in my previous emails.

 

Hope this = will help understand where the problem comes from.

Have a nic= e day,

 

 

Lo=EFc

 

De : Jonathan Hurley [mailto:jhurley@hortonwo= rks.com] 
Envoy=E9 : je= udi 7 mai 2015 18:53
=C0 : user@ambari.apache.org
Objet : Re: R= estarting nodes to avoid HTTP 403

 

All of this really seems to point to some kind of fi= rewall/proxy issue on the agent hosts. From your agent named "vm-03cfbc97-f027-46fe-8e65-cb8c54edf377.frida.pri= v.atos.fr", could you try the following python code:

 

>>> import urllib2

>>> response =3D urllib2.urlopen("http://vm-03cfbc97-f027-46= fe-8e65-cb8c54edf377.frida.priv.atos.fr:8042/ws/v1/node/info&quo= t;, timeout=3D10.0)

>>> print(response.code)

 

I’m really curious if executing that from your= agent host results in a 200 or a 403.

 

On May 7, 2015, at 11:45 AM, Chanel Lo=EFc <loic.c= hanel@worldline.com> wrote:

 

Hi Jonathan,<= /p>

 

Here are the answers to your qu= estions :

 

1) Which operating system are y= ou running the agents on?

=E0 = ;CentOS6

 

2) Is there a linux proxy setup= , such as “export http_proxy=3Dfoo” - curl doesn’t respec= t this proxy setting but python’s urllib2 does, which has caused some= issues before

=E0 = ;There is a proxy, but there is no such = setup (I mean “echo $http_proxy” returns nothing, and I do not remember setting Ambari properly in order to access to the Intern= et via the proxy)

 

3) Which stack are you deployin= g?

=E0 = ;I am deploying HDP-2.2.4.2-2

 

Have a nice weekend,

 

 

Lo=EFc

 

 

De : Jonathan Hurley [mailto:jhurley@hortonworks.com] 
Envoy=E9 : je= udi 7 mai 2015 17:34
=C0 : user@ambari= .apache.org
Objet : Re: R= estarting nodes to avoid HTTP 403

 

The logs indicate that the alerts are running correc= tly and are simply hitting a 403. Normally, you might encounter this kind o= f problem from Python when making web connections without specifying a know= n user agent header. Python’s default header sometimes causes issues since it’s not standard. However, tha= t problem would continue to happen after you restarted the agents. The fact= that a simple agent restart completely fixes the issue is baffling. <= o:p>

 

I’ve certainly never seen this type of behavio= r before. I’d like to know a few more details on your environment:

1) Which operating system are you running the agents= on?

2) Is there a linux proxy setup, such as “expo= rt http_proxy=3Dfoo” - curl doesn’t respect this proxy setting = but python’s urllib2 does, which has caused some issues before

3) Which stack are you deploying?

 

On May 7, 2015, at 4:05 AM, Chanel Lo=EFc <loic.ch= anel@worldline.com> wrote:

 

Hi Jonathan,<= /o:p>

 <= /p>

You will f= ind in attachment what I get from the API you gave me the URL.<= /o:p>

As far as = the log file is concerned, here is an extract from the one on the NameNode = :

 

WARNING 20= 15-05-07 10:01:47,221 base_alert.py:365 - [Alert][namenode_directory_status= ] HA nameservice value is present but there are no aliases for {{hdfs-site/dfs.ha.namenodes.{{ha-nameservice}}}}

WARNING 20= 15-05-07 10:01:47,222 base_alert.py:365 - [Alert][datanode_health_summary] = HA nameservice value is present but there are no aliases for {{hdfs-site/dfs.ha.namenodes.{{ha-nameservice}}}}

INFO 2015-= 05-07 10:01:47,228 scheduler.py:509 - Running job "e89e2c29-1f2c-4bf5-= a37c-7a6c5b43433a (trigger: interval[0:01:00], next run at: 2015-05-07 10:02:47.211937)" (scheduled at 2015-05-07 10:01:47.211937)

INFO 2015-= 05-07 10:01:47,229 scheduler.py:509 - Running job "ad0e0fd3-d5f3-471f-= 955e-ec10f53cdd5b (trigger: interval[0:01:00], next run at: 2015-05-07 10:01:47.212773)" (scheduled at 2015-05-07 10:01:47.212773)

WARNING 20= 15-05-07 10:01:47,230 base_alert.py:365 - [Alert][namenode_webui] HA namese= rvice value is present but there are no aliases for {{hdfs-site/dfs.ha.name= nodes.{{ha-nameservice}}}}

INFO 2015-= 05-07 10:01:47,230 scheduler.py:527 - Job "71988556-19e3-4871-92e0-e3c= 0a838df13 (trigger: interval[0:01:00], next run at: 2015-05-07 10:02:47.206667)" executed successfully

INFO 2015-= 05-07 10:01:47,238 scheduler.py:509 - Running job "2de4dc85-8993-4c68-= 9915-db25c4313d6e (trigger: interval[0:01:00], next run at: 2015-05-07 10:02:47.213529)" (scheduled at 2015-05-07 10:01:47.213529)

WARNING 20= 15-05-07 10:01:47,240 base_alert.py:140 - [Alert][datanode_health_summary] = Unable to execute alert. HTTP Error 403: Forbidden

INFO 2015-= 05-07 10:01:47,242 scheduler.py:527 - Job "5bcf7b5d-73e7-4d59-bc0a-f47= 72d4e3166 (trigger: interval[0:01:00], next run at: 2015-05-07 10:02:47.208176)" executed successfully

INFO 2015-= 05-07 10:01:47,241 scheduler.py:509 - Running job "ff1fc102-bdec-4858-= baf5-8b60de4488e4 (trigger: interval[0:01:00], next run at: 2015-05-07 10:01:47.214826)" (scheduled at 2015-05-07 10:01:47.214826)

WARNING 20= 15-05-07 10:01:47,244 base_alert.py:365 - [Alert][yarn_resourcemanager_webu= i] HA nameservice value is present but there are no aliases for {{yarn-site/yarn.resourcemanager.ha.rm-ids}}

INFO 2015-= 05-07 10:01:47,240 scheduler.py:527 - Job "72e5031e-4c2a-4236-b09e-6a7= 49100bc9a (trigger: interval[0:01:00], next run at: 2015-05-07 10:02:47.203349)" executed successfully

INFO 2015-= 05-07 10:01:47,250 scheduler.py:509 - Running job "a083e3c2-d4b2-430b-= 9b64-b60c783a06a5 (trigger: interval[0:01:00], next run at: 2015-05-07 10:01:47.218706)" (scheduled at 2015-05-07 10:01:47.218706)

INFO 2015-= 05-07 10:01:47,250 scheduler.py:509 - Running job "7e9470cb-043b-48a8-= 990a-0050f2c63311 (trigger: interval[0:01:00], next run at: 2015-05-07 10:02:47.217519)" (scheduled at 2015-05-07 10:01:47.217519)

WARNING 20= 15-05-07 10:01:47,253 base_alert.py:140 - [Alert][namenode_directory_status= ] Unable to execute alert. HTTP Error 403: Forbidden

 

So yes, I = can see some output in the logs corresponding to the alerts I get on the Am= bari web app.

Please tel= l me if you need any complementary information about my problem,

 

Thanks,

 

 

Lo=EFc

 

 

De : Jonathan Hurley [mailto:jhurley@hortonworks.com] 
Envoy=E9 : me= rcredi 6 mai 2015 16:24
=C0 : user@ambari= .apache.org
Objet : Re: R= estarting nodes to avoid HTTP 403

 

OK, so I think I have a clear picture of how you get= to this situation. I’d still like to know a few things:

 

1) When you have the warnings present in the web cli= ent, can you try the alerts URL I posted below to see the actual alerts com= ing back from the API. I’m mostly interesting in whether any come bac= k in the first place, and what the most recent timestamp was (indicating they are actually running and still repor= ting a warning status)

 

2) When the alerts are present in the web client, do= you see any output in the agent log file that I mentioned for alerts that = start with [Alert].

 

On May 6, 2015, at 5:14 AM, Chanel Lo=EFc <loic.ch= anel@worldline.com> wrote:

 

I encounte= r the warnings and critical alerts when deploying the cluster. I install Am= bari server 2.0 on a VM and Ambari agent 2.0 on 4 others VMs. Then, I give the Ambari server a blueprint coming from a functioning clust= er to instantiate my new cluster and have a quickstart configuration.

 

Then, when= logging into the Ambari web application to ensure everything is running pr= operly, I have these alerts concerning the HTTP 403 errors returned by all host VMs but the one which only handles Ambari metrics.

 

As I am no= t sure my explanations are quite understandable, do not hesitate to tell me= if something remains unclear.

Thanks,

 

 

De : Jonathan Hurley [mailto:jhurley@hortonworks.com] 
Envoy=E9 : ma= rdi 5 mai 2015 20:38
=C0 : user@ambari= .apache.org
Objet : Re: R= estarting nodes to avoid HTTP 403

 

If restarting the agents fixes everything, can you e= xplain when you first encounter the warnings? Is this only after a cluster = deployment? You can also check to see if this is some sort of web client is= sue by issuing the following GET:

 

 

This will show you alerts which are actually being r= eturned from the agents in a warning or critical state. 

 

For reference, you can also look in /var/log/ambari-= agent/ambari-agent.log to see if you see any alert issues. Most important m= essages are prefixed with "[Alert]”. 

 

On May 5, 2015, at 11:43 AM, Chanel Lo=EFc <loic.c= hanel@worldline.com> wrote:

 

Indeed, I = did not gave so much information about my problem, sorry about that. Here i= s your answers :

 

1) Version of Ambari 

=E0 = ;I’m using Ambari 2.0<= /o:p>

 

2) Whether the environment is k= erberized

=E0 = ;Not it’s not. Kerberos security i= s not enabled on this cluster.

 

3) Are you running the Ambari a= gent as root, or another user

=E0 = ;I am running it as root

 

4) Any information from the amb= ari-agent.log file that might seem to indicate a problem<= /p>

=E0 = ;Here is another weird thing I did not m= entioned : I could not find logs referring to the problem.

 

5) You said that restarting the= agents resolves the issue. Does it continue to happen after restarting? If= so, how long before new warnings start to show up

=E0 = ;Restarting the agent totally resolves t= he problem. It does not happen anymore, and everything run quite normally.

 

 

De : Jonathan Hurley [mailto:jhurley@hortonworks.com] 
Envoy=E9 : ma= rdi 5 mai 2015 17:36
=C0 : user@ambari= .apache.org
Objet : Re: R= estarting nodes to avoid HTTP 403

 

Can you provide some more information on your enviro= nment, such as:

 

1) Version of Ambari

2) Whether the environment is kerberized<= /p>

3) Are you running the Ambari agent as root, or anot= her user.

4) Any information from the ambari-agent.log file th= at might seen to indicate a problem

5) You said that restarting the agents resolves the = issue. Does it continue to happen after restarting? If so, how long before = new warnings start to show up.

 

I’m guessing you have a kerberized environment= running Ambari 2.0. Ambari will use curl in this case to attempt to make a= connection to the web endpoints. It uses the keytabs and principals define= d on the alert definition. For NameNode, as an example, it would use: 

 

hdfs-site/dfs.web.authentication.kerberos.keytab&nbs= p;

hdfs-site/dfs.web.authentication.kerberos.principal&= nbsp;

 

You’ll want to verify that these properties ar= e correctly set and that the keytab file is accessible to the agent user.

 

It could also be a cache problem as the agents cache= the kerberos credentials in the agent’s temp directory. How long it = takes after alerts start producing warnings would in determining if it̵= 7;s a caching issue.

 

 

On May 5, 2015, at 10:49 AM, Chanel Lo=EFc <loic.c= hanel@worldline.com> wrote:

 

Hi,

 =

I have currently some is= sues with cluster nodes. According to the Ambari web User Interface, I have= 11 alerts linked to the fact that all nodes (except the monitoring one) return HTTP 403 response for services such as NameNode web UI, DataNo= de web UI, or NodeManager Health.

 =

The first weird thing is= the fact that I can easily see that the so-called Forbidden ports are actu= ally quite available (via cURL for example) and indicate that the cluster is totally ok.

The second is the fact t= hat the 11 alerts magically disappear when rebooting the agent on the hosts= related to the errors.

 =

Does anyone know where t= hese errors might come from ?

 =

Thanks,

 =

 =

Lo=EFc=

 


Ce message et les pi=E8ces jointes sont confidentiels et r=E9serv=E9s =E0 l= 'usage exclusif de ses destinataires. Il peut =E9galement =EAtre prot=E9g= =E9 par le secret professionnel. Si vous recevez ce message par erreur, mer= ci d'en avertir imm=E9diatement l'exp=E9diteur et de le d=E9truire. L'int=E9grit=E9 du message ne pouvant =EAtre assur=E9e sur Int= ernet, la responsabilit=E9 de Worldline ne pourra =EAtre recherch=E9e quant= au contenu de ce message. Bien que les meilleurs efforts soient faits pour= maintenir cette transmission exempte de tout virus, l'exp=E9diteur ne donne aucune garantie =E0 cet =E9gard et sa responsabili= t=E9 ne saurait =EAtre recherch=E9e pour tout dommage r=E9sultant d'un viru= s transmis.

This e-mail and the documents attached are confidential and intended solely= for the addressee; it may also be privileged. If you receive this e-mail i= n error, please notify the sender immediately and destroy it. As its integr= ity cannot be secured on the Internet, the Worldline liability cannot be triggered for the message content. Altho= ugh the sender endeavours to maintain a computer virus-free network, the se= nder does not warrant that this transmission is virus-free and will not be = liable for any damages resulting from any virus transmitted.

 

 


Ce message et les pi=E8ces jointes sont confidentiels et r=E9serv=E9s =E0 l= 'usage exclusif de ses destinataires. Il peut =E9galement =EAtre prot=E9g= =E9 par le secret professionnel. Si vous recevez ce message par erreur, mer= ci d'en avertir imm=E9diatement l'exp=E9diteur et de le d=E9truire. L'int=E9grit=E9 du message ne pouvant =EAtre assur=E9e sur Int= ernet, la responsabilit=E9 de Worldline ne pourra =EAtre recherch=E9e quant= au contenu de ce message. Bien que les meilleurs efforts soient faits pour= maintenir cette transmission exempte de tout virus, l'exp=E9diteur ne donne aucune garantie =E0 cet =E9gard et sa responsabili= t=E9 ne saurait =EAtre recherch=E9e pour tout dommage r=E9sultant d'un viru= s transmis.

This e-mail and the documents attached are confidential and intended solely= for the addressee; it may also be privileged. If you receive this e-mail i= n error, please notify the sender immediately and destroy it. As its integr= ity cannot be secured on the Internet, the Worldline liability cannot be triggered for the message content. Altho= ugh the sender endeavours to maintain a computer virus-free network, the se= nder does not warrant that this transmission is virus-free and will not be = liable for any damages resulting from any virus transmitted.

 

 


Ce message et les pi=E8ces jointes sont confidentiels et r=E9serv=E9s =E0 l= 'usage exclusif de ses destinataires. Il peut =E9galement =EAtre prot=E9g= =E9 par le secret professionnel. Si vous recevez ce message par erreur, mer= ci d'en avertir imm=E9diatement l'exp=E9diteur et de le d=E9truire. L'int=E9grit=E9 du message ne pouvant =EAtre assur=E9e sur Int= ernet, la responsabilit=E9 de Worldline ne pourra =EAtre recherch=E9e quant= au contenu de ce message. Bien que les meilleurs efforts soient faits pour= maintenir cette transmission exempte de tout virus, l'exp=E9diteur ne donne aucune garantie =E0 cet =E9gard et sa responsabili= t=E9 ne saurait =EAtre recherch=E9e pour tout dommage r=E9sultant d'un viru= s transmis.

This e-mail and the documents attached are confidential and intended solely= for the addressee; it may also be privileged. If you receive this e-mail i= n error, please notify the sender immediately and destroy it. As its integr= ity cannot be secured on the Internet, the Worldline liability cannot be triggered for the message content. Altho= ugh the sender endeavours to maintain a computer virus-free network, the se= nder does not warrant that this transmission is virus-free and will not be = liable for any damages resulting from any virus transmitted.

 

 


Ce message et les pi=E8ces jointes sont confidentiels et r=E9serv=E9s =E0 l= 'usage exclusif de ses destinataires. Il peut =E9galement =EAtre prot=E9g= =E9 par le secret professionnel. Si vous recevez ce message par erreur, mer= ci d'en avertir imm=E9diatement l'exp=E9diteur et de le d=E9truire. L'int=E9grit=E9 du message ne pouvant =EAtre assur=E9e sur Int= ernet, la responsabilit=E9 de Worldline ne pourra =EAtre recherch=E9e quant= au contenu de ce message. Bien que les meilleurs efforts soient faits pour= maintenir cette transmission exempte de tout virus, l'exp=E9diteur ne donne aucune garantie =E0 cet =E9gard et sa responsabili= t=E9 ne saurait =EAtre recherch=E9e pour tout dommage r=E9sultant d'un viru= s transmis.

This e-mail and the documents attached are confidential and intended solely= for the addressee; it may also be privileged. If you receive this e-mail i= n error, please notify the sender immediately and destroy it. As its integr= ity cannot be secured on the Internet, the Worldline liability cannot be triggered for the message content. Altho= ugh the sender endeavours to maintain a computer virus-free network, the se= nder does not warrant that this transmission is virus-free and will not be = liable for any damages resulting from any virus transmitted.
<API alerts.txt>

 

 


Ce message et les pi=E8ces jointes sont confidentiels et r=E9serv=E9s =E0 l= 'usage exclusif de ses destinataires. Il peut =E9galement =EAtre prot=E9g= =E9 par le secret professionnel. Si vous recevez ce message par erreur, mer= ci d'en avertir imm=E9diatement l'exp=E9diteur et de le d=E9truire. L'int=E9grit=E9 du message ne pouvant =EAtre assur=E9e sur Int= ernet, la responsabilit=E9 de Worldline ne pourra =EAtre recherch=E9e quant= au contenu de ce message. Bien que les meilleurs efforts soient faits pour= maintenir cette transmission exempte de tout virus, l'exp=E9diteur ne donne aucune garantie =E0 cet =E9gard et sa responsabili= t=E9 ne saurait =EAtre recherch=E9e pour tout dommage r=E9sultant d'un viru= s transmis.

This e-mail and the documents attached are confidential and intended solely= for the addressee; it may also be privileged. If you receive this e-mail i= n error, please notify the sender immediately and destroy it. As its integr= ity cannot be secured on the Internet, the Worldline liability cannot be triggered for the message content. Altho= ugh the sender endeavours to maintain a computer virus-free network, the se= nder does not warrant that this transmission is virus-free and will not be = liable for any damages resulting from any virus transmitted.

 

 


Ce message et les pi=E8ces jointes sont confidentiels et r=E9serv=E9s =E0 l= 'usage exclusif de ses destinataires. Il peut =E9galement =EAtre prot=E9g= =E9 par le secret professionnel. Si vous recevez ce message par erreur, mer= ci d'en avertir imm=E9diatement l'exp=E9diteur et de le d=E9truire. L'int=E9grit=E9 du message ne pouvant =EAtre assur=E9e sur Int= ernet, la responsabilit=E9 de Worldline ne pourra =EAtre recherch=E9e quant= au contenu de ce message. Bien que les meilleurs efforts soient faits pour= maintenir cette transmission exempte de tout virus, l'exp=E9diteur ne donne aucune garantie =E0 cet =E9gard et sa responsabili= t=E9 ne saurait =EAtre recherch=E9e pour tout dommage r=E9sultant d'un viru= s transmis.

This e-mail and the documents attached are confidential and intended solely= for the addressee; it may also be privileged. If you receive this e-mail i= n error, please notify the sender immediately and destroy it. As its integr= ity cannot be secured on the Internet, the Worldline liability cannot be triggered for the message content. Altho= ugh the sender endeavours to maintain a computer virus-free network, the se= nder does not warrant that this transmission is virus-free and will not be = liable for any damages resulting from any virus transmitted.

 



Ce message et les pi=E8ces jointes sont confidentiels et r=E9serv=E9s =E0 l= 'usage exclusif de ses destinataires. Il peut =E9galement =EAtre prot=E9g= =E9 par le secret professionnel. Si vous recevez ce message par erreur, mer= ci d'en avertir imm=E9diatement l'exp=E9diteur et de le d=E9truire. L'int=E9grit=E9 du message ne pouvant =EAtre assur=E9e sur Int= ernet, la responsabilit=E9 de Worldline ne pourra =EAtre recherch=E9e quant= au contenu de ce message. Bien que les meilleurs efforts soient faits pour= maintenir cette transmission exempte de tout virus, l'exp=E9diteur ne donne aucune garantie =E0 cet =E9gard et sa responsabili= t=E9 ne saurait =EAtre recherch=E9e pour tout dommage r=E9sultant d'un viru= s transmis.

This e-mail and the documents attached are confidential and intended solely= for the addressee; it may also be privileged. If you receive this e-mail i= n error, please notify the sender immediately and destroy it. As its integr= ity cannot be secured on the Internet, the Worldline liability cannot be triggered for the message content. Altho= ugh the sender endeavours to maintain a computer virus-free network, the se= nder does not warrant that this transmission is virus-free and will not be = liable for any damages resulting from any virus transmitted.
 --_000_3D14BFE16DE13B4A941B2ECAC978CA8A7C09C89466FRVDX100fr01a_--