Return-Path: X-Original-To: apmail-ambari-user-archive@www.apache.org Delivered-To: apmail-ambari-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3AE0D17917 for ; Mon, 23 Mar 2015 06:03:56 +0000 (UTC) Received: (qmail 77760 invoked by uid 500); 23 Mar 2015 06:03:48 -0000 Delivered-To: apmail-ambari-user-archive@ambari.apache.org Received: (qmail 77727 invoked by uid 500); 23 Mar 2015 06:03:48 -0000 Mailing-List: contact user-help@ambari.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ambari.apache.org Delivered-To: mailing list user@ambari.apache.org Received: (qmail 77717 invoked by uid 99); 23 Mar 2015 06:03:47 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 23 Mar 2015 06:03:47 +0000 X-ASF-Spam-Status: No, hits=3.2 required=5.0 tests=FSL_HELO_BARE_IP_2,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of yusaku@hortonworks.com designates 64.78.52.187 as permitted sender) Received: from [64.78.52.187] (HELO relayvx12c.securemail.intermedia.net) (64.78.52.187) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 23 Mar 2015 06:03:43 +0000 Received: from securemail.intermedia.net (localhost [127.0.0.1]) by emg-ca-1-2.localdomain (Postfix) with ESMTP id A535153DE2 for ; Sun, 22 Mar 2015 23:03:22 -0700 (PDT) Subject: Re: Couple of question. MIME-Version: 1.0 x-echoworx-emg-received: Sun, 22 Mar 2015 23:03:22.590 -0700 x-echoworx-msg-id: d4bace2e-19ff-45ed-a977-3e276705d960 x-echoworx-action: delivered Received: from 10.254.155.17 ([10.254.155.17]) by emg-ca-1-2 (JAMES SMTP Server 2.3.2) with SMTP ID 3 for ; Sun, 22 Mar 2015 23:03:22 -0700 (PDT) Received: from MBX080-W4-CO-1.exch080.serverpod.net (unknown [10.224.117.101]) by emg-ca-1-2.localdomain (Postfix) with ESMTP id 5184D53DE2 for ; Sun, 22 Mar 2015 23:03:22 -0700 (PDT) Received: from MBX080-W4-CO-2.exch080.serverpod.net (10.224.117.102) by MBX080-W4-CO-1.exch080.serverpod.net (10.224.117.101) with Microsoft SMTP Server (TLS) id 15.0.1044.25; Sun, 22 Mar 2015 23:03:21 -0700 Received: from MBX080-W4-CO-2.exch080.serverpod.net ([10.224.117.102]) by mbx080-w4-co-2.exch080.serverpod.net ([10.224.117.102]) with mapi id 15.00.1044.021; Sun, 22 Mar 2015 23:03:21 -0700 From: Yusaku Sako To: "user@ambari.apache.org" Thread-Topic: Couple of question. Thread-Index: AQHQV1xXoVA8coZM4EiJHNkDMDNLGJ0h3M4AgAFIIYCAAEwMAIABdLAAgAFQp4CAA1BdAIAAKMeA Date: Mon, 23 Mar 2015 06:03:20 +0000 Message-ID: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [73.202.80.178] x-source-routing-agent: Processed Content-Type: multipart/alternative; boundary="_000_D134F8DA5A997yusakuhortonworkscom_" X-Virus-Checked: Checked by ClamAV on apache.org --_000_D134F8DA5A997yusakuhortonworkscom_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Angel, Thanks for your kind words. > Is true that It is not possible to use Ambari with an existing Hadoop ins= tallation? So, it is necessary to install all the platform using Ambari. This could be a problem for our platform team. That is true. Currently, you cannot directly manage and monitor an existing Hadoop cluste= r that was installed outside of Ambari. However, some folks have successfully performed "take over" procedures to b= ring such existing clusters under Ambari's management. This involves a bit of surgery such as using Ambari's Install Wizard on the= existing cluster but using fake directories for DataNodes (and others), an= d switching the directories to the actual ones after Install Wizard is done= , etc. This is not very straightforward and can get you in trouble if you = are not careful, but it is an option. Yusaku From: Angel Cervera Claudio > Reply-To: "user@ambari.apache.org" > Date: Monday, March 23, 2015 5:37 AM To: "user@ambari.apache.org" > Subject: Re: Couple of question. Hi Yusaku. Awesome community support! With people like you, I am sure that Ambari will be the best monitor tool! My last question (I don't want bored you :) ) : Is true that It is not possible to use Ambari with an existing Hadoop insta= llation? So, it is necessary to install all the platform using Ambari. This could be a problem for our platform team. Regards. 2015-03-21 1:01 GMT+00:00 Yusaku Sako >: Hi Angel, > Ganglia is going to be replace by Ambari Metric System. Will be the same = for Nagios? Yes, Ambari will ship with its own alerting system in 2.0. Nagios is no lo= nger supported. > If we can not use Ambari, we are thinking in develop an adhoc application= for metering and monitoring. We are thinking in use Time Serie Databases t= o store metric, like Druid or OpenTSDB. Is Amabari going to use any databas= es of this type? Ambari's Metric System (AMS) uses HBase as its implementation, like OpenTSD= B. In addition, AMS uses Phoenix to support SQL queries. I believe the storage layer for AMS was designed so that it can be swapped = out, but I will let others who are more familiar comment on that. > And the question that maybe you are tired to listen. :) Do you know when = is planed to release the version 2.0.0? A release vote for 2.0 just went out. If everything goes well, 2.0 will sh= ip within days. Thanks, Yusaku From: Angel Cervera Claudio > Reply-To: "user@ambari.apache.org" > Date: Friday, March 20, 2015 6:56 AM To: "user@ambari.apache.org" > Subject: Re: Couple of question. Hi Yusaku. Thank you for your response. Other three question: * Ganglia is going to be replace by Ambari Metric System. Will be the s= ame for Nagios? * If we can not use Ambari, we are thinking in develop an adhoc applica= tion for metering and monitoring. We are thinking in use Time Serie Databas= es to store metric, like Druid or OpenTSDB. Is Amabari going to use any dat= abases of this type? * And the question that maybe you are tired to listen. :) Do you know w= hen is planed to release the version 2.0.0? Regards and thanks for your time. 2015-03-19 6:42 GMT+00:00 Yusaku Sako >: Hi Angel, On Ambari 1.7.0 and earlier, Ganglia is used to capture information about m= emory usage, network usage, and other system metrics, as well as Hadoop-spe= cific metrics. Ambari 2.0.0 and onwards, Ambari Metric System will replace Ganglia as the = underlying metrics collection framework. In either system, you can tune sampling/aggregation frequencies and retenti= on policies so that you can control how long the collected samples/aggregat= ed values are kept at what granularity. Yusaku From: Angel Cervera Claudio > Reply-To: "user@ambari.apache.org" > Date: Thursday, March 19, 2015 4:10 AM To: "user@ambari.apache.org" > Subject: Re: Couple of question. Hi Yasuku. Thanks for your responses. Respect Historical data: One of the requirements that we have is to be able= to show/compare historical information. For example, it is necessary to be able to compare the memory used yesterda= y (when we executed the new version of a heavy MapReduce process) with the = memory consumed two month ago. This is only one example. Regards. 2015-03-18 6:35 GMT+00:00 Yusaku Sako >: Hi Angel, Please see my response inline below: From: Angel Cervera Claudio > Reply-To: "user@ambari.apache.org" > Date: Friday, March 6, 2015 12:50 AM To: "user@ambari.apache.org" > Subject: Couple of question. Hi everybody. I am looking for the way of custom Ambari with new widgets and views. Researching for few days (I never worked before with this tool), I have a c= ouple of questions: * Is there any easy way to create new widget to add in the main page of= Ambari? * Is there any possibility to have a different layout (different widget= s and views) per user. Something like profiles. * Is there possible to secure by roles and widgets/views/services (auth= orization)? [YS] These will be covered by new features being worked on to make the dash= boards customizable via https://issues.apache.org/jira/browse/AMBARI-9792. You will be able to define a default layout, as well as customized layouts = per user, create new widgets based on different metrics, share those widget= s, etc. Views (and their features) can be authorized per the Ambari Views = framework already. * I did not found anything about history data (real historical data, pe= r years). Is it possible? [YS] Can you clarify what you mean? Do you mean various time-series data e= mitted by the system/various components? I downloaded the source code for 1.7 (github tag 1.7), but in the pom.xml s= aid that it is the version 1.3.0-SNAPSHOT. Curiosly, there are not release = 1.3 mention in the ambari.apache.org page. Why? [YS] That's just an artifact of the source code in Ambari 1.7.0. If you wa= nt to rebuild, you can issue "mvn versions:set -DnewVersion=3D1.7.0.0" to s= et the version in the pom.xml files. You are correct that there never was = a 1.3 release. Regards. --_000_D134F8DA5A997yusakuhortonworkscom_ Content-Type: text/html; charset="iso-8859-1" Content-ID: Content-Transfer-Encoding: quoted-printable
Hi Angel,

Thanks for your kind words.

> Is true that It is not possible to use Ambari with an existi= ng Hadoop installation? So, it is necessary to install all the platform usi= ng Ambari.
This could be a problem for our platform team.

That is true.
Currently, you cannot directly manage and monitor an existing Hadoop c= luster that was installed outside of Ambari.
However, some folks have successfully performed "take over" = procedures to bring such existing clusters under Ambari's management.
This involves a bit of surgery such as using Ambari's Install Wizard o= n the existing cluster but using fake directories for DataNodes (and others= ), and switching the directories to the actual ones after Install Wizard is= done, etc.  This is not very straightforward and can get you in trouble if you are not careful, but it is an option.

Yusaku


From: Angel Cervera Claudio <angelcervera@silyan.com>
Reply-To: "user@ambari.apache.org" <user@ambari.apache.org>
Date: Monday, March 23, 2015 5:37 A= M
To: "user@ambari.apache.org" <user@ambari.apache.org>
Subject: Re: Couple of question.

Hi Yusaku.
Awesome community support!
With people like you, I am sure that Ambari will be the best monitor tool!<= br>
My last question (I don't want bored you :) ) :

Is true that It is not possible to use Ambari with an existing Hadoop = installation? So, it is necessary to install all the platform using Ambari.=
This could be a problem for our platform team.

Regards.


2015-03-21 1:01 GMT+00:00 Yusaku Sako <yusaku@hortonworks.com>:
Hi Angel,

> Ganglia is going to be replace by Ambari Metric System. Will be t= he same for Nagios?
Yes, Ambari will ship with its own alerting system in 2.0.  Nagio= s is no longer supported.

> If we can not use Ambari, we are thinking in develop an adho= c application for metering and monitoring. We are thinking in use Time Seri= e Databases to store metric, like Druid or OpenTSDB. Is Amabari going to us= e any databases of this type?
Ambari's Metric System (AMS) uses HBase as its implementation, like Op= enTSDB.  In addition, AMS uses Phoenix to support SQL queries.
I believe the storage layer for AMS was designed so that it can be swa= pped out, but I will let others who are more familiar comment on that.

> And the question that maybe you are tired to listen. :) Do you kn= ow when is planed to release the version 2.0.0?
A release vote for 2.0 just went out.  If everything goes well, 2= .0 will ship within days.

Thanks,
Yusaku


From: Angel Cerver= a Claudio <= angelcervera@silyan.com>
Reply-To: "user@ambari.apache.org" &= lt;user@ambari.= apache.org>
Date: Friday, March 20, 2015= 6:56 AM

To: "user@ambari.apache.org" <user@ambari.apache= .org>
Subject: Re: Couple of question.

Hi Yusaku.
Thank you for your response.

Other three question:
  • Ganglia is going to be replace by Ambari Metric System. Will be the sam= e for Nagios?
  • If we can not use Ambari, we are thinking in develop = an adhoc application for metering and monitoring. We are thinking in use Ti= me Serie Databases to store metric, like Druid or OpenTSDB. Is Amabari goin= g to use any databases of this type?
  • And the question that maybe yo= u are tired to listen. :) Do you know when is planed to release the version= 2.0.0?

Regards and thanks for your time.


2015-03-19 6:42 GMT+00:00 Yusaku Sako <yusaku@hortonworks.com>:
Hi Angel,

On Ambari 1.7.0 and earlier, Ganglia is used to capture information ab= out memory usage, network usage, and other system metrics, as well as Hadoo= p-specific metrics.
Ambari 2.0.0 and onwards, Ambari Metric System will replace Ganglia as= the underlying metrics collection framework.
In either system, you can tune sampling/aggregation frequencies and re= tention policies so that you can control how long the collected samples/agg= regated values are kept at what granularity.

Yusaku

From: Angel Cervera Claudio &= lt;angelcerver= a@silyan.com>
Reply-To: "user@ambari.apache.org" &= lt;user@ambari.= apache.org>
Date: Thursday, March 19, 20= 15 4:10 AM
To: "user@ambari.apache.org" <user@ambari.apache= .org>
Subject: Re: Couple of question.

Hi Yasuku.
Thanks for your responses.

Respect Historical data: One of the requirements that we have is to be able= to show/compare historical information.
For example, it is necessary to be able to compare the memory used yesterda= y (when we executed the new version of a heavy MapReduce process) with the = memory consumed two month ago.
This is only one example.

Regards.


2015-03-18 6:35 GMT+00:00 Yusaku Sako <yusaku@hortonworks.com>:
Hi Angel,

Please see my response inline below:

From: Angel Cervera Claudio <angelcervera@sily= an.com>
Reply-To: "user@ambari.apache.org" &= lt;user@ambari.= apache.org>
Date: Friday, March 6, 2015 12:50 A= M
To: "user@ambari.apache.org" <user@ambari.apache= .org>
Subject: Couple of question.

Hi everybody.

I am looking for the way of custom Ambari with new widgets and views.<= /div>
Researching for few days (I never worked before with this tool), I hav= e a couple of questions:
  • Is there any easy way to create new widget to add in the main page of A= mbari?
  • Is there any possibility to have a different layout (different widgets = and views) per user. Something like profiles.
  • Is there possible to secure by roles and widgets/views/services (author= ization)?
[YS] These will be covered by new features being worked on to make the= dashboards customizable via https://issues.apache.org/jira/brows= e/AMBARI-9792.
You will be able to define a default layout, as well as customized lay= outs per user, create new widgets based on different metrics, share those w= idgets, etc.  Views (and their features) can be authorized per the Amb= ari Views framework already.  
  • I did not found anything about history data (real historical data, per = years). Is it possible?
[YS] Can you clarify what you mean?  Do you mean various time-ser= ies data emitted by the system/various components?

I downloaded the source code for 1.7 (github tag 1.7), but in the pom.= xml said that it is the version 1.3.0-SNAPSHOT. Curiosly, there are not rel= ease 1.3 mention in the ambari.apache.org page. Why?

[YS] That's just an artifact of the source code in Ambari 1.7.0. = If you want to rebuild, you can issue "mvn versions:set –DnewVe= rsion=3D1.7.0.0" to set the version in the pom.xml files.  You ar= e correct that there never was a 1.3 release.

Regards.






 --_000_D134F8DA5A997yusakuhortonworkscom_--