ambari-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sumit Mohanty <smoha...@hortonworks.com>
Subject Re: Ambari Custom Service Questions.
Date Sun, 10 May 2015 17:18:23 GMT
Inline ...

________________________________
From: Christopher Jackson <jackson.christopher.lee@gmail.com>
Sent: Sunday, May 10, 2015 8:49 AM
To: user@ambari.apache.org
Cc: Sumit Mohanty; Alejandro Fernandez
Subject: Re: Ambari Custom Service Questions.

Thanks for this information. I have a few follow up questions asked inline. Thanks.

Regards,
Christopher Jackson

On May 9, 2015, at 9:25 PM, Sumit Mohanty <smohanty@hortonworks.com<mailto:smohanty@hortonworks.com>>
wrote:

One addition to question 4.
________________________________
From: Alejandro Fernandez <afernandez@hortonworks.com<mailto:afernandez@hortonworks.com>>
Sent: Saturday, May 09, 2015 1:44 PM
To: user@ambari.apache.org<mailto:user@ambari.apache.org>; Christopher Jackson
Subject: Re: Ambari Custom Service Questions.

Hi Christopher, these are all very good questions, and it would be useful to supplement the
wiki with them.
Comments inline.

On 5/9/15, 11:59 AM, "Christopher Jackson" <jackson.christopher.lee@gmail.com<mailto:jackson.christopher.lee@gmail.com>>
wrote:

Hi All,

I’ve been in the process of creating a custom Ambari service over the past week and have
quite a few general questions in which I haven’t found answers for in documentation or on
the wiki. I was hoping some of you could help answer any of the following questions. Thanks
in advance.

1) I’ve noticed that when restarting a services component that is of type ‘CLIENT’ that
its install and configure method are invoked. I’m wondering if it’s in intended and if
so why? For components of type ‘MASTER’ a restart doesn’t seem to invoke install and
configure again, it just invokes stop then start. I ask about this because in my custom service
I have a CLIENT component in which there are some steps I do in the install stage that I don’t
want repeated every time its restarted.

Alejandro> For clients, a "Restart" or "Refresh Configs" essentially only needs to make
sure that the client libraries are present and the configs are setup. Since technically a
client cannot be restarted because it is not a daemon, the code is written in such a way that
it is idempotent, so no harm in installing libs that are already present, or settings configs
that are already there.
For Masters, they have independent commands to Install the libs, and Restart the daemon process.
For your client, is there any artifact you can check to avoid running your one-time-install
multiple times?

Chris> Thank you for the explanation. Yes I can work around this by checking if certain
artifacts exist.


2) Can someone explain the implication of the auto-deploy and its child elements in the context
where the following snippet would be placed in the metainfo.xml file of a custom service component
(MY_SERVICE/MY_COMPONENT_MASTER)?

<dependency>
<name>HBASE/HBASE_CLIENT</name>
<scope>host</scope>
<auto-deploy>
<enabled>true</enabled>
<co-locate>MY_SERVICE/MY_COMPONENT_CLIENT</co-locate>
</auto-deploy>
</dependency>

Alejandro> This is a really good question. Some components depend on others, and those
dependencies need to be installed automatically, either anywhere in the cluster, the same
host, or on the same host that contains another component. You probably don't need the "co-locate"
tag, since it is used to indicate masters that must be together, and is configured in the
UI during the Service Install Wizard.

Chris> As part of my custom service I am adding libraries to both the HDFS and HBASE Services.
My custom services Client component ensures these libraries get installed on the system and
symlinked to the appropriate lib folder. Here is my concern. If a user installs a cluster
with my service and then later adds an additional node to the cluster with HDFS or HBASE installed
on that node how can I ensure that my custom services client component also gets installed?
Is there a way to do that without defining a custom stack or modifying the HBASE and HDFS
Service definitions?

Sumit> At this point user needs to explicitly install the custom service client as well.
The capability that is needed is for a component to specify in its own metainfo that it is
a mandatory dependency to another component. Feel free to create a JIRA for this feature.


3) When creating a configuration file for a custom service what are the valid entries for
a <property-type> tag? PASSWORD, and what else? Is there any other child elements of
‘property’ that are useful? Perhaps anything that allows you to provide a regular expression
for validation?

Alejandro> These are PASSWORD, USER, GROUP, TEXT.  These only contain name, value, description.
If you need to use regex to validate a property, that means the UI should take care of it,
so take a look at ambari-web module, particularly, config_property_helper.js

4) Is there some function to restart a service in resource_management or other ambari python
library? Or should I just be restarting services using the command line tools and ensuring
to update the appropriate pid files? I ask this question because I’ve noticed I cannot restart
a service using the Ambari API as part of the installation/configuration steps of my custom
service, as the restart commands are queued while the custom service installation/configuration
is running, and will cause a timeout. I’m looking for a solution to this problem, what’s
recommended if not one of the approaches I’ve asked about above?

Alejandro> Once the service is defined in the metainfo.xml file, along with the python
file to use, it's up to that script to decide how to install and restart your service. Ambari
doesn't couple config changes with forcing the service to restart automatically, this is because
if a user makes a config change or installs something, Ambari only highlights that the service
needs to be restarted, but it's up to the user to decide when to do it. If you wanted to do
automatic restarts upon config changes, then that your python script would then have to call
the restart() method. Take a look at script.py

Sumit> In general the pattern of calling Ambari Server APIs from the implementation of
install/configure/start of a component definition is not supported. This is because only one
command can be executed at anytime on a host. In theory, you could make the call from the
install/configure/start implementation and not wait for the call to complete. Can you explain
the scenario a bit more? Are you restarting your custom service or some other service from
the install/configuration of the custom service.

Chris> As part of my custom service I alter configurations for HDFS, HBASE, OOZIE, ZOOKEEPER,
MAPREDUCE2, and YARN. I add libraries to HDFS, HBASE, and OOZIE server (Symlink libraries
from my component into the specified services lib folder). To add the libraries to OOZIE server
I’m stopping it, executing the oozie-setup shell script and then restarting it. So to make
sure these services pick up the new configurations, additional libraries I wanted to restart
them. Is there a better way to handle adding the libraries for these services?

Sumit> There are two parts to this - a) host level changes and b) Ambari/cluster level
operations. Host level changes is done by the install/configure implementation of the CUSTOM-SERVICE-CLIENT.
The rest of the operations need Ambari API calls.

What you have is correct - as in doing the lib installation etc. as part of CLIENT install.
Its the calls to Ambari APIs from the install script that is non-standard (I assume thats
what you have now). Is there a way you can call the APIs from outside of Ambari - when user
adds hosts or adds service? Perhaps a capability is needed where UI allows customizing of
what happens when host is added or service is added. You can still call the APIs from CLIENT
install/start scripts if you do not wait for completion.

5) How can I allow for the removal of a custom service from the ambari console? I know there
is a sequence of Ambari API commands I can run to: stop the service, stop each of its components,
and then finally delete the service but is there any way to incorporate that into the Ambari
UI instead of having a script on the filesystem that invokes those calls?

Alejandro> Ambari UI allows deleting a component as long as it is stopped. This is done
from the Host Details page. But deleting a service entirely is more complicated and I don't
believe the API supports this today (probably because it would have do delete records from
a lot of tables and is not a common use case).

Thanks so much,
Christopher Jackson




Mime
View raw message