ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez (JIRA)" <>
Subject [jira] [Updated] (AMBARI-12339) Simulate Ambari Agents to test 3k Node Cluster
Date Wed, 08 Jul 2015 20:20:44 GMT


Alejandro Fernandez updated AMBARI-12339:
    Attachment: Screen Shot 2015-07-08 at 1.19.14 PM.png
                Screen Shot 2015-07-08 at 1.18.55 PM.png

> Simulate Ambari Agents to test 3k Node Cluster
> ----------------------------------------------
>                 Key: AMBARI-12339
>                 URL:
>             Project: Ambari
>          Issue Type: Story
>          Components: ambari-agent
>    Affects Versions: 2.2.0
>            Reporter: Alejandro Fernandez
>             Fix For: 2.2.0
>         Attachments: Screen Shot 2015-07-08 at 1.18.55 PM.png, Screen Shot 2015-07-08
at 1.19.14 PM.png
> Ambari's design is based on a single server with multiple agents. The agents heartbeat
every x seconds to the server, and the server may reply with a list of commands to run. Once
the agent completes a set of commands, it returns a response in its next heartbeat, so that
the server can update the result. The response contains the command id, output code, stdout,
and stderr.
> In practice, a production cluster will have 100-1000 agents, each of which is running
on a VM. However, for developers working on Ambmari, it is unrealistic to test the scalability
of their code by spinning up 100 VMs and having the agents execute actual commands. Creating
so many VMs is costly in terms of infrastructure, difficult for developers to manage and keep
> Each agent is tracked by its hostname and IP. In this case, the hostname will need to
be different, and most likely all use the same IP.
> Ambari needs a way to simulate large-scale clusters, in which the agents send dummy heartbeats
and responses.
> Some of the responses are acknowledgements to running commands, while others are just
STATUS commands (e.g., I'm alive, NameNode is up, DataNode is up, etc.)
> We need a way to simulate the agents that allows us to test,
> Server accepting heartbeats from hundreds of dummy agents
> Server sending requests to run commands, and accepting dummy data
> *Goal:*
> The developers need an easy way to simulate the agents and specify how many, and where
the agents will reside (e.g., same host as the server, or different machine).
> The ultimate goal is to be able to do scale testing on functionality like,
> Ambari Metrics
> Ambari Alerts
> Rolling Upgrade 
> So that the customers can benefit from features that have been tested at scale.

This message was sent by Atlassian JIRA

View raw message