ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pengcheng Xu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMBARI-12339) Simulate Ambari Agents to test 3k Node Cluster
Date Thu, 09 Jul 2015 20:34:05 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-12339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Pengcheng Xu updated AMBARI-12339:
----------------------------------
    Attachment: AMBARI-12339.patch

add Apache License and clean code

> Simulate Ambari Agents to test 3k Node Cluster
> ----------------------------------------------
>
>                 Key: AMBARI-12339
>                 URL: https://issues.apache.org/jira/browse/AMBARI-12339
>             Project: Ambari
>          Issue Type: Story
>          Components: ambari-agent
>    Affects Versions: 2.2.0
>            Reporter: Alejandro Fernandez
>            Assignee: Pengcheng Xu
>             Fix For: 2.2.0
>
>         Attachments: AMBARI-12339.patch, AMBARI-12339.patch, Screen Shot 2015-07-08 at
1.18.55 PM.png, Screen Shot 2015-07-08 at 1.19.14 PM.png
>
>
> Ambari's design is based on a single server with multiple agents. The agents heartbeat
every x seconds to the server, and the server may reply with a list of commands to run. Once
the agent completes a set of commands, it returns a response in its next heartbeat, so that
the server can update the result. The response contains the command id, output code, stdout,
and stderr.
> In practice, a production cluster will have 100-1000 agents, each of which is running
on a VM. However, for developers working on Ambmari, it is unrealistic to test the scalability
of their code by spinning up 100 VMs and having the agents execute actual commands. Creating
so many VMs is costly in terms of infrastructure, difficult for developers to manage and keep
operational.
> Each agent is tracked by its hostname and IP. In this case, the hostname will need to
be different, and most likely all use the same IP.
> Ambari needs a way to simulate large-scale clusters, in which the agents send dummy heartbeats
and responses.
> Some of the responses are acknowledgements to running commands, while others are just
STATUS commands (e.g., I'm alive, NameNode is up, DataNode is up, etc.)
> We need a way to simulate the agents that allows us to test,
> Server accepting heartbeats from hundreds of dummy agents
> Server sending requests to run commands, and accepting dummy data
>  
> *Goal:*
> The developers need an easy way to simulate the agents and specify how many, and where
the agents will reside (e.g., same host as the server, or different machine).
> The ultimate goal is to be able to do scale testing on functionality like,
> Ambari Metrics
> Ambari Alerts
> Rolling Upgrade 
> So that the customers can benefit from features that have been tested at scale.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message