Return-Path: 
 <yarn-issues-return-73105-apmail-hadoop-yarn-issues-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 905ED18BDF
	for <apmail-hadoop-yarn-issues-archive@minotaur.apache.org>;
 Mon,  9 Nov 2015 16:39:11 +0000 (UTC)
Received: (qmail 50773 invoked by uid 500); 9 Nov 2015 16:39:11 -0000
Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org
Received: (qmail 50716 invoked by uid 500); 9 Nov 2015 16:39:11 -0000
Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:yarn-issues-help@hadoop.apache.org>
List-Unsubscribe: <mailto:yarn-issues-unsubscribe@hadoop.apache.org>
List-Post: <mailto:yarn-issues@hadoop.apache.org>
List-Id: <yarn-issues.hadoop.apache.org>
Reply-To: yarn-issues@hadoop.apache.org
Delivered-To: mailing list yarn-issues@hadoop.apache.org
Received: (qmail 50656 invoked by uid 99); 9 Nov 2015 16:39:11 -0000
Received: from arcas.apache.org (HELO arcas) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 09 Nov 2015 16:39:11 +0000
Received: from arcas.apache.org (localhost [127.0.0.1])
	by arcas (Postfix) with ESMTP id 312DB2C1F75
	for <yarn-issues@hadoop.apache.org>; Mon,  9 Nov 2015 16:39:11 +0000 (UTC)
Date: Mon, 9 Nov 2015 16:39:11 +0000 (UTC)
From: "Chang Li (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12741589.1410813850000.17086.1447087151198@Atlassian.JIRA>
In-Reply-To: <JIRA.12741589.1410813850000@Atlassian.JIRA>
References: <JIRA.12741589.1410813850000@Atlassian.JIRA>
 <JIRA.12741589.1410813850346@arcas>
Subject: [jira] [Commented] (YARN-2556) Tool to measure the performance of
 the timeline server
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


    [ https://issues.apache.org/jira/browse/YARN-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14996832#comment-14996832 ] 

Chang Li commented on YARN-2556:
--------------------------------

Hi [~xgong], here is the usage print out by the tool  {code} 
Usage: [-m <maps>] number of mappers (default: 1)
     [-v] timeline service version
     [-mtype <mapper type in integer>]
          1. simple entity write mapper
          2. jobhistory files replay mapper
     [-s <(KBs)test>] number of KB per put (mtype=1, default: 1 KB)
     [-t] package sending iterations per mapper (mtype=1, default: 100)
     [-d <path>] root path of job history files (mtype=2)
     [-r <replay mode>] (mtype=2)
          1. write all entities for a job in one put (default)
          2. write one entity at a time{code}
there are two different modes to test, one is simple entity writer, where each mapper create your specified size of entities and put them to timeline server. The other mode of test is by replaying jobhistory files, which offer a more realistic test. In the case of jobhistory file replay test, you put testing jobhistory files(both the job history file and job conf file) under a directory, and then you specify the testing dir by -d option. You specify the test mode by -mtype option. 
Right now the usage won't get printed out if you pass no options, but only print out when you pass the wrong options. When you give no parameters, the test run with simple entity write mode and default setting. So maybe we want to print out this usage if we don't pass any parameter?

> Tool to measure the performance of the timeline server
> ------------------------------------------------------
>
>                 Key: YARN-2556
>                 URL: https://issues.apache.org/jira/browse/YARN-2556
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Jonathan Eagles
>            Assignee: Chang Li
>              Labels: BB2015-05-TBR
>             Fix For: 2.8.0
>
>         Attachments: YARN-2556-WIP.patch, YARN-2556-WIP.patch, YARN-2556.1.patch, YARN-2556.10.patch, YARN-2556.11.patch, YARN-2556.12.patch, YARN-2556.13.patch, YARN-2556.13.whitespacefix.patch, YARN-2556.14.patch, YARN-2556.14.whitespacefix.patch, YARN-2556.15.patch, YARN-2556.2.patch, YARN-2556.3.patch, YARN-2556.4.patch, YARN-2556.5.patch, YARN-2556.6.patch, YARN-2556.7.patch, YARN-2556.8.patch, YARN-2556.9.patch, YARN-2556.patch, yarn2556.patch, yarn2556.patch, yarn2556_wip.patch
>
>
> We need to be able to understand the capacity model for the timeline server to give users the tools they need to deploy a timeline server with the correct capacity.
> I propose we create a mapreduce job that can measure timeline server write and read performance. Transactions per second, I/O for both read and write would be a good start.
> This could be done as an example or test job that could be tied into gridmix.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)