hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chang Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2556) Tool to measure the performance of the timeline server
Date Mon, 09 Nov 2015 16:39:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14996832#comment-14996832

Chang Li commented on YARN-2556:

Hi [~xgong], here is the usage print out by the tool  {code} 
Usage: [-m <maps>] number of mappers (default: 1)
     [-v] timeline service version
     [-mtype <mapper type in integer>]
          1. simple entity write mapper
          2. jobhistory files replay mapper
     [-s <(KBs)test>] number of KB per put (mtype=1, default: 1 KB)
     [-t] package sending iterations per mapper (mtype=1, default: 100)
     [-d <path>] root path of job history files (mtype=2)
     [-r <replay mode>] (mtype=2)
          1. write all entities for a job in one put (default)
          2. write one entity at a time{code}
there are two different modes to test, one is simple entity writer, where each mapper create
your specified size of entities and put them to timeline server. The other mode of test is
by replaying jobhistory files, which offer a more realistic test. In the case of jobhistory
file replay test, you put testing jobhistory files(both the job history file and job conf
file) under a directory, and then you specify the testing dir by -d option. You specify the
test mode by -mtype option. 
Right now the usage won't get printed out if you pass no options, but only print out when
you pass the wrong options. When you give no parameters, the test run with simple entity write
mode and default setting. So maybe we want to print out this usage if we don't pass any parameter?

> Tool to measure the performance of the timeline server
> ------------------------------------------------------
>                 Key: YARN-2556
>                 URL: https://issues.apache.org/jira/browse/YARN-2556
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Jonathan Eagles
>            Assignee: Chang Li
>              Labels: BB2015-05-TBR
>             Fix For: 2.8.0
>         Attachments: YARN-2556-WIP.patch, YARN-2556-WIP.patch, YARN-2556.1.patch, YARN-2556.10.patch,
YARN-2556.11.patch, YARN-2556.12.patch, YARN-2556.13.patch, YARN-2556.13.whitespacefix.patch,
YARN-2556.14.patch, YARN-2556.14.whitespacefix.patch, YARN-2556.15.patch, YARN-2556.2.patch,
YARN-2556.3.patch, YARN-2556.4.patch, YARN-2556.5.patch, YARN-2556.6.patch, YARN-2556.7.patch,
YARN-2556.8.patch, YARN-2556.9.patch, YARN-2556.patch, yarn2556.patch, yarn2556.patch, yarn2556_wip.patch
> We need to be able to understand the capacity model for the timeline server to give users
the tools they need to deploy a timeline server with the correct capacity.
> I propose we create a mapreduce job that can measure timeline server write and read performance.
Transactions per second, I/O for both read and write would be a good start.
> This could be done as an example or test job that could be tied into gridmix.

This message was sent by Atlassian JIRA

View raw message