hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2556) Tool to measure the performance of the timeline server
Date Tue, 26 May 2015 15:50:19 GMT

    [ https://issues.apache.org/jira/browse/YARN-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14559282#comment-14559282

Sangjin Lee commented on YARN-2556:

[~lichangleo], thanks much for updating the patch! I'm sorry I didn't get to review this sooner.
I'll spend some time today to review this in more detail.

That said, I'd like to raise one important point as a goal: *it would be great if we can have
a single TimelineServicePerformance class that can accommodate both v1 and v2 timeline service
tests*. That way, any v1- or v2-specific code can be isolated to the specific writers. If
we came this close but are still left with two versions of TimelineServicePerformance, that
would not be such a great outcome.

I think the latest patch is almost there. We might need to make the writer base class more
generic, as it still involves types that are specific to v1. Perhaps we can simply use the
mapper class in TimelineServicePerformance (i.e. remove EntityWriter altogether from TimelineServicePerformance),
and move any v1- or v2-specific writers into their own classes outside TimelineServicePerformance.

I know the option handling is going to be a sticky point, as it will be difficult to have
specific mappers inject their own option handling. Maybe that's something we might have to
live with. I'm curious if you have ideas to make even that generic.

> Tool to measure the performance of the timeline server
> ------------------------------------------------------
>                 Key: YARN-2556
>                 URL: https://issues.apache.org/jira/browse/YARN-2556
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Jonathan Eagles
>            Assignee: Chang Li
>              Labels: BB2015-05-TBR
>         Attachments: YARN-2556-WIP.patch, YARN-2556-WIP.patch, YARN-2556.1.patch, YARN-2556.10.patch,
YARN-2556.2.patch, YARN-2556.3.patch, YARN-2556.4.patch, YARN-2556.5.patch, YARN-2556.6.patch,
YARN-2556.7.patch, YARN-2556.8.patch, YARN-2556.9.patch, YARN-2556.patch, yarn2556.patch,
yarn2556.patch, yarn2556_wip.patch
> We need to be able to understand the capacity model for the timeline server to give users
the tools they need to deploy a timeline server with the correct capacity.
> I propose we create a mapreduce job that can measure timeline server write and read performance.
Transactions per second, I/O for both read and write would be a good start.
> This could be done as an example or test job that could be tied into gridmix.

This message was sent by Atlassian JIRA

View raw message