hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vrushali C (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14840) Tool to estimate resource requirements of an application pipeline based on prior executions
Date Tue, 05 Sep 2017 23:08:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16154532#comment-16154532
] 

Vrushali C commented on HADOOP-14840:
-------------------------------------

Thanks Subru for the response! 

No, we don't need to break it up into smaller patches. My question was more out of curiosity
and was wondering if this was still at an initial stage and if so, would there be sub tasks.
Good to hear that you already have something running internally and want to contribute it
back to the community. Will look forward to the design doc with the perspective of understanding
it for future use cases for TSv2. Will also look through the paper. 

thanks!

> Tool to estimate resource requirements of an application pipeline based on prior executions
> -------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-14840
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14840
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: tools
>            Reporter: Subru Krishnan
>            Assignee: Rui Li
>
> We have been working on providing SLAs for job execution on Hadoop. At high level this
involves 2 parts: deriving the resource requirements of a job and guaranteeing the estimated
resources at runtime. The {{YARN ReservationSystem}} (YARN-1051/YARN-2572/YARN-5326) enable
the latter and in this JIRA, we propose to add a tool to Hadoop to predict the  resource requirements
of a job based on past executions of the job. The system (aka *Morpheus*) deep dive can be
found in our OSDI'16 paper [here|https://www.usenix.org/conference/osdi16/technical-sessions/presentation/jyothi].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message