hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Subru Krishnan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-14840) Tool to estimate resource requirements of an application pipeline based on prior executions
Date Tue, 05 Sep 2017 20:33:00 GMT
Subru Krishnan created HADOOP-14840:
---------------------------------------

             Summary: Tool to estimate resource requirements of an application pipeline based
on prior executions
                 Key: HADOOP-14840
                 URL: https://issues.apache.org/jira/browse/HADOOP-14840
             Project: Hadoop Common
          Issue Type: New Feature
          Components: tools
            Reporter: Subru Krishnan


We have been working on providing SLAs for job execution on Hadoop. At high level this involves
2 parts: deriving the resource requirements of a job and guaranteeing the estimated resources
at runtime. The {{YARN ReservationSystem}} (YARN-1051/YARN-2572/YARN-5326) enable the latter
and in this JIRA, we propose to add a tool to Hadoop to predict the  resource requirements
of a job based on past executions of the job. The system (aka *Morpheus*) deep dive can be
found in our OSDI'16 paper [here|https://www.usenix.org/conference/osdi16/technical-sessions/presentation/jyothi].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org


Mime
View raw message