hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yesha Vora (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8913) Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop Cluster
Date Mon, 29 Oct 2018 23:41:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16667888#comment-16667888
] 

Yesha Vora commented on YARN-8913:
----------------------------------

[~eyang], Thanks for feedback. Please find attached patch#3 addressing your suggestions. 

> Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop Cluster
> --------------------------------------------------------------------------------
>
>                 Key: YARN-8913
>                 URL: https://issues.apache.org/jira/browse/YARN-8913
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Yesha Vora
>            Assignee: Yesha Vora
>            Priority: Blocker
>         Attachments: YARN-8913.001.patch, YARN-8913.002.patch, YARN-8913.003.patch
>
>
> MaWo application can be used to run Hadoop UT faster in a Hadoop cluster.
>  Develop helper scripts to orchestrate end-to-end workflow for running Hadoop UT using
MaWo app.
> Pre-requisite:
>  * A Hadoop Cluster with HDFS and YARN installed
>  * Enable Docker on YARN feature
>  
> Helper-scripts
>  * MaWo_Driver
>  ** create a docker image with latest hadoop source code
>  ** create payload to MaWo app (This is input to mawo app where Each MaWo Task = UT execution
of each Hadoop Module)
>  ** Upload payload file to HDFS
>  ** Update MaWo-Launch.json to resolve RM_HOST / Docker Image etc dynamically
>  ** Launch MaWo app in Hadoop cluster



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message