hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7129) Application Catalog for YARN applications
Date Fri, 22 Mar 2019 19:21:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16799282#comment-16799282
] 

Eric Yang commented on YARN-7129:
---------------------------------

Patch 033 includes [~stevel@apache.org] and [~jeagles] feedbacks.

Time benchmark for building application catalog on a 2015 i7 MacBook Pro with Redhat 7 as
guest vm.
Benchmark command in hadoop-yarn-applications-catalog folder:

{code}
mvn clean package -Pdocker
{code}

Fresh checkout with no docker cache:
{code}
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] YARN Application Catalog ........................... SUCCESS [  1.090 s]
[INFO] YARN Application Catalog Webapp .................... SUCCESS [ 23.444 s]
[INFO] YARN Application Catalog Docker Image .............. SUCCESS [02:19 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:44 min
[INFO] Finished at: 2019-03-22T15:16:19-04:00
[INFO] Final Memory: 108M/906M
[INFO] ------------------------------------------------------------------------
{code}

After maven and docker cache are built once:
{code}
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] YARN Application Catalog ........................... SUCCESS [  1.268 s]
[INFO] YARN Application Catalog Webapp .................... SUCCESS [ 24.019 s]
[INFO] YARN Application Catalog Docker Image .............. SUCCESS [ 32.326 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 58.348 s
[INFO] Finished at: 2019-03-22T14:31:19-04:00
[INFO] Final Memory: 106M/981M
[INFO] ------------------------------------------------------------------------
{code}

Skipping docker build (mvn clean package):
{code}
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] YARN Application Catalog ........................... SUCCESS [  1.151 s]
[INFO] YARN Application Catalog Webapp .................... SUCCESS [ 25.091 s]
[INFO] YARN Application Catalog Docker Image .............. SUCCESS [  0.144 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 27.074 s
[INFO] Finished at: 2019-03-22T14:35:18-04:00
[INFO] Final Memory: 87M/917M
[INFO] ------------------------------------------------------------------------
{code}

Base on the result, it's good evidence to say that if developer can be patient on fresh build
once.  Having docker build inline does not introduce much overhead for developer that needs
to rapidly reiterate building process.

When using profile to generate artifact, some part of code might not be exercised daily, and
hide bugs from surfacing until release time.  I hope Hadoop community will consider to make
docker build process inline some day.

> Application Catalog for YARN applications
> -----------------------------------------
>
>                 Key: YARN-7129
>                 URL: https://issues.apache.org/jira/browse/YARN-7129
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: applications
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>            Priority: Major
>             Fix For: 3.3.0
>
>         Attachments: YARN Appstore.pdf, YARN-7129.001.patch, YARN-7129.002.patch, YARN-7129.003.patch,
YARN-7129.004.patch, YARN-7129.005.patch, YARN-7129.006.patch, YARN-7129.007.patch, YARN-7129.008.patch,
YARN-7129.009.patch, YARN-7129.010.patch, YARN-7129.011.patch, YARN-7129.012.patch, YARN-7129.013.patch,
YARN-7129.014.patch, YARN-7129.015.patch, YARN-7129.016.patch, YARN-7129.017.patch, YARN-7129.018.patch,
YARN-7129.019.patch, YARN-7129.020.patch, YARN-7129.021.patch, YARN-7129.022.patch, YARN-7129.023.patch,
YARN-7129.024.patch, YARN-7129.025.patch, YARN-7129.026.patch, YARN-7129.027.patch, YARN-7129.028.patch,
YARN-7129.029.patch, YARN-7129.030.patch, YARN-7129.031.patch, YARN-7129.032.patch, YARN-7129.033.patch
>
>
> YARN native services provides web services API to improve usability of application deployment
on Hadoop using collection of docker images.  It would be nice to have an application catalog
system which provides an editorial and search interface for YARN applications.  This improves
usability of YARN for manage the life cycle of applications.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message