spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sebastian Piu (JIRATEST)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-8294) Break down large methods in YARN code
Date Sun, 21 Jan 2018 18:12:03 GMT

    [ https://issues-test.apache.org/jira/browse/SPARK-8294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16265593#comment-16265593
] 

Sebastian Piu commented on SPARK-8294:
--------------------------------------

I had a quick look and all those methods seem to have been refactored now, is this issue still
relevant?

> Break down large methods in YARN code
> -------------------------------------
>
>                 Key: SPARK-8294
>                 URL: https://issues-test.apache.org/jira/browse/SPARK-8294
>             Project: Spark
>          Issue Type: Sub-task
>          Components: YARN
>    Affects Versions: 1.4.0
>            Reporter: Andrew Or
>            Priority: Major
>
> What large methods am I talking about?
> Client#prepareLocalResources ~ 170 lines
> ExecutorRunnable#prepareCommand ~ 100 lines
> Client#setupLaunchEnv ~ 100 lines
> ... many others that hover around 80 - 90 lines
> There are several things wrong with this. First, it's difficult to follow / review the
code. Second, it's difficult to test it at a fine-granularity. In the past we as a community
has been reluctant to add new regression tests for YARN changes. This stems from the fact
that it is difficult to write tests, and the cost is that we can't really ensure the correctness
of the code easily.



--
This message was sent by Atlassian JIRA
(v7.6.0#76001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message