spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ilganeli <>
Subject [GitHub] spark pull request: [SPARK-5931][CORE] Use consistent naming for t...
Date Sat, 28 Mar 2015 05:29:28 GMT
GitHub user ilganeli reopened a pull request:

    [SPARK-5931][CORE] Use consistent naming for time properties

    I've added new utility methods to do the conversion from times specified as e.g. 120s,
240ms, 360us to convert to a consistent internal representation. I've updated usage of these
constants throughout the code to be consistent. 
    I believe I've captured all usages of time-based properties throughout the code. I've
also updated variable names in a number of places to reflect their units for clarity and updated
documentation where appropriate. 

You can merge this pull request into a Git repository by running:

    $ git pull SPARK-5931

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5236
commit 59bf9e1d590e49f451aaa7feeb5165572f205358
Author: Ilya Ganelin <>
Date:   2015-03-27T21:15:26Z

    [SPARK-5931] Updated Utils and JavaUtils classes to add helper methods to handle time
strings. Updated time strings in a few places to properly parse time

commit 404f8c3762d7e23fdf3ef8d158d68f5b5aa38360
Author: Ilya Ganelin <>
Date:   2015-03-27T21:25:15Z

    Updated usage of spark.core.connection.ack.wait.timeout

commit 7db6d2a73ea4de5bb52bcdf8e4af7edcc06a10dd
Author: Ilya Ganelin <>
Date:   2015-03-27T21:27:57Z

    Updated usage of spark.akka.timeout

commit 4933fda30ce468d72e646536489468e295800775
Author: Ilya Ganelin <>
Date:   2015-03-27T21:30:05Z

    Updated usage of

commit c9f5cad6a35cc2f405d874a5ca62fa3f92ef0a8b
Author: Ilya Ganelin <>
Date:   2015-03-27T21:39:15Z


commit 21ef3dd6165022a1edc396c50817641d3321b1f7
Author: Ilya Ganelin <>
Date:   2015-03-27T21:40:52Z

    updated spark.shuffle.sasl.timeout

commit 064ebd67faf655f03d0e02763f53f1df60b3e891
Author: Ilya Ganelin <>
Date:   2015-03-27T21:50:51Z

    Updated usage of spark.cleaner.ttl

commit 7320c87289ded3f61607070a4098d00dc5ee17be
Author: Ilya Ganelin <>
Date:   2015-03-27T21:56:23Z

    updated spark.akka.heartbeat.interval

commit 272c2159195948ccae9b399126f8e85015e1aaae
Author: Ilya Ganelin <>
Date:   2015-03-27T22:00:57Z

    Updated spark.locality.wait

commit 3352d34d925650fdba34b05fc3521e15d18fcd11
Author: Ilya Ganelin <>
Date:   2015-03-27T22:02:59Z

    Updated spark.scheduler.maxRegisteredResourcesWaitingTime

commit 3f1cfc84e669df7c2198ed21d776fd391977f34f
Author: Ilya Ganelin <>
Date:   2015-03-27T22:05:04Z

    Updated spark.scheduler.revive.interval

commit 6d1518e8870dc91fec86d75be0cac11d3e1dc2b4
Author: Ilya Ganelin <>
Date:   2015-03-27T22:07:57Z

    Upated spark.speculation.interval

commit 2fcc91c7cd062ca67c5b9e1565d13533e7013c2f
Author: Ilya Ganelin <>
Date:   2015-03-27T22:13:51Z

    Updated spark.dynamicAllocation.executorIdleTimeout

commit 5181597688a6d6f0d9be311784582d033c908225
Author: Ilya Ganelin <>
Date:   2015-03-27T22:17:58Z

    Updated spark.dynamicAllocation.schedulerBacklogTimeout

commit c6a00954f33994c35e9d76cb0222b35ce46bab69
Author: Ilya Ganelin <>
Date:   2015-03-27T22:22:49Z

    Updated spark.core.connection.auth.wait.timeout

commit cde9bffef6ded083a9dfa80486687ddd07e85640
Author: Ilya Ganelin <>
Date:   2015-03-27T22:25:44Z

    Updated spark.streaming.blockInterval

commit 42477aadc1f39911d325a0c703fae2ad55a6c42a
Author: Ilya Ganelin <>
Date:   2015-03-27T22:28:32Z

    Updated configuration doc with note on specifying time properties

commit 9a29d8d3f502f43c237859e9f75f5d6c76662915
Author: Ilya Ganelin <>
Date:   2015-03-27T23:16:58Z

    Fixed misuse of time in streaming context test

commit 34f87c26dd936305fec3d00d6a8cc1e6bc48734c
Author: Ilya Ganelin <>
Date:   2015-03-28T05:27:58Z

    Update Utils.scala
    Fixed ordering to fix overlap of s with ms and us.

commit 8f741e19f66b011decb3cca285606a733a08990b
Author: Ilya Ganelin <>
Date:   2015-03-28T05:28:50Z

    Fixed overlap of (s) with (ms) and (us) in cascading if.


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message