spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sro...@apache.org
Subject spark git commit: [SPARK-12136][STREAMING] rddToFileName does not properly handle prefix and suffix parameters
Date Thu, 10 Dec 2015 12:54:13 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-1.6 f6d866173 -> b5e5812f9


[SPARK-12136][STREAMING] rddToFileName does not properly handle prefix and suffix parameters

The original code does not properly handle the cases where the prefix is null, but suffix
is not null - the suffix should be used but is not.

The fix is using StringBuilder to construct the proper file name.

Author: bomeng <bmeng@us.ibm.com>
Author: Bo Meng <mengbo@bos-macbook-pro.usca.ibm.com>

Closes #10185 from bomeng/SPARK-12136.

(cherry picked from commit e29704f90dfe67d9e276d242699ac0a00f64fb91)
Signed-off-by: Sean Owen <sowen@cloudera.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b5e5812f
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/b5e5812f
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/b5e5812f

Branch: refs/heads/branch-1.6
Commit: b5e5812f9ef8aa8d133a75bb8aa8dd8680130efa
Parents: f6d8661
Author: bomeng <bmeng@us.ibm.com>
Authored: Thu Dec 10 12:53:53 2015 +0000
Committer: Sean Owen <sowen@cloudera.com>
Committed: Thu Dec 10 12:54:08 2015 +0000

----------------------------------------------------------------------
 .../org/apache/spark/streaming/StreamingContext.scala  | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/b5e5812f/streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala
----------------------------------------------------------------------
diff --git a/streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala b/streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala
index 6fb8ad3..53324e7 100644
--- a/streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala
+++ b/streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala
@@ -887,12 +887,13 @@ object StreamingContext extends Logging {
   }
 
   private[streaming] def rddToFileName[T](prefix: String, suffix: String, time: Time): String
= {
-    if (prefix == null) {
-      time.milliseconds.toString
-    } else if (suffix == null || suffix.length ==0) {
-      prefix + "-" + time.milliseconds
-    } else {
-      prefix + "-" + time.milliseconds + "." + suffix
+    var result = time.milliseconds.toString
+    if (prefix != null && prefix.length > 0) {
+      result = s"$prefix-$result"
+    }
+    if (suffix != null && suffix.length > 0) {
+      result = s"$result.$suffix"
     }
+    result
   }
 }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message