spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mengxr <...@git.apache.org>
Subject [GitHub] spark pull request #22449: [SPARK-22666][ML][FOLLOW-UP] Return a correctly f...
Date Tue, 18 Sep 2018 16:12:35 GMT
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/22449#discussion_r218498363
  
    --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/image/ImageFileFormat.scala
---
    @@ -85,7 +85,9 @@ private[image] class ImageFileFormat extends FileFormat with DataSourceRegister
             val filteredResult = if (imageSourceOptions.dropInvalid) {
               resultOpt.toIterator
             } else {
    -          Iterator(resultOpt.getOrElse(ImageSchema.invalidImageRow(origin)))
    +          val basePath = Path.getPathWithoutSchemeAndAuthority(path).toString()
    --- End diff --
    
    Seems authority got dropped here.
    
    ~~~
    scala> Path.getPathWithoutSchemeAndAuthority(new Path("s3://dbc/test/ajdj/dfdfd"))
    res10: org.apache.hadoop.fs.Path = /test/ajdj/dfdfd
    ~~~


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message