spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From cloud-fan <...@git.apache.org>
Subject [GitHub] spark pull request #20147: [SPARK-22940][SQL] HiveExternalCatalogVersionsSui...
Date Thu, 04 Jan 2018 05:16:23 GMT
Github user cloud-fan commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20147#discussion_r159582192
  
    --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala
---
    @@ -85,6 +90,43 @@ class HiveExternalCatalogVersionsSuite extends SparkSubmitTestUtils
{
         new File(tmpDataDir, name).getCanonicalPath
       }
     
    +  private def getFileFromUrl(urlString: String, targetDir: String, filename: String):
Boolean = {
    +    val conf = new SparkConf
    +    val securityManager = new SecurityManager(conf)
    +    val hadoopConf = new Configuration
    +
    +    val outDir = new File(targetDir)
    +    if (!outDir.exists()) {
    +      outDir.mkdirs()
    +    }
    +
    +    try {
    +      val result = Utils.doFetchFile(urlString, outDir, filename, conf, securityManager,
hadoopConf)
    --- End diff --
    
    `Utils.doFetchFile` is a little overkill for this case, maybe we can just implement a
simple downloading function with java.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message