spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From joshro...@apache.org
Subject spark git commit: [SPARK-12712] Fix failure in ./dev/test-dependencies when run against empty .m2 cache
Date Thu, 09 Jun 2016 08:01:12 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-1.6 5830828ef -> bb917fc65


[SPARK-12712] Fix failure in ./dev/test-dependencies when run against empty .m2 cache

This patch fixes a bug in `./dev/test-dependencies.sh` which caused spurious failures when
the script was run on a machine with an empty `.m2` cache. The problem was that extra log
output from the dependency download was conflicting with the grep / regex used to identify
the classpath in the Maven output. This patch fixes this issue by adjusting the regex pattern.

Tested manually with the following reproduction of the bug:

```
rm -rf ~/.m2/repository/org/apache/commons/
./dev/test-dependencies.sh
```

Author: Josh Rosen <joshrosen@databricks.com>

Closes #13568 from JoshRosen/SPARK-12712.

(cherry picked from commit 921fa40b14082bfd1094fa49fb3b0c46a79c1aaa)
Signed-off-by: Josh Rosen <joshrosen@databricks.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/bb917fc6
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/bb917fc6
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/bb917fc6

Branch: refs/heads/branch-1.6
Commit: bb917fc659ec62718214f2f2fceb03a90515ac3e
Parents: 5830828
Author: Josh Rosen <joshrosen@databricks.com>
Authored: Thu Jun 9 00:51:24 2016 -0700
Committer: Josh Rosen <joshrosen@databricks.com>
Committed: Thu Jun 9 00:52:43 2016 -0700

----------------------------------------------------------------------
 dev/test-dependencies.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/bb917fc6/dev/test-dependencies.sh
----------------------------------------------------------------------
diff --git a/dev/test-dependencies.sh b/dev/test-dependencies.sh
index efb49f7..7367a8b 100755
--- a/dev/test-dependencies.sh
+++ b/dev/test-dependencies.sh
@@ -84,7 +84,7 @@ for HADOOP_PROFILE in "${HADOOP_PROFILES[@]}"; do
   echo "Generating dependency manifest for $HADOOP_PROFILE"
   mkdir -p dev/pr-deps
   $MVN $HADOOP_MODULE_PROFILES -P$HADOOP_PROFILE dependency:build-classpath -pl assembly
\
-    | grep "Building Spark Project Assembly" -A 5 \
+    | grep "Dependencies classpath:" -A 1 \
     | tail -n 1 | tr ":" "\n" | rev | cut -d "/" -f 1 | rev | sort \
     | grep -v spark > dev/pr-deps/spark-deps-$HADOOP_PROFILE
 done


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message