spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <so...@cloudera.com>
Subject Re: What is d3kbcqa49mib13.cloudfront.net ?
Date Wed, 13 Sep 2017 18:49:41 GMT
Ah right yeah I know it's an S3 bucket. Thanks for the context. Although I
imagine the reasons it was set up no longer apply so much (you can get a
direct mirror download link), and so it would probably be possible to
retire this, there's also no big rush to. I wasn't clear from the thread
whether it was agreed that the non-Apache link should be the default
though.

On Wed, Sep 13, 2017 at 6:27 PM Shivaram Venkataraman <
shivaram@eecs.berkeley.edu> wrote:

> The bucket comes from Cloudfront, a CDN thats part of AWS. There was a
> bunch of discussion about this back in 2013
>
> https://lists.apache.org/thread.html/9a72ff7ce913dd85a6b112b1b2de536dcda74b28b050f70646aba0ac@1380147885@%3Cdev.spark.apache.org%3E
>
> Shivaram
>
> On Wed, Sep 13, 2017 at 9:30 AM, Sean Owen <sowen@cloudera.com> wrote:
> > Not a big deal, but Mark noticed that this test now downloads Spark
> > artifacts from the same 'direct download' link available on the downloads
> > page:
> >
> >
> https://github.com/apache/spark/blob/master/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala#L53
> >
> > https://d3kbcqa49mib13.cloudfront.net/spark-$version-bin-hadoop2.7.tgz
> >
> > I don't know of any particular problem with this, which is a parallel
> > download option in addition to the Apache mirrors. It's also the default.
> >
> > Does anyone know what this bucket is and if there's a strong reason we
> can't
> > just use mirrors?
>

Mime
View raw message