spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <r...@databricks.com>
Subject Re: A proposal for Spark 2.0
Date Tue, 10 Nov 2015 23:53:29 GMT
On Tue, Nov 10, 2015 at 3:35 PM, Nicholas Chammas <
nicholas.chammas@gmail.com> wrote:

>
> > 3. Assembly-free distribution of Spark: don’t require building an
> enormous assembly jar in order to run Spark.
>
> Could you elaborate a bit on this? I'm not sure what an assembly-free
> distribution means.
>
>
Right now we ship Spark using a single assembly jar, which causes a few
different problems:

- total number of classes are limited on some configurations

- dependency swapping is harder


The proposal is to just avoid a single fat jar.

Mime
View raw message