Mailing-List: contact dev-help@spark.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@spark.apache.org
Received-SPF: pass (nike.apache.org: domain of sowen@cloudera.com designates
 209.85.220.179 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <JIRA.12711897.1399015832413.218330.1399015934570@arcas>
References: <JIRA.12711897.1399015832413@arcas>
	<JIRA.12711897.1399015832413.218330.1399015934570@arcas>
Date: Fri, 2 May 2014 10:24:29 +0100
Message-ID: 
 <CAMAsSd+qHdi_7DqypBpRpJ96NG3iddWCzSdDQF2QyY0xPb+zFQ@mail.gmail.com>
Subject: Re: [jira] [Created] (SPARK-1698) Improve spark integration
From: Sean Owen <sowen@cloudera.com>
To: dev@spark.apache.org
Cc: issues@spark.apache.org
Content-Type: multipart/alternative; boundary=001a11c361dc44b2ed04f86759d1

--001a11c361dc44b2ed04f86759d1
Content-Type: text/plain; charset=UTF-8

#1 and #2 are not relevant the issue of jar size. These can be problems in
general, but don't think there have been issues attributable to file
clashes. Shading has mechanisms to deal with this anyway.

#3 is a problem in general too, but is not specific to shading. Where
versions collide, build processes like Maven and shading must be used to
resolve them. But this happens regardless of whether you shade a fat jar.

#4 is a real problem specific to Java 6. It does seem like it will be
important to identify and remove more unnecessary dependencies to work
around it.

But shading per se is not the problem, and it is important to make a
packaged jar for the app. What are you proposing? Dependencies to be
removed?
On May 2, 2014 8:32 AM, "Guoqiang Li (JIRA)" <jira@apache.org> wrote:

> Guoqiang Li created SPARK-1698:
> ----------------------------------
>
>              Summary: Improve spark integration
>                  Key: SPARK-1698
>                  URL: https://issues.apache.org/jira/browse/SPARK-1698
>              Project: Spark
>           Issue Type: Improvement
>           Components: Build, Deploy
>             Reporter: Guoqiang Li
>             Assignee: Guoqiang Li
>              Fix For: 1.0.0
>
>
> Use the shade plugin to create a big JAR with all the dependencies can
> cause a few problems
> 1. Missing jar's meta information
> 2. Some file is covered, eg: plugin.xml
> 3. Different versions of the jar may co-exist
> 4. Too big, java 6 does not support
>
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.2#6252)
>

--001a11c361dc44b2ed04f86759d1--