spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Gruno <humbed...@apache.org>
Subject Re: [ANNOUNCE] Announcing Spark 1.5.1
Date Sun, 11 Oct 2015 15:49:27 GMT
Here's my issue:

How am I to audit that the dependencies you bundle are in fact what you
claim they are?  How do I know they don't contain malware or - in light
of recent events - emissions test rigging? ;)

I am not interested in a git tag - that means nothing in the ASF voting
process, you cannot vote on a tag, only on a release candidate. The VCS
in use is irrelevant in this issue. If you can point me to a release
candidate archive that was voted upon and does not contain binary
applications, all is well.

If there is no such thing, and we cannot come to an understanding, I
will exercise my ASF Members' rights and bring this to the attention of
the board of directors and ask for a clarification of the legality of this.

I find it highly irregular. Perhaps it is something some projects do in
the Java community, but that doesn't make it permissible in my view.

With regards,
Daniel.


On 10/11/2015 05:42 PM, Sean Owen wrote:
> Still confused. Why are you saying we didn't vote on an archive? refer
> to the email I linked, which includes both the git tag and a link to
> all generated artifacts (also in my email).
> 
> So, there are two things at play here:
> 
> First, I am not sure what you mean that a source distro can't have
> binary files. It's supposed to have the source code of Spark, and
> shouldn't contain binary Spark. Nothing you listed are Spark binaries.
> However, a distribution might have a lot of things in it that support
> the source build, like copies of tools, test files, etc.  That
> explains I think the first couple lines that you identified.
> 
> Still, I am curious why you are saying that would invalidate a source
> release? I have never heard anything like that.
> 
> Second, I do think there are some binaries in here that aren't
> supposed to be there, like the build/ directory stuff. IIRC these were
> included accidentally and won't be in the next release. At least, I
> don't see why they need to be bundled. These are just local copies of
> third party tools though, and don't really matter. As it happens, the
> licenses that get distributed with the source distro even cover all of
> this stuff. I think that's not supposed to be there, but, also don't
> see it's 'invalid' as a result.
> 
> 
> On Sun, Oct 11, 2015 at 4:33 PM, Daniel Gruno <humbedooh@apache.org> wrote:
>> On 10/11/2015 05:29 PM, Sean Owen wrote:
>>> Of course, but what's making you think this was a binary-only
>>> distribution?
>>
>> I'm not saying binary-only, I am saying your source release contains
>> binary programs, which would invalidate a release vote. Is there a
>> release candidate package, that is voted on (saying you have a git tag
>> does not satisfy this criteria, you need to vote on an actual archive of
>> files, otherwise there is no cogent proof of the release being from that
>> specific git tag).
>>
>> Here's what I found in your source release:
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/sql/hive/src/test/resources/data/files/TestSerDe.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/sql/hive/src/test/resources/regression-test-SPARK-8489/test.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/sql/hive/src/test/resources/TestUDTF.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/R/pkg/inst/test_support/sparktestjar_2.10-1.0.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/zinc-0.3.5.3/lib/scala-reflect.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/zinc-0.3.5.3/lib/sbt-interface.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/zinc-0.3.5.3/lib/compiler-interface-sources.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/zinc-0.3.5.3/lib/incremental-compiler.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/zinc-0.3.5.3/lib/scala-compiler.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/zinc-0.3.5.3/lib/zinc.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/zinc-0.3.5.3/lib/scala-library.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/misc/scala-devel/plugins/continuations.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/lib/scala-reflect.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/lib/akka-actors.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/lib/typesafe-config.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/lib/scala-actors-migration.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/lib/scala-actors.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/lib/scalap.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/lib/scala-swing.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/lib/scala-compiler.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/lib/scala-library.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/src/scala-reflect-src.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/src/scala-swing-src.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/src/scalap-src.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/src/scala-actors-src.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/src/scala-partest-src.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/src/scala-library-src.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/src/fjbg-src.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/src/scala-compiler-src.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/scala-2.10.4/src/msil-src.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/apache-maven-3.3.3/boot/plexus-classworlds-2.5.2.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/apache-maven-3.3.3/lib/guava-18.0.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/apache-maven-3.3.3/lib/wagon-http-2.9-shaded.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/apache-maven-3.3.3/lib/jsr250-api-1.0.jar
>>
>> Binary application (application/jar; charset=binary) found in
>> spark-1.5.1/build/apache-maven-3.3.3/lib/javax.inject-1.jar
>>
>>
>>
>> The downloads page points you directly to the source
>>> distro: http://spark.apache.org/downloads.html
>>>
>>> Look for the last vote, and you'll find it was of course a vote on
>>> source (and binary) artifacts:
>>> http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-Release-Apache-Spark-1-5-1-RC1-tt14310.html#none
>>> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
>>>
>>> On Sun, Oct 11, 2015 at 4:23 PM, Daniel Gruno <humbedooh@apache.org> wrote:
>>>> On 10/11/2015 05:12 PM, Sean Owen wrote:
>>>>> The Spark releases include a source distribution and several binary
>>>>> distributions. This is pretty normal for Apache projects. What are you
>>>>> referring to here?
>>>>
>>>> Surely the _source_ distribution does not contain binaries? How else can
>>>> you vote on a release if you don't know what it contains?
>>>>
>>>> You can produce convenience downloads that contain binary files, yes,
>>>> but surely you need a source-only package which is the one you vote on,
>>>> that does not contain any binaries. Do you have such a thing? And where
>>>> may I find it?
>>>>
>>>> With regards,
>>>> Daniel.
>>>>
>>>>>
>>>>> On Sun, Oct 11, 2015 at 3:26 PM, Daniel Gruno <humbedooh@apache.org>
wrote:
>>>>>> Out of curiosity: How can you vote on a release that contains 34
binary files? Surely a source code release should only contain source code and not binaries,
as you cannot verify the content of these.
>>>>>>
>>>>>> Looking forward to a response.
>>>>>>
>>>>>> With regards,
>>>>>> Daniel.
>>>>>>
>>>>>> On 10/2/2015, 4:42:31 AM, Reynold Xin <rxin@databricks.com>
wrote:
>>>>>>> Hi All,
>>>>>>>
>>>>>>> Spark 1.5.1 is a maintenance release containing stability fixes.
This
>>>>>>> release is based on the branch-1.5 maintenance branch of Spark.
We
>>>>>>> *strongly recommend* all 1.5.0 users to upgrade to this release.
>>>>>>>
>>>>>>> The full list of bug fixes is here: http://s.apache.org/spark-1.5.1
>>>>>>>
>>>>>>> http://spark.apache.org/releases/spark-release-1-5-1.html
>>>>>>>
>>>>>>>
>>>>>>> (note: it can take a few hours for everything to be propagated,
so you
>>>>>>> might get 404 on some download links, but everything should be
in maven
>>>>>>> central already)
>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>>>>>> For additional commands, e-mail: dev-help@spark.apache.org
>>>>>>
>>>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message