spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Azuryy Yu <azury...@gmail.com>
Subject Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Date Mon, 16 Dec 2013 06:25:44 GMT
Maybe I am not give a clear description. I am runing Spark on yarn. instead
of Mesos. I just want build Spark with protobuf2.5. I am not care about
Mesos.

I've changed Spark pom.xml to probobuf2.5 manually.




On Mon, Dec 16, 2013 at 2:02 PM, Matei Zaharia <matei.zaharia@gmail.com>wrote:

> Mesos will almost certainly compile fine with protobuf 2.5. The protobuf
> compiler and binary format is forward-compatible across releases, it’s just
> the Java artifacts that aren’t. You’ll need to ask Mesos to provide a
> version with protobuf 2.5, and use that with these versions of Hadoop.
>
> Matei
>
> On Dec 15, 2013, at 7:00 PM, Liu, Raymond <raymond.liu@intel.com> wrote:
>
> > That issue is for 0.9's solution.
> >
> > And if you mean for 0.8.1, when you build against hadoop 2.2 Yarn,
> protobuf is already using 2.5.0 instead of 2.4.1. so it will works fine
> with hadoop 2.2
> > And regarding on 0.8.1 you build against hadoop 2.2 Yarn, while run upon
> mesos... strange combination, I am not sure, might have problem. If have
> problem, you might need to build mesos against 2.5.0, I don't test that, if
> you got time, mind take a test?
> >
> > Best Regards,
> > Raymond Liu
> >
> >
> > -----Original Message-----
> > From: Liu, Raymond [mailto:raymond.liu@intel.com]
> > Sent: Monday, December 16, 2013 10:48 AM
> > To: dev@spark.incubator.apache.org
> > Subject: RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
> >
> > Hi Azuryy
> >
> > Please Check https://spark-project.atlassian.net/browse/SPARK-995 for
> this protobuf version issue
> >
> > Best Regards,
> > Raymond Liu
> >
> > -----Original Message-----
> > From: Azuryy Yu [mailto:azuryyyu@gmail.com]
> > Sent: Monday, December 16, 2013 10:30 AM
> > To: dev@spark.incubator.apache.org
> > Subject: Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
> >
> > Hi here,
> > Do we have plan to upgrade protobuf from 2.4.1 to 2.5.0? PB has some
> uncompatable API between these two versions.
> > Hadoop-2.x using protobuf-2.5.0
> >
> >
> > but if some guys want to run Spark on mesos, then mesos using
> > protobuf-2.4.1 currently. so we may discuss here for a better solution.
> >
> >
> >
> > On Mon, Dec 16, 2013 at 7:42 AM, Azuryy Yu <azuryyyu@gmail.com> wrote:
> >
> >> Thanks Patrick.
> >> On 16 Dec 2013 02:43, "Patrick Wendell" <pwendell@gmail.com> wrote:
> >>
> >>> You can checkout the docs mentioned in the vote thread. There is also
> >>> a pre-build binary for hadoop2 that is compiled for YARN 2.2
> >>>
> >>> - Patrick
> >>>
> >>> On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu <azuryyyu@gmail.com> wrote:
> >>>> yarn 2.2, not yarn 0.22, I am so sorry.
> >>>>
> >>>>
> >>>> On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu <azuryyyu@gmail.com>
> wrote:
> >>>>
> >>>>> Hi,
> >>>>> Spark-0.8.1 supports yarn 0.22 right? where to find the release
note?
> >>>>> Thanks.
> >>>>>
> >>>>>
> >>>>> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra <
> >>> henry.saputra@gmail.com>wrote:
> >>>>>
> >>>>>> Yeah seems like it. He was ok with our prev release.
> >>>>>> Let's wait for his reply
> >>>>>>
> >>>>>> On Saturday, December 14, 2013, Patrick Wendell wrote:
> >>>>>>
> >>>>>>> Henry - from that thread it looks like sebb's concern was
> >>>>>>> something different than this.
> >>>>>>>
> >>>>>>> On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra <
> >>>>>> henry.saputra@gmail.com>
> >>>>>>> wrote:
> >>>>>>>> Hi Patrick,
> >>>>>>>>
> >>>>>>>> Yeap I agree, but technically ASF VOTE release on source
> >>>>>>>> only,
> >>> there
> >>>>>>>> even debate about it =), so putting it in the vote staging
> >>> artifact
> >>>>>>>> could confuse people because in our case we do package
3rd
> >>>>>>>> party libraries in the binary jars.
> >>>>>>>>
> >>>>>>>> I have sent email to sebb asking clarification about
his
> >>>>>>>> concern
> >>> in
> >>>>>>>> general@ list.
> >>>>>>>>
> >>>>>>>> - Henry
> >>>>>>>>
> >>>>>>>> On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell <
> >>> pwendell@gmail.com
> >>>>>>>
> >>>>>>> wrote:
> >>>>>>>>> Hey Henry,
> >>>>>>>>>
> >>>>>>>>> One thing a lot of people do during the vote is
test the
> >>> binaries and
> >>>>>>>>> make sure they work. This is really valuable. If
you'd like
> >>>>>>>>> I
> >>> could
> >>>>>>>>> add a caveat to the vote thread explaining that
we are only
> >>> voting on
> >>>>>>>>> the source.
> >>>>>>>>>
> >>>>>>>>> - Patrick
> >>>>>>>>>
> >>>>>>>>> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra
<
> >>>>>>> henry.saputra@gmail.com> wrote:
> >>>>>>>>>> Actually we should be fine putting the binaries
there as
> >>>>>>>>>> long
> >>> as the
> >>>>>>>>>> VOTE is for the source.
> >>>>>>>>>>
> >>>>>>>>>> Let's verify with sebb in the general@ list
about his concern.
> >>>>>>>>>>
> >>>>>>>>>> - Henry
> >>>>>>>>>>
> >>>>>>>>>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra
<
> >>>>>>> henry.saputra@gmail.com> wrote:
> >>>>>>>>>>> Hi Patrick, as sebb has mentioned let's
move the binaries
> >>>>>>>>>>> from
> >>> the
> >>>>>>>>>>> voting directory in your people.apache.org
directory.
> >>>>>>>>>>> ASF release voting is for source code and
not binaries,
> >>>>>>>>>>> and technically we provide binaries for
convenience.
> >>>>>>>>>>>
> >>>>>>>>>>> And add link to the KEYS location in the
dist[1] to let
> >>>>>>>>>>> verify
> >>>>>>> signatures.
> >>>>>>>>>>>
> >>>>>>>>>>> Sorry for the late response to the VOTE
thread, guys.
> >>>>>>>>>>>
> >>>>>>>>>>> - Henry
> >>>>>>>>>>>
> >>>>>>>>>>> [1]
> >>>>>> https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick
Wendell <
> >>>>>> pwendell@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>>>>> The vote is now closed. This vote passes
with 5 PPMC +1's
> >>>>>>>>>>>> and
> >>> no 0
> >>>>>>> or -1
> >>>>>>>>>>>> votes.
> >>>>>>>>>>>>
> >>>>>>>>>>>> +1 (5 Total)
> >>>>>>>>>>>> Matei Zaharia*
> >>>>>>>>>>>> Nick Pentreath*
> >>>>>>>>>>>> Patrick Wendell*
> >>>>>>>>>>>> Prashant Sharma*
> >>>>>>>>>>>> Tom Graves*
> >>>>>>>>>>>>
> >>>>>>>>>>>> 0 (0 Total)
> >>>>>>>>>>>>
> >>>>>>>>>>>> -1 (0 Total)
> >>>>>>>>>>>>
> >>>>>>>>>>>> * = Binding Vote
> >>>>>>>>>>>>
> >>>>>>>>>>>> As per the incubator release guide [1]
I'll be sending
> >>>>>>>>>>>> this
> >>> to the
> >>>>>>>>>>>> general incubator list for a final vote
from IPMC members.
> >>>>>>>>>>>>
> >>>>>>>>>>>> [1]
> >>>>>>>>>>>>
> >>>>>>>
> >>>>>>
> >>> http://incubator.apache.org/guides/releasemanagement.html#best-practi
> >>> ce-incubator-release-
> >>>>>>>>>>>> vote
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Thu, Dec 12, 2013 at 8:59 AM, Evan
Chan
> >>>>>>>>>>>> <ev@ooyala.com>
> >>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> I'd be personally fine with a standard
workflow of
> >>> assemble-deps
> >>>>>> +
> >>>>>>>>>>>>> packaging just the Spark files as
separate packages, if
> >>>>>>>>>>>>> it
> >>>>>> speeds up
> >>>>>>>>>>>>> everyone's development time.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Wed, Dec 11, 2013 at 1:10 PM,
Mark Hamstra <
> >>>>>>> mark@clearstorydata.com
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> I don't know how to make sense
of the numbers, but
> >>>>>>>>>>>>>> here's
> >>> what
> >>>>>>> I've got
> >>>>>>>>>>>>>> from a very small sample size.
> >>>>>>
> >>>>>
> >>>>>
> >>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message