spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pascal Voitot Dev <pascal.voitot....@gmail.com>
Subject Re: proposal: replace lift-json with spray-json
Date Fri, 14 Feb 2014 22:48:56 GMT
No worry!
I didn't want to be too harsh too :D

upper-case was just to make it clear!

Pascal

On Fri, Feb 14, 2014 at 11:21 PM, Evan Chan <ev@ooyala.com> wrote:

> Pascal,
>
> Ah, I stand corrected, thanks.
>
> On Mon, Feb 10, 2014 at 11:49 PM, Pascal Voitot Dev
> <pascal.voitot.dev@gmail.com> wrote:
> > Evan,
> >
> > Excuse me but that's WRONG that play-json pulls all play deps!
> >
> > PLAY/JSON has NO HEAVY DEP ON PLAY!
> >
> > I personally worked to make it an independent module in play!
> > So play/json has just one big dep which is Jackson!
> >
> > I agree that jackson is the right way to go as a beginning.
> > But for scala developers, a higher thin layer like play/json is useful to
> > bring typesafety...
> >
> > Pascal
> >
> > On Tue, Feb 11, 2014 at 1:31 AM, Evan Chan <ev@ooyala.com> wrote:
> >
> >> By the way, I did a benchmark on JSON parsing performance recently.
> >>  Based on that, spray-json was about 10x slower than the Jackson-based
> >> parsers.  I recommend json4s-jackson, because jackson is almost
> >> certainly already a dependency of Sparks (many other Java libraries
> >> use it), so the dependencies are very lightweight.   I didn't
> >> benchmark Argonaut or play-json, partly because play-json pulled in
> >> all the Play dependencies, tho as someone else commented in this
> >> thread, they plan to split it out.
> >>
> >> On Mon, Feb 10, 2014 at 12:22 AM, Pascal Voitot Dev
> >> <pascal.voitot.dev@gmail.com> wrote:
> >> > Hi again,
> >> >
> >> > If spark just need Json with serialization/deserialization of basic
> >> > structures and some potential simple validations for webUI, let's
> remind
> >> > that play/json (without any other dependency than Jackson) is about
> >> 200-300
> >> > lines of code... the only dependency is jackson which is the best json
> >> > parser that I know. The rest of code is about typesafety &
> composition...
> >> > If Spark need some json veryveryvery performant JSON
> (de)serialization,
> >> we
> >> > will have to look on things like pickling and potentially some
> streaming
> >> > parsers (I think this is a domain under work right now...)
> >> >
> >> > Pascal
> >> >
> >> >
> >> > On Mon, Feb 10, 2014 at 1:50 AM, Will Benton <willb@redhat.com>
> wrote:
> >> >
> >> >> Matei, sorry if I was unclear:  I'm referring to downstream operating
> >> >> system distributions (like Fedora or Debian) that  have policies
> >> requiring
> >> >> that all packages are built from source (using only tools already
> >> packaged
> >> >> in the distribution).  So end-users (and distributions with different
> >> >> policies) don't have to build Lift to get the lift-json artifact, but
> >> it is
> >> >> a concern for many open-source communities.
> >> >>
> >> >> best,
> >> >> wb
> >> >>
> >> >>
> >> >> ----- Original Message -----
> >> >> > From: "Matei Zaharia" <matei.zaharia@gmail.com>
> >> >> > To: dev@spark.incubator.apache.org
> >> >> > Sent: Sunday, February 9, 2014 4:29:20 PM
> >> >> > Subject: Re: proposal:  replace lift-json with spray-json
> >> >> >
> >> >> > Will, why are you saying that downstream distributes need to build
> >> all of
> >> >> > Lift to package lift-json? Spark just downloads it from Maven
> Central,
> >> >> where
> >> >> > it's a JAR with no external dependencies. We don't have any
> >> dependency on
> >> >> > the rest of lift.
> >> >> >
> >> >> > Matei
> >> >> >
> >> >> > On Feb 9, 2014, at 11:28 AM, Will Benton <willb@redhat.com>
wrote:
> >> >> >
> >> >> > > lift-json is a nice library, but Lift is a pretty heavyweight
> >> >> dependency to
> >> >> > > track just for its JSON support.  (lift-json is relatively
> >> >> self-contained
> >> >> > > as a dependency from an end-user's perspective, but downstream
> >> >> > > distributors need to build all of Lift in order to package
the
> JSON
> >> >> > > support.)  I understand that this has come up before (cf.
> SPARK-883)
> >> >> and
> >> >> > > that the uncertain future of JSON support in the Scala standard
> >> >> library is
> >> >> > > the motivator for relying on an external library.
> >> >> > >
> >> >> > > I'm proposing replacing lift-json in Spark with something
more
> >> >> lightweight.
> >> >> > > I've evaluated apparent project liveness and dependency scope
for
> >> most
> >> >> of
> >> >> > > the current Scala JSON libraries and believe the best candidate
> is
> >> >> > > spray-json (https://github.com/spray/spray-json), the JSON
> library
> >> >> used by
> >> >> > > the Spray HTTP toolkit. spray-json is Apache-licensed, actively
> >> >> developed,
> >> >> > > and builds and works independently of Spray with only one
> external
> >> >> > > dependency.
> >> >> > >
> >> >> > > It looks to me like a pretty straightforward change (although
> >> >> > > JsonProtocol.scala would be a little more verbose since it
> couldn't
> >> use
> >> >> > > the Lift JSON DSL), and I'd like to do it.  I'm writing now
to
> ask
> >> for
> >> >> > > some community feedback before making the change (and submitting
> a
> >> JIRA
> >> >> > > and PR).  If no one has any serious objections (to the effort
in
> >> >> general
> >> >> > > or to to the choice of spark-json in particular), I'll go
ahead
> and
> >> do
> >> >> it,
> >> >> > > but if anyone has concerns, I'd be happy to discuss and address
> them
> >> >> > > before getting started.
> >> >> > >
> >> >> > >
> >> >> > > thanks,
> >> >> > > wb
> >> >> >
> >> >> >
> >> >>
> >>
> >>
> >>
> >> --
> >> --
> >> Evan Chan
> >> Staff Engineer
> >> ev@ooyala.com  |
> >>
>
>
>
> --
> --
> Evan Chan
> Staff Engineer
> ev@ooyala.com  |
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message