aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jake Farrell <jfarr...@apache.org>
Subject Re: Update on GSoC work for Aurora and some updates
Date Mon, 09 Jun 2014 14:11:54 GMT
Suro seems interesting, but it is a lot to add for a dependency that
everyone may not want or need. If scribe is all ready present in an
environment then to use it only introduces one new dependency, all the sub
dependencies of scribe are all ready part of Aurora. Chukwa and Flume could
also be considered, but like Suro would add a fair amount to the
dependencies. Since there are a number of options here I would say keeping
it flexible and allowing for interchangeability would be the best so Aurora
can fit into any existing environment easier

-Jake





On Mon, Jun 9, 2014 at 1:36 AM, Ahmed Ali <ahmed.aley@gmail.com> wrote:

> Hi Jake,
>
> Took me a while to retrieve my old Harddisk's data.
> Since we will only be interested in adding the client as a dependency and
> not the server, I have benchmarked the build operation when Suro-Client
> dep. is added to the gradle file and without it. They both take very
> similar times on average (with compiling with the suro-client dependency
> sometimes taking ~1 minute 20 seconds and without the suro client sometimes
> taking~1 minute 40 seconds but there average is quite similar).
> These are the suro client dependencies compared to Aurora dependencies.
> please correct me if I am wrong on any of these :
> Group                          Artifact
> Compared to Aurora
>
> com.fasterxml.jackson.core
>  <http://mvnrepository.com/artifact/com.fasterxml.jackson.core>
> jackson-annotations
> <
> http://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-annotations
> >
> Aurora but  an older version org.codehaus.jackson
>  com.fasterxml.jackson.core
> <http://mvnrepository.com/artifact/com.fasterxml.jackson.core>
> jackson-core
> <http://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-core
> >
> trans.
> dependency for Aurora  com.fasterxml.jackson.core
> <http://mvnrepository.com/artifact/com.fasterxml.jackson.core>
> jackson-databind
> <
> http://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind
> >
> trans. dependency for Aurora  com.fasterxml.jackson.datatype
> <http://mvnrepository.com/artifact/com.fasterxml.jackson.datatype>
> jackson-datatype-guava
> <
> http://mvnrepository.com/artifact/com.fasterxml.jackson.datatype/jackson-datatype-guava
> >
> Not
> in Aurora  com.google.guava
> <http://mvnrepository.com/artifact/com.google.guava> guava
> <http://mvnrepository.com/artifact/com.google.guava/guava> in Aurora
>  com.leansoft <http://mvnrepository.com/artifact/com.leansoft> bigqueue
> <http://mvnrepository.com/artifact/com.leansoft/bigqueue> Not in Aurora
>  com.netflix.archaius
> <http://mvnrepository.com/artifact/com.netflix.archaius> archaius-core
> <http://mvnrepository.com/artifact/com.netflix.archaius/archaius-core> Not
> in Aurora  com.netflix.eureka
> <http://mvnrepository.com/artifact/com.netflix.eureka> eureka-client
> <http://mvnrepository.com/artifact/com.netflix.eureka/eureka-client> Not
> in
> Aurora  com.netflix.governator
> <http://mvnrepository.com/artifact/com.netflix.governator> governator
> <http://mvnrepository.com/artifact/com.netflix.governator/governator> Not
> in Aurora  com.netflix.netflix-commons
> <http://mvnrepository.com/artifact/com.netflix.netflix-commons>
> netflix-commons-util
> <
> http://mvnrepository.com/artifact/com.netflix.netflix-commons/netflix-commons-util
> >
> Not
> in Aurora  com.netflix.ribbon
> <http://mvnrepository.com/artifact/com.netflix.ribbon> ribbon-core
> <http://mvnrepository.com/artifact/com.netflix.ribbon/ribbon-core> Not in
> Aurora  com.netflix.ribbon
> <http://mvnrepository.com/artifact/com.netflix.ribbon> ribbon-eureka
> <http://mvnrepository.com/artifact/com.netflix.ribbon/ribbon-eureka> Not
> in
> Aurora  com.netflix.servo
> <http://mvnrepository.com/artifact/com.netflix.servo> servo-core
> <http://mvnrepository.com/artifact/com.netflix.servo/servo-core> Not in
> Aurora  com.netflix.suro
> <http://mvnrepository.com/artifact/com.netflix.suro> suro-core
> <http://mvnrepository.com/artifact/com.netflix.suro/suro-core> Not in
> Aurora
> com.ning <http://mvnrepository.com/artifact/com.ning> compress-lzf
> <http://mvnrepository.com/artifact/com.ning/compress-lzf> Not in Aurora
> commons-io <http://mvnrepository.com/artifact/commons-io> commons-io
> <http://mvnrepository.com/artifact/commons-io/commons-io>  trans.
> dependency for Aurora  joda-time
> <http://mvnrepository.com/artifact/joda-time> joda-time
> <http://mvnrepository.com/artifact/joda-time/joda-time>  trans. dependency
> for Aurora  junit <http://mvnrepository.com/artifact/junit> junit
> <http://mvnrepository.com/artifact/junit/junit>  in Aurora
>  log4j <http://mvnrepository.com/artifact/log4j> log4j
> <http://mvnrepository.com/artifact/log4j/log4j> in Aurora
>  org.apache.hadoop
> <http://mvnrepository.com/artifact/org.apache.hadoop> hadoop-core
> <http://mvnrepository.com/artifact/org.apache.hadoop/hadoop-core>  will be
> probably removed from Suro since they have an open ticket to do that
> org.apache.thrift <http://mvnrepository.com/artifact/org.apache.thrift>
> libthrift <http://mvnrepository.com/artifact/org.apache.thrift/libthrift>
> in
> Aurora  org.mockito <http://mvnrepository.com/artifact/org.mockito>
> mockito-core <http://mvnrepository.com/artifact/org.mockito/mockito-core>
> Not in Aurora  org.slf4j <http://mvnrepository.com/artifact/org.slf4j>
> slf4j-api <http://mvnrepository.com/artifact/org.slf4j/slf4j-api>  in
> Aurora
>
> Some of the transitive dependencies for Aurora and Suro require different
> versions, in almost all cases I go with the version used by the aurora
> dependencies.
> I am still open to both options, having it as a dependency or as a plug-in.
> The main advantages of having it as a dependency is the easiness of usage.
> The main disadvantage is the increase in the number of dependencies (and
> size). What do people feel about the two approaches?
>
> Best,
> --Ahmed
>
>
> On Fri, Jun 6, 2014 at 12:25 AM, Ahmed Ali <ahmed.aley@gmail.com> wrote:
>
> > Hi,
> > Let me do some testing tomorrow and get back with an answer. I think you
> > have a good point.
> >
> > Thanks,
> > Ahmed
> > On Jun 5, 2014 10:44 PM, "Jake Farrell" <jfarrell@apache.org> wrote:
> >
> >> Hey Ahmed
> >> This sounds like it would be a good addition as a plugin, Suro has a
> fair
> >> amount of dependencies and looks like it can connect via a log4j
> appender
> >> [1], not sure if it needs to be a direct dependency
> >>
> >> -Jake
> >>
> >> [1]:
> >> https://github.com/Netflix/suro/wiki/Suro-log4j-appender-configuration
> >>
> >>
> >> On Thu, Jun 5, 2014 at 3:45 PM, Ahmed Ali <ahmed.aley@gmail.com> wrote:
> >>
> >> > Hi,
> >> >
> >> > Sorry for the late update!
> >> > I have been mostly occupied with making both Suro and Aurora work
> >> together.
> >> > Now the testing is done on my local machine.
> >> >
> >> > The plan in to file a few tickets in Jira detailing every change I
> will
> >> do.
> >> > I will add Suro as a dependency, add it to the vagrant script, add
> >> logging
> >> > to Suro all over the client and the scheduler, and make sure there is
> >> > proper documentation for how to use Suro not just for Aurora, but for
> >> > anything running on top or below ( jobs, tasks...). I will file the
> >> tickets
> >> > , link to them and link to code in my next update. My hard-disk went
> to
> >> > Limbo land late today so I am sending my update from my phone. The
> good
> >> > news is that I already have the new hard-disk and working on
> recovering
> >> the
> >> > data. Hopefully will be back to speed by tomorrow evening and send an
> >> > update after the weekend with link to code.
> >> >
> >> > Best,
> >> > --Ahmed
> >> > On May 28, 2014 12:27 AM, "Ahmed Aley" <ahmeda@cs.umu.se> wrote:
> >> >
> >>
> >
>
>
> --
> Ahmed Ali ElDin M. Hassan,  PhD  Student
> Department of Computing Science
> Umea University, SE-901 87 UmeƄ, Sweden
> ahmeda@cs.umu.se
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message