aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ahmed Ali <ahmed.a...@gmail.com>
Subject Re: Update on GSoC work for Aurora and some updates
Date Mon, 09 Jun 2014 05:36:18 GMT
Hi Jake,

Took me a while to retrieve my old Harddisk's data.
Since we will only be interested in adding the client as a dependency and
not the server, I have benchmarked the build operation when Suro-Client
dep. is added to the gradle file and without it. They both take very
similar times on average (with compiling with the suro-client dependency
sometimes taking ~1 minute 20 seconds and without the suro client sometimes
taking~1 minute 40 seconds but there average is quite similar).
These are the suro client dependencies compared to Aurora dependencies.
please correct me if I am wrong on any of these :
Group                          Artifact
Compared to Aurora

com.fasterxml.jackson.core
 <http://mvnrepository.com/artifact/com.fasterxml.jackson.core>
jackson-annotations
<http://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-annotations>
Aurora but  an older version org.codehaus.jackson
 com.fasterxml.jackson.core
<http://mvnrepository.com/artifact/com.fasterxml.jackson.core> jackson-core
<http://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-core>
trans.
dependency for Aurora  com.fasterxml.jackson.core
<http://mvnrepository.com/artifact/com.fasterxml.jackson.core>
jackson-databind
<http://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind>
trans. dependency for Aurora  com.fasterxml.jackson.datatype
<http://mvnrepository.com/artifact/com.fasterxml.jackson.datatype>
jackson-datatype-guava
<http://mvnrepository.com/artifact/com.fasterxml.jackson.datatype/jackson-datatype-guava>
Not
in Aurora  com.google.guava
<http://mvnrepository.com/artifact/com.google.guava> guava
<http://mvnrepository.com/artifact/com.google.guava/guava> in Aurora
 com.leansoft <http://mvnrepository.com/artifact/com.leansoft> bigqueue
<http://mvnrepository.com/artifact/com.leansoft/bigqueue> Not in Aurora
 com.netflix.archaius
<http://mvnrepository.com/artifact/com.netflix.archaius> archaius-core
<http://mvnrepository.com/artifact/com.netflix.archaius/archaius-core> Not
in Aurora  com.netflix.eureka
<http://mvnrepository.com/artifact/com.netflix.eureka> eureka-client
<http://mvnrepository.com/artifact/com.netflix.eureka/eureka-client> Not in
Aurora  com.netflix.governator
<http://mvnrepository.com/artifact/com.netflix.governator> governator
<http://mvnrepository.com/artifact/com.netflix.governator/governator> Not
in Aurora  com.netflix.netflix-commons
<http://mvnrepository.com/artifact/com.netflix.netflix-commons>
netflix-commons-util
<http://mvnrepository.com/artifact/com.netflix.netflix-commons/netflix-commons-util>
Not
in Aurora  com.netflix.ribbon
<http://mvnrepository.com/artifact/com.netflix.ribbon> ribbon-core
<http://mvnrepository.com/artifact/com.netflix.ribbon/ribbon-core> Not in
Aurora  com.netflix.ribbon
<http://mvnrepository.com/artifact/com.netflix.ribbon> ribbon-eureka
<http://mvnrepository.com/artifact/com.netflix.ribbon/ribbon-eureka> Not in
Aurora  com.netflix.servo
<http://mvnrepository.com/artifact/com.netflix.servo> servo-core
<http://mvnrepository.com/artifact/com.netflix.servo/servo-core> Not in
Aurora  com.netflix.suro
<http://mvnrepository.com/artifact/com.netflix.suro> suro-core
<http://mvnrepository.com/artifact/com.netflix.suro/suro-core> Not in Aurora
com.ning <http://mvnrepository.com/artifact/com.ning> compress-lzf
<http://mvnrepository.com/artifact/com.ning/compress-lzf> Not in Aurora
commons-io <http://mvnrepository.com/artifact/commons-io> commons-io
<http://mvnrepository.com/artifact/commons-io/commons-io>  trans.
dependency for Aurora  joda-time
<http://mvnrepository.com/artifact/joda-time> joda-time
<http://mvnrepository.com/artifact/joda-time/joda-time>  trans. dependency
for Aurora  junit <http://mvnrepository.com/artifact/junit> junit
<http://mvnrepository.com/artifact/junit/junit>  in Aurora
 log4j <http://mvnrepository.com/artifact/log4j> log4j
<http://mvnrepository.com/artifact/log4j/log4j> in Aurora  org.apache.hadoop
<http://mvnrepository.com/artifact/org.apache.hadoop> hadoop-core
<http://mvnrepository.com/artifact/org.apache.hadoop/hadoop-core>  will be
probably removed from Suro since they have an open ticket to do that
org.apache.thrift <http://mvnrepository.com/artifact/org.apache.thrift>
libthrift <http://mvnrepository.com/artifact/org.apache.thrift/libthrift> in
Aurora  org.mockito <http://mvnrepository.com/artifact/org.mockito>
mockito-core <http://mvnrepository.com/artifact/org.mockito/mockito-core>
Not in Aurora  org.slf4j <http://mvnrepository.com/artifact/org.slf4j>
slf4j-api <http://mvnrepository.com/artifact/org.slf4j/slf4j-api>  in
Aurora

Some of the transitive dependencies for Aurora and Suro require different
versions, in almost all cases I go with the version used by the aurora
dependencies.
I am still open to both options, having it as a dependency or as a plug-in.
The main advantages of having it as a dependency is the easiness of usage.
The main disadvantage is the increase in the number of dependencies (and
size). What do people feel about the two approaches?

Best,
--Ahmed


On Fri, Jun 6, 2014 at 12:25 AM, Ahmed Ali <ahmed.aley@gmail.com> wrote:

> Hi,
> Let me do some testing tomorrow and get back with an answer. I think you
> have a good point.
>
> Thanks,
> Ahmed
> On Jun 5, 2014 10:44 PM, "Jake Farrell" <jfarrell@apache.org> wrote:
>
>> Hey Ahmed
>> This sounds like it would be a good addition as a plugin, Suro has a fair
>> amount of dependencies and looks like it can connect via a log4j appender
>> [1], not sure if it needs to be a direct dependency
>>
>> -Jake
>>
>> [1]:
>> https://github.com/Netflix/suro/wiki/Suro-log4j-appender-configuration
>>
>>
>> On Thu, Jun 5, 2014 at 3:45 PM, Ahmed Ali <ahmed.aley@gmail.com> wrote:
>>
>> > Hi,
>> >
>> > Sorry for the late update!
>> > I have been mostly occupied with making both Suro and Aurora work
>> together.
>> > Now the testing is done on my local machine.
>> >
>> > The plan in to file a few tickets in Jira detailing every change I will
>> do.
>> > I will add Suro as a dependency, add it to the vagrant script, add
>> logging
>> > to Suro all over the client and the scheduler, and make sure there is
>> > proper documentation for how to use Suro not just for Aurora, but for
>> > anything running on top or below ( jobs, tasks...). I will file the
>> tickets
>> > , link to them and link to code in my next update. My hard-disk went to
>> > Limbo land late today so I am sending my update from my phone. The good
>> > news is that I already have the new hard-disk and working on recovering
>> the
>> > data. Hopefully will be back to speed by tomorrow evening and send an
>> > update after the weekend with link to code.
>> >
>> > Best,
>> > --Ahmed
>> > On May 28, 2014 12:27 AM, "Ahmed Aley" <ahmeda@cs.umu.se> wrote:
>> >
>>
>


-- 
Ahmed Ali ElDin M. Hassan,  PhD  Student
Department of Computing Science
Umea University, SE-901 87 UmeƄ, Sweden
ahmeda@cs.umu.se

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message