bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@apache.org>
Subject Re: Bigtop for Goraci
Date Fri, 07 Jun 2013 06:20:04 GMT
Hi Lewis.

very interesting stuff, indeed. I am a little bit confused though: when you
are saying "does anyone have an idea about how we can get this working in
tandem" what parts needs to be coupled here? The suite and DB implementations?

Or is it something else? Appreciate more info.

Regards,
  Cos

On Thu, Jun 06, 2013 at 11:08PM, Lewis John Mcgibbney wrote:
> Hi Bigtopers,
> 
> I am over here asking questions as I am a bit lost right now.
> Over in Gora we have a real nice test suite called Goraci [0]. Basically
> the test runs many ingest clients that continually create linked lists
> containing 25 million nodes. At some point the clients are stopped and a
> map reduce job is run to ensure no linked list has a hole. A hole indicates
> data was lost. Gennerally speaking the more nodes in the cluster, the
> better a chance there is of us finding that data is lost.
> 
> Now for part two... in Gora we currently have datastore implementations for
> Accumulo, Avro, Cassandra, HBase and Amazon Dynamodb... what we don not
> have, is a mechanism to run the ingestion test against each datastore as a
> controlled job meaning that we can subsequently gather metrics and infer
> behaviour across Gora datastores.
> 
> I have not gone to our friends @Infra yet as I would rather do my homework
> first and exhaust the avenues where I could contribute to getting this off
> of the ground.
> 
> My questions are therefore very very simple... does anyone have an idea
> about how we can get this working in tandem? Is this prime territory for
> Bigtop?
> 
> Thanks very much in advance.
> Best
> Lewis
> 
> [0] https://github.com/keith-turner/goraci
> 
> -- 
> *Lewis*

Mime
View raw message