bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@apache.org>
Subject Re: Terasort/Teragen in smokes
Date Wed, 28 Aug 2013 04:02:12 GMT
On Tue, Aug 27, 2013 at 10:18PM, Jay Vyas wrote:
> hmm ok. now. Thinking about teragen makes me think of benchmarking..
> 
>  In the longer term we could add benchmarking jobs to all the submodules not
>  just mapreduce.  For example there are hi bench and ycsb workloads which
>  might be usable or pulled in as bigtop components ... Iff of course
>  benchmarking is in the cards for bigtop?

It indeed is!

I think doing tera-gen/sort so it can be parameterized will provide a good
basis for future benchmarking (as a bit of reflection: I have did simplistic
yet efficient way of benchmarking HDFS and MR a couple years ago, but my
employer back then has never let it go into the open. Go figure...)

And I have a way of building YCSB against a particular version of Hadoop, so I
guess I will have it packaged as a benchmarking test pretty soon.

Cos

> On Aug 27, 2013, at 7:39 PM, Roman Shaposhnik <rvs@apache.org> wrote:
> 
> > On Tue, Aug 27, 2013 at 4:26 PM, Jay Vyas <jayunit100@gmail.com> wrote:
> >> Hi guys:
> >> 
> >> I run TeraSort/TeraGen as additions to bigtop in some shell scripts.
> >> 
> >> Any interest in these as an update to TestHadoopExamples in the MapReduce
> >> smokes?
> >> 
> >> If so I could patch them in :)
> > 
> > Sure! Sounds like a useful addition.
> > 
> > Thanks,
> > Roman.

Mime
View raw message