bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roman Shaposhnik <>
Subject Re: New jiras (thanks) :)
Date Sat, 05 Oct 2013 00:20:06 GMT
Hi Jay,

thanks for the kind words. Here's one thing that I wanted to comment on:

On Wed, Sep 25, 2013 at 6:47 AM, Jay Vyas <> wrote:
> Thanks for all these new jiras focused on end users roman / cons !!(documentation, simplifying
hive, jenkins)...
> The hive one is sooo true: it's annoying that I need a hive server running to do smoke
> Would like to just smoke test hive ETL without the extra stuff for simpler clusters.
> I think in general there should be different types of smokes so that people don't require
a whole
> installed infrastructure to run smokes.  Although the vendor distros expect all services
> individual users or small startups don't necessarily run all the ecosystem services ,
> they just run the bare Apis, and smoke testing those is still of value on a cluster.

I think this boils down to a few fundamental improvements that we'd have to have
in our test framework to make it flexible:
    * let tests declare (via annotations or somehow else) what
environment they expect
    * have more topologies defined via puppet or similar provisioning frameworks
    * embrace containers or similar to not require a full fledged VMs

The above 3 would get us a lot of mileage.

> We do that instead of running the complex hive ETL. For pig we do something's similar
> since we want to test basic pig API and not only the new .11 Apis, which are the only
> supported currently in bigtop .
> Anyways.. Thanks for keeping an eye out in this! Jiras like this will make bigtop more
> accessible to the masses !:)

I think its all about more and more folks feeling compelled to contribute small
incremental improvements. Lets hope this will continue!


View raw message