Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of eelco.hillenius@gmail.com
 designates 209.85.211.198 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:from:date:message-id:subject:to:content-type
         :content-transfer-encoding;
        b=n6OOK0ivGQbFvmQY9imjmh6M/noOO8hx1YCkoW9JS3LEkKrVmzELyBsHD/V6hfT6cK
         HD3a6JVuGYrJ14FT37oj7hagADKyirHwSYeUTgHww/gVvTIgfxyDSqY8ee3xbdt24B9J
         b84bANSkYrzwF8J2rDtjvmIYNSPTusBbq+yss=
MIME-Version: 1.0
From: Eelco Hillenius <eelco.hillenius@gmail.com>
Date: Fri, 14 Aug 2009 12:30:09 -0700
Message-ID: <ceb609d80908141230w5945ddcdn6f10f92f0e97c04a@mail.gmail.com>
Subject: comparing sub/ 3rd party projects that abstract map/reduce
To: general@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Hi,

Would people mind sharing their opinions about the relative strengths
and weaknesses of Hive, Chukwa and Pig (and possibly other
alternatives)?

I'm only just getting into Hadoop, but it seems to me these libs have
a considerable overlap depending on what you use them for? I plan to
check those sub projects out anyway, and I'm definitively not trying
to start a flame war here, but it would be great to hear some opinions
about what areas these projects are particularly useful for and what
might need some work etc.

As a bit of context, we (Teachscape) are considering Hadoop for
storing audit log files and extracting information from them. The
audit logging we do is application specific, e.g. user Foo deleted
Survey X, user Bar moved organization node AAB to AB, etc, and besides
the need to run a couple of fixed reports weekly (mainly that give our
customers some insight in how they are using our application), I
expect us to want to create queries on the fly to e.g. track down
problems. I don't expect our developers to have a problem writing Map/
Reduce programs, but I do like the idea of a higher level way of
extracting information.

Any thoughts would be greatly appreciated,

Eelco