hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: Current security implementation in Hadoop
Date Thu, 23 Jul 2009 18:11:22 GMT
Your architecture is a bit unusual in that you seem to be proposing that
users get direct access to the hadoop storage layer.

More common is to have a controller layer that mediates requests to store or
read data.

With that layer of abstraction, you can deal with some of the problems
associated with file update.  See the recent hbase work, for instance.

Even with that layer of abstraction and the recent massive improvements to
hbase, Hadoop still tends to be much better for batch processing rather than
real-time support of ad hoc user data reads and writes.  Depending on the
data you have and the update patterns, you might be much happier with a
clustered key-value store like Voldemort or Cassandra.  Voldemort especially
has very nice capabilities for dumping large amounts of data from hadoop
into a large store.  It also works to support real-time (ish) random reads
and writes.

On Thu, Jul 23, 2009 at 6:44 AM, Giovanni Tusa <giovantus@gmail.com> wrote:

> Could you also suggest me some other useful links, maybe with examples if
> any, on how to implement such a mechanism?

Ted Dunning, CTO

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message