hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Cafarella" <michael.cafare...@gmail.com>
Subject Re: HBase Design Ideas, terminology
Date Mon, 05 Jun 2006 08:10:44 GMT
Hi Stefan,

I'm continuing to plug away at code myself.  My goal is to have a
(though feature-poor) system before committing any code.  I don't want to
the Hadoop workspace with experimental stuff.  (I feel like I did that a
too much in the early days of DFS, and I want to avoid that experience ;)

A wiki for current docs, status, etc, is a good idea.  I'll look to add that
to the Hadoop pages.  (Also, let's talk offline about how we can best work
on this.  Once the first commit is finished, then I think it should enter
the general bug-patch-commit development mode.)

Storing namenode and MapReduce status in a BigTable-like structure is
but seems dangerous.  (All the database stuff is built on top of DFS
and MapReduce.  What happens when the database needs a file in DFS,
which needs a record from the database?  Oh lord...)


On 5/30/06, Stefan Groschupf <sg@media-style.com> wrote:
> Hi Michael,
> what you think are the next steps?
> I really would love to help in the implementation,
> ideally we define interfaces or create lists of functionality and may
> be people can provide implementations for the different components.
> Or is your idea to provide a first implementation and continue on a
> patch file kind of development?
> What you think to create a wiki page were we at least list all
> components within a short description and list of functionalities.
> We can start with your emails and discuss each component in detail
> here in the list.
> Makes that sense?
> BTW,
> Yoram and me discussed that a kind of big table could be a
> interesting storage for data that are hold in memory in jobtracker
> and namenode and cause some scalability problems.
> What you think?
> Greetings,
> Stefan

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message