hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Cafarella" <michael.cafare...@gmail.com>
Subject Re: HBase Design Ideas, terminology
Date Mon, 05 Jun 2006 08:10:44 GMT
Hi Stefan,

I'm continuing to plug away at code myself.  My goal is to have a
fully-working
(though feature-poor) system before committing any code.  I don't want to
clutter
the Hadoop workspace with experimental stuff.  (I feel like I did that a
little
too much in the early days of DFS, and I want to avoid that experience ;)

A wiki for current docs, status, etc, is a good idea.  I'll look to add that
to the Hadoop pages.  (Also, let's talk offline about how we can best work
together
on this.  Once the first commit is finished, then I think it should enter
the general bug-patch-commit development mode.)

Storing namenode and MapReduce status in a BigTable-like structure is
intriguing,
but seems dangerous.  (All the database stuff is built on top of DFS
and MapReduce.  What happens when the database needs a file in DFS,
which needs a record from the database?  Oh lord...)

--Mike

On 5/30/06, Stefan Groschupf <sg@media-style.com> wrote:
>
> Hi Michael,
>
> what you think are the next steps?
> I really would love to help in the implementation,
> ideally we define interfaces or create lists of functionality and may
> be people can provide implementations for the different components.
> Or is your idea to provide a first implementation and continue on a
> patch file kind of development?
>
> What you think to create a wiki page were we at least list all
> components within a short description and list of functionalities.
> We can start with your emails and discuss each component in detail
> here in the list.
>
> Makes that sense?
>
> BTW,
> Yoram and me discussed that a kind of big table could be a
> interesting storage for data that are hold in memory in jobtracker
> and namenode and cause some scalability problems.
> What you think?
>
>
> Greetings,
> Stefan
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message