hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Hsieh <...@cloudera.com>
Subject Re: [Shadow Regions / Read Replicas ] Wal per region?
Date Wed, 04 Dec 2013 01:59:13 GMT
On Tue, Dec 3, 2013 at 3:07 PM, Enis Söztutar <enis.soz@gmail.com> wrote:

> On Tue, Dec 3, 2013 at 2:03 PM, Jonathan Hsieh <jon@cloudera.com> wrote:
>
> > On Tue, Dec 3, 2013 at 11:42 AM, Enis Söztutar <enis.soz@gmail.com>
> wrote:
> >
> > > On Mon, Dec 2, 2013 at 10:20 PM, Jonathan Hsieh <jon@cloudera.com>
> > wrote:
> > >
> > > > > Deveraj:
> > > > > Jonathan Hsieh, WAL per region (WALpr) would give you the locality
> > (and
> > > > hence HDFS short
> > > > > circuit) of reads if you were to couple it with the favored nodes.
> > The
> > > > cost is of course more WAL
> > > > > files... In the current situation (no WALpr) it would create quite
> > some
> > > > traffic cross machine, no?
> > > >
> > > > I think we all agree that wal per region isn't efficient on today's
> > > > spinning hard drive world where we are limited to a relatively low
> > budget
> > > > or seeks (though may be better in the future with SSD's).
> > > >
> > >
> > > WALpr makes sense in fully SSD world and if hdfs had journaling for
> > writes.
> > > I don't think anybody
> > > is working on this yet.
> >
> >
> > what do you mean by journaling for writes?  do you mean where sync
> > operations update length at the nn on every call?
> >
>
> I think hdfs guys were using "super sync" for referring to that. I was
> referring to
> journaling file system (
> http://en.wikipedia.org/wiki/Journaling_file_system)
> where the writes to
> multiple files are persisted to a journal disk so that you do not pay the
> constant seeks for writing to
> a lot of files (for regions wals) in parallel.
>
>
>

Wait, we have a system that provides the ability to write data for a bunch
of buckets to a particular disk before rewriting them to others in a split
out read optimized from..
Isn't this what HBase and its HLog basically provides?  :)

Joking aside, can you give a quick example of the semantics it would have
so I can grok what you are talking about?

Jon

-- 
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message