hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <lhofha...@yahoo.com>
Subject Re: hbase 0.94.0
Date Fri, 03 Feb 2012 01:43:59 GMT
I think we all agree on that. :)

The sticky point is: When can we remove old code that is *only* needed to migrate even older
installations? And should we spent a lot of our resources on supporting upgrades that skip

I think that ties in with time-based released and the point that Ted made earlier that time-based
released are hard to swallow until we have cross-version wire-compatibility.
If we had that, one could upgrade a cluster from (say) 0.89fb to 0.96, but deploying 0.92,
then 0.94, then 0.96 all with controlled rolling restarts. Sure, tedious, but with wire-compatibility
it would be transparent to the application.

We need to weigh that inconvenience against the resources needed of supporting and testing
multi version jumps.

-- Lars

 From: Nicolas Spiegelberg <nspiegelberg@fb.com>
To: "dev@hbase.apache.org" <dev@hbase.apache.org> 
Sent: Thursday, February 2, 2012 5:24 PM
Subject: Re: hbase 0.94.0
>>I think HFile upgrade in particular is more complicated than you think.
>>We currently have production traffic running with HFileV1.  It has a
>>SLA.  We can't afford to take the entire downtime to rewrite 100GB (or
>>whatever) worth of data.  We need to do this while the cluster is live.
>AFAIK that's how it's done, V1 files are being rewritten to V2 when a
>compaction happens. You don't have to do some offline processing
>before getting the cluster back online.

Correct.  Sorry for the confusion.  I just meant that we shouldn't get rid
of the HFileV1 Reader because it allows online upgrades to happen as
compactions happen.  Trying to remove this functionality or refactor this
to some transformation utility would have negative consequences.  LSMT is
designed with immutability as a core principle, so I think we should
incline towards creating online mutation of persistent data as opposed to
migration utilities.
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message