Mailing-List: contact hbase-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of imyousuf@gmail.com designates
 209.85.212.48 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=d+ansk9txBAA5mZ12jP4b2yK9hU8tzRyWXLKFGwYjUxP8sOvrNfSWXifxmN1bMd4v5
         W/AhGkyYWMbIIQ8svcylJHZ764mEi9letFKZ7bTrmxhf4nC6aQxqAjbD85U6231u+lbK
         HEpcj6D5K/+wIhkolHaKfa4BHT9PkjfpRiccM=
MIME-Version: 1.0
In-Reply-To: 
 <8D66B74984F9564BBB25C3C67D630F2D6812E0E4@SC-MBXC1.TheFacebook.com>
References: <AANLkTikOejC7AcWesXKUEscis6ZjADKtdqJwbfFEiRhx@mail.gmail.com>
	 <8D66B74984F9564BBB25C3C67D630F2D6812DF45@SC-MBXC1.TheFacebook.com>
	 <AANLkTike_-PpOgE3lc4TPoB_dExAFN12bh3J9CYyvO0M@mail.gmail.com>
	 <8D66B74984F9564BBB25C3C67D630F2D6812E0E4@SC-MBXC1.TheFacebook.com>
Date: Mon, 17 May 2010 23:21:36 +0600
Message-ID: <AANLkTilpN9tH6mBIYQbmT_58uAkMh3ZI5zuu2m_4N8hj@mail.gmail.com>
Subject: Re: Availability Transaction and data integrity
From: Imran M Yousuf <imyousuf@gmail.com>
To: hbase-user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Ah ok. Thanks for clearing it JG.

Regards,

Imran

On Mon, May 17, 2010 at 11:10 PM, Jonathan Gray <jgray@facebook.com> wrote:
> What you read from the slides was about removing HBase's SPOF, not Hadoop=
's.
>
> From 0.20, HBase doesn't have a SPOF in that you can have backup masters.=
 =A0In more recent testing we've shown that while we support backup masters=
, there are still circumstances where the second master won't know what to =
do and your cluster could be left in an inconsistent state.
>
> That is being dealt with in 0.21 over in HBASE-2485.
>
> JG
>
>> -----Original Message-----
>> From: Imran M Yousuf [mailto:imyousuf@gmail.com]
>> Sent: Monday, May 17, 2010 10:01 AM
>> To: hbase-user@hadoop.apache.org
>> Subject: Re: Availability Transaction and data integrity
>>
>> Thanks, my answers are inline too.
>>
>> On Mon, May 17, 2010 at 9:50 PM, Jonathan Gray <jgray@facebook.com>
>> wrote:
>> > Answers inline.
>> >
>> >> <snip />
>> >> * We will go live from January 2011, in that time frame should we
>> >> develop using 0.21-SNAPSHOT or should we stick to 0.20.x? Ideally I
>> >> would not want to go ahead with a snapshot in production and also
>> >> would not want to make an upgrade within few months (because of some
>> >> problems noticed in the mailing list regarding upgrade and I am a
>> bit
>> >> skeptical about it in general).
>> >
>> > If you need data durability (no data loss under node failure) then
>> you have no choice but to go with 0.21 once it is released. =A0This is
>> not supported on the 0.20 line.
>> >
>> > There are a number of organizations who will be going live into
>> production on 0.21 in Q3 2010. =A0You can be sure that there will be a
>> very well tested and stable 0.21 release by January 2011.
>> >
>>
>> Extremely great! Then we are on the right track working with 0.21
>>
>> >> * Transaction was a contrib module of HBase but it seems recently
>> >> removed from the 0.21-SNAPSHOT. In light of it what would be the way
>> >> to achieve transaction?
>> >
>> > It is still available but is being moved to GitHub. =A0You can still
>> use it, it has just been moved out of the core code.
>> >
>> >> * NN was (if I am not mistaken) a SPoF, I also learnt that its
>> >> supposed to be fixed in 0.21, is that in trunk already?
>> >
>> > I'm not sure where you heard this was fixed in 0.21 as my
>> understanding is that it is not fixed in 0.21.
>> >
>> > There is work being done at Facebook (and I believe parallel work
>> being done elsewhere) to add a true backup NameNode. =A0Once stabilized
>> this will be released and available to the public though it may not be
>> put into an official Hadoop release in an 0.21 timeframe.
>> >
>>
>> I see, in that case I misunderstood the statement from Slide #10 of
>> "ApacheCon2009: Practical HBase" - 'Removed SPoF, multi-master w/
>> automatic failover (ZK)'. What was this statement referring to as
>> master?
>>
>> Great to learn that its being worked on..
>>
>> >> * What kind of data loss should we design to?
>> >
>> > On 0.21, you should not have data loss. =A0We are doing a lot of
>> testing on this to ensure stability and durability.
>> >
>>
>> This is extremely good news, it seems both my primary concerns are
>> getting addressed!
>>
>> >> * Is there any professional service provider who could help us train
>> >> for deployment, help optimize and in case we need emergency provide
>> >> service? (P.S. I contacted Cloudera via email 11 days back and still
>> >> waiting for a reply, may be they are not interested any alternate
>> >> would do great!)
>> >
>> > Cloudera should be able to help. =A0They're active on this list but
>> perhaps don't want to use this forum to sell services. =A0I would ping
>> them again off the list.
>> >
>>
>> Hmm, ok will try again with them.
>>
>> Thanks a lot!
>>
>> Imran
>>
>> >>
>> >> I eagerly hope for some help and guideline on these queries.
>> >>
>> >> --
>> >> Imran M Yousuf
>> >> Entrepreneur & Software Engineer
>> >> Smart IT Engineering
>> >> Dhaka, Bangladesh
>> >> Email: imran@smartitengineering.com
>> >> Blog: http://imyousuf-tech.blogs.smartitengineering.com/
>> >> Mobile: +880-1711402557
>> >
>>
>>
>>
>> --
>> Imran M Yousuf
>> Entrepreneur & Software Engineer
>> Smart IT Engineering
>> Dhaka, Bangladesh
>> Email: imran@smartitengineering.com
>> Blog: http://imyousuf-tech.blogs.smartitengineering.com/
>> Mobile: +880-1711402557
>


--=20
Imran M Yousuf
Entrepreneur & Software Engineer
Smart IT Engineering
Dhaka, Bangladesh
Email: imran@smartitengineering.com
Blog: http://imyousuf-tech.blogs.smartitengineering.com/
Mobile: +880-1711402557