Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: <CAAT7MkpBJ9LvQX16Mz3_uUnYXcmBe46mW9WFKkm9=L-8RkVvQw@mail.gmail.com>
References: <CALte62zeSZp=kG9B7x-3Zb=g2D5=VzgM1nLQV7q7CZs22het+Q@mail.gmail.com>
 <CAAAYAnOvB62xXpu5t7sx2tfsFkdXt6z09nbA3XDQ=74py-f2rA@mail.gmail.com>
 <CA90C159-F08C-43A3-8BCB-F4CF645C69CF@gmail.com> <CAAT7MkoJXARGE_VyZCtp25_ObZRg2gEvKVZeW2FZz4my0opG+w@mail.gmail.com>
 <CALte62zOS6A+zgKZAEGk4iW6yB0fvbQ5fhEwqm7PRf=nvLV_ZQ@mail.gmail.com>
 <CALte62zF2c6Xx9i36O6Eex9jQv_sMMNd+CfVVwv+ze_seJZ6LQ@mail.gmail.com>
 <CADcMMgGq=ZFgU0hZ_QCNQwUSc9W2bYWrXFWQ5H_WxL7q3SiNGg@mail.gmail.com>
 <CAAT7Mkq0pVBm3k4Tc5GU0YF920P-FuWd=dbOUMDfvqhSs_AA2g@mail.gmail.com>
 <CAOtJ30qP8J-K7mPgQuYWj40rSOja7FJ42_K3NGZ7yVccARkscQ@mail.gmail.com>
 <CALte62zBHztksS6YhC_CQM-FfggqOCHPg7hRL1LphS4PfGaX8w@mail.gmail.com>
 <CAOtJ30qSduwOjmMygfrcyFZwCRk2FH6h7Z1pdBAkwg8D=t7ZHw@mail.gmail.com> <CAAT7MkpBJ9LvQX16Mz3_uUnYXcmBe46mW9WFKkm9=L-8RkVvQw@mail.gmail.com>
From: =?UTF-8?B?5byg6ZOO?= <palomino219@gmail.com>
Date: Wed, 7 Dec 2016 18:06:31 +0800
Message-ID: <CAAAYAnOATo2x_T5AQADP0ia+7A4U-bkcmG1EFVjzZuRTex6ZXg@mail.gmail.com>
Subject: Re: meta region written by 2.0 cannot be opened by 1.x
To: dev@hbase.apache.org
Content-Type: multipart/alternative; boundary=001a114e16b4dcb66f05430eac77
archived-at: Wed, 07 Dec 2016 10:06:39 -0000

--001a114e16b4dcb66f05430eac77
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

I believe there are other features that require upgrading to a specific
version in each 1.x release before rolling upgrading to 2.0? I think this
is a typical trick to add incompatible changes...

2016-12-07 18:02 GMT+08:00 ramkrishna vasudevan <
ramkrishna.s.vasudevan@gmail.com>:

> If its going to be a pain then we need to support writing the older class
> name in the FFT (trailer) and then use our internal mapping.
>
> On Wed, Dec 7, 2016 at 3:22 PM, Anoop John <anoop.hbase@gmail.com> wrote:
>
> > I read the comments in the issue 16189..  This concern was raised
> > there already..   This is what I replied there.
> > "Ya we may need to make such a need. User on 1.x has to first upgrade
> > (rolling upgrade supported) to latest version in 1.x and then to 2.0.
> > This was discussed some other place also. I believe in the mail thread
> > where we discussed abt rolling upgrade support in 2.0"
> >
> > Not able to find the original mail thread which discussed abt rolling
> > upgrade support in 2.0    Pls correct if took some thing wrong way..
> > Ya I agree that this way of indirection might be inconvenience for
> > users.
> >
> >
> > -Anoop-
> >
> > On Wed, Dec 7, 2016 at 3:13 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> > > bq. write old name only and within code we will have to map it with n=
ew
> > > name.
> > >
> > > This is a viable approach.
> > > Even if we document upgrading to 1.2.3+, some users may not perform
> this
> > > step before upgrading to 2.0
> > >
> > > On Tue, Dec 6, 2016 at 10:04 PM, Anoop John <anoop.hbase@gmail.com>
> > wrote:
> > >
> > >> Ya that bug raised by Enis was very valid..  So this means when
> > >> rolling upgrade happens, if some of the other RSs with version <1.2.=
3
> > >> , which is not having this fix,  this issue might come up!
> > >> How to address then?   Do we need to enforce a 1.2.3+ upgrade 1st an=
d
> > >> then only 2.0 rolling upgrade?
> > >>
> > >> Or else we will need to fix in 2.0..  We write the new Comparator
> > >> class name in FFT (trunk code)   To fix, we might have to write old
> > >> name only and within code we will have to map it with new name. It
> > >> will be ugly! It can be fixed only in 3.0 may be..  But that can mak=
e
> > >> the rolling upgrade story easier for users..   Just saying the
> > >> possibilities..
> > >>
> > >> Thanks Ted to bring it up again.
> > >>
> > >> -Anoop-
> > >>
> > >> On Wed, Dec 7, 2016 at 10:04 AM, ramkrishna vasudevan
> > >> <ramkrishna.s.vasudevan@gmail.com> wrote:
> > >> > I think when Enis reported the issue (
> > >> > https://issues.apache.org/jira/browse/HBASE-16189), in rolling
> > upgrade
> > >> > regions could move around. So a region created in 2.0 moved to a
> > server
> > >> > with 1.x.
> > >> >
> > >> > Regards
> > >> > Ram
> > >> >
> > >> >
> > >> > On Wed, Dec 7, 2016 at 1:27 AM, Stack <stack@duboce.net> wrote:
> > >> >
> > >> >> On Tue, Dec 6, 2016 at 10:19 AM, Ted Yu <yuzhihong@gmail.com>
> wrote:
> > >> >>
> > >> >> > Looking at http://hbase.apache.org/book.html#executing.the.0.96=
.
> > >> upgrade
> > >> >> ,
> > >> >> > there is step of running "bin/hbase upgrade -check"
> > >> >> >
> > >> >> > How about adding a sample hfile which contains
> > >> >> > CellComparator$MetaCellComparator
> > >> >> > so that the upgrade check can read and verify ?
> > >> >> >
> > >> >> >
> > >> >>
> > >> >> We don't support downgrade. Never have.
> > >> >> St.Ack
> > >> >>
> > >> >>
> > >> >>
> > >> >> > On Tue, Dec 6, 2016 at 8:50 AM, Ted Yu <yuzhihong@gmail.com>
> > wrote:
> > >> >> >
> > >> >> > > The build I used yesterday didn't include HBASE-16189
> > >> >> > > <https://issues.apache.org/jira/browse/HBASE-16189>
> > >> >> > >
> > >> >> > > Once it is included, the cluster can be downgraded fine.
> > >> >> > >
> > >> >> > > I wonder how users would know that their existing deployment
> has
> > >> >> > > HBASE-16189 <https://issues.apache.org/jira/browse/HBASE-1618=
9
> >
> > >> before
> > >> >> > > upgrading to 2.0 release.
> > >> >> > >
> > >> >> > > On Tue, Dec 6, 2016 at 2:29 AM, ramkrishna vasudevan <
> > >> >> > > ramkrishna.s.vasudevan@gmail.com> wrote:
> > >> >> > >
> > >> >> > >> @Ted
> > >> >> > >> Does your version have this fix
> > >> >> > >> https://issues.apache.org/jira/browse/HBASE-16189
> > >> >> > >>
> > >> >> > >> Regards
> > >> >> > >> Ram
> > >> >> > >>
> > >> >> > >> On Tue, Dec 6, 2016 at 3:56 PM, Ted Yu <yuzhihong@gmail.com>
> > >> wrote:
> > >> >> > >>
> > >> >> > >> > Is the assumption that hbase:meta would not split ?
> > >> >> > >> >
> > >> >> > >> > In other thread, Francis Liu was proposing supporting
> > splittable
> > >> >> > >> > hbase:meta in 2.0 release.
> > >> >> > >> >
> > >> >> > >> > Cheers
> > >> >> > >> >
> > >> >> > >> > > On Dec 6, 2016, at 2:20 AM, =E5=BC=A0=E9=93=8E <palomino=
219@gmail.com>
> > wrote:
> > >> >> > >> > >
> > >> >> > >> > > Could this be solved by hosting meta only on master?
> > >> >> > >> > >
> > >> >> > >> > > BTW, MetaCellComparator is introduced in HBASE-10800.
> > >> >> > >> > >
> > >> >> > >> > > Thanks.
> > >> >> > >> > >
> > >> >> > >> > > 2016-12-06 17:44 GMT+08:00 Ted Yu <yuzhihong@gmail.com>:
> > >> >> > >> > >
> > >> >> > >> > >> Hi,
> > >> >> > >> > >> When I restarted a cluster with 1.1 , I found that
> > hbase:meta
> > >> >> > region
> > >> >> > >> > >> (written to by the previously deployed 2.0) couldn't be
> > >> opened:
> > >> >> > >> > >>
> > >> >> > >> > >> Caused by: java.io.IOException:
> > >> >> > >> > >> org.apache.hadoop.hbase.io.hfile.CorruptHFileException:
> > >> Problem
> > >> >> > >> reading
> > >> >> > >> > >> HFile Trailer from file hdfs://yz1.xx.com:8020/apps/
> > >> >> > >> hbase/data/data/
> > >> >> > >> > >> hbase/meta/1588230740/info/
> 599fc8a37311414e876803312009a9
> > 86
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HStore.
> > openStoreFiles(
> > >> >> > >> HStore.java:
> > >> >> > >> > >> 579)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HStore.
> > loadStoreFiles(
> > >> >> > >> HStore.java:
> > >> >> > >> > >> 534)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HStore.<init>(
> > >> >> > HStore.java:275)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegion.
> > instantiateHSto
> > >> >> > >> re(HRegion.
> > >> >> > >> > >> java:5150)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegion$1.call(
> > HRegion.
> > >> >> > >> java:912)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegion$1.call(
> > HRegion.
> > >> >> > >> java:909)
> > >> >> > >> > >>        at java.util.concurrent.
> FutureTask.run(FutureTask.
> > >> >> java:266)
> > >> >> > >> > >>        at
> > >> >> > >> > >> java.util.concurrent.Executors$RunnableAdapter.call(
> > >> >> > >> Executors.java:511)
> > >> >> > >> > >>        at java.util.concurrent.
> FutureTask.run(FutureTask.
> > >> >> java:266)
> > >> >> > >> > >>        ... 3 more
> > >> >> > >> > >> Caused by: org.apache.hadoop.hbase.io.
> > >> >> hfile.CorruptHFileException:
> > >> >> > >> > Problem
> > >> >> > >> > >> reading HFile Trailer from file hdfs://
> > >> >> > >> > >> yz1.xx.com:8020/apps/hbase/data/data/hbase/
> > meta/1588230740/
> > >> >> > >> > >> info/599fc8a37311414e876803312009a986
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.io.
> hfile.HFile.pickReaderVersion(
> > >> >> > >> > HFile.java:483)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.io.hfile.HFile.createReader(
> > >> >> > HFile.java:511)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.
> > >> >> > >> > >> <init>(StoreFile.java:1128)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.StoreFileInfo.
> > >> >> > >> > >> open(StoreFileInfo.java:267)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.StoreFile.open(
> > StoreFil
> > >> >> > >> e.java:409)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.StoreFile.
> > >> >> > >> > >> createReader(StoreFile.java:517)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HStore.
> > createStoreFileA
> > >> >> > >> ndReader(
> > >> >> > >> > >> HStore.java:687)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HStore.access$000(
> > >> >> > >> HStore.java:130)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HStore$1.call(
> > >> >> > HStore.java:554)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HStore$1.call(
> > >> >> > HStore.java:551)
> > >> >> > >> > >>        ... 6 more
> > >> >> > >> > >> Caused by: java.io.IOException: java.lang.
> > >> >> ClassNotFoundException:
> > >> >> > >> > >> org.apache.hadoop.hbase.CellComparator$
> MetaCellComparator
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.
> > getCompara
> > >> >> > >> torClass(
> > >> >> > >> > >> FixedFileTrailer.java:581)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.
> > >> >> > deserializeFromPB(
> > >> >> > >> > >> FixedFileTrailer.java:300)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.
> > >> >> > >> > >> deserialize(FixedFileTrailer.java:242)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.
> > >> >> readFromStream(
> > >> >> > >> > >> FixedFileTrailer.java:407)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.io.
> hfile.HFile.pickReaderVersion(
> > >> >> > >> > HFile.java:468)
> > >> >> > >> > >>        ... 15 more
> > >> >> > >> > >> Caused by: java.lang.ClassNotFoundException:
> > >> >> > >> > >> org.apache.hadoop.hbase.CellComparator$
> MetaCellComparator
> > >> >> > >> > >>        at java.net.URLClassLoader.
> > >> findClass(URLClassLoader.java:
> > >> >> > 381)
> > >> >> > >> > >>        at java.lang.ClassLoader.
> > loadClass(ClassLoader.java:
> > >> 424)
> > >> >> > >> > >>        at sun.misc.Launcher$AppClassLoader.loadClass(
> > >> >> > Launcher.java:
> > >> >> > >> 331)
> > >> >> > >> > >>        at java.lang.ClassLoader.
> > loadClass(ClassLoader.java:
> > >> 357)
> > >> >> > >> > >>        at java.lang.Class.forName0(Native Method)
> > >> >> > >> > >>        at java.lang.Class.forName(Class.java:264)
> > >> >> > >> > >>        at
> > >> >> > >> > >> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.
> > getCompara
> > >> >> > >> torClass(
> > >> >> > >> > >> FixedFileTrailer.java:579)
> > >> >> > >> > >>
> > >> >> > >> > >> When user does rolling upgrade from 1.1 to 2.0, the abo=
ve
> > may
> > >> >> cause
> > >> >> > >> > problem
> > >> >> > >> > >> if hbase:meta region is updated by server running 2.0 b=
ut
> > >> later
> > >> >> > >> > assigned to
> > >> >> > >> > >> a region server which still runs 1.1 (due to crash of t=
he
> > >> server
> > >> >> > >> running
> > >> >> > >> > >> 2.0, e.g.)
> > >> >> > >> > >>
> > >> >> > >> > >> I want to get community feedback on the severity of thi=
s
> > >> issue.
> > >> >> > >> > >>
> > >> >> > >> > >> Thanks
> > >> >> > >> > >>
> > >> >> > >> >
> > >> >> > >>
> > >> >> > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
>

--001a114e16b4dcb66f05430eac77--