hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Geoff Hendrey" <ghend...@decarta.com>
Subject RE: version mismatch exception
Date Wed, 17 Aug 2011 16:54:57 GMT
Hi St.Ack,

Keying off of what you said: " Did you update the
info:regioninfo cell so it has a new hregioninfo with same start and
end row?   You know this makes a new region, rather than extend the
range of the previous region? (So the old region will be in the
filesystem still with the old data).  You should also make sure the
original region is closed before you delete the row from .META."

So I take it the best approach is to:
1) close the original region (the region whose .META.'s endkey we want to repoint)
2) delete the region's row from .META.
3) Put a new row into .META., the new row's hregioninfo having the desired endkey such that
there is no more "hole" in .META.

I'm trying to nail down exactly the sequence of steps we should take so that we don't have
to do scary manual surgery of -ROOT- and .META. like we did yesterday. We we're pretty much
fumbling around in the dark trying to figure out the structure of -ROOT-'s HDFS files, and
same for .META. after our first failed attempt to update the endrow. We did figure it out,
and removed the files from .META. and -ROOT- that prevented hbase from coming up. Our error
was that instead of updating the endrow in .META. we inadvertently put a new row into .META.
with default timestamp, and that basically shot everything to hell. I couldn't find docs on
the structure of -ROOT- and .META. HDFS files, but we sort of pieced it together and we're
able to remove the newly created files in -ROOT- and .META. based on the their creation times
and grepping their content, after which hbase was able to come back up without error.

So, apologies for going slow on this, and really trying to exactly nail down the set of steps
we should proceed with in order to avoid another self-inflicted corruption.

Best,
geoff


-----Original Message-----
From: Rohit Nigam 
Sent: Monday, August 15, 2011 8:46 PM
To: Stack; user@hbase.apache.org
Cc: Search
Subject: RE: version mismatch exception

So I got the info:regioninfo value from the .META. table for the key and I did a put in the
.META. table with the  change in the endkey. Yes I did change the info:regioninfo cell with
just changing the  endkey. I didn't see the change as the master didn't come up and kept throwing
the version mismatch exception. Somehow the version got changed , no idea how? , I had to
run the add_table.rb for that table to restore the whole thing for the master to be up. So
my case is to just  update the endkey of a row in the .META. table for a table region because
the chain is broken ,how do I do that so that this exception doesn't happen.
Rohit

-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
Sent: Monday, August 15, 2011 8:29 PM
To: user@hbase.apache.org
Cc: Search
Subject: Re: version mismatch exception

On Mon, Aug 15, 2011 at 2:45 PM, Rohit Nigam <rnigam@decarta.com> wrote:
> I changed the endkey of one of the records in the '.META.' table because
> of the chaining issue we experienced ,  using the program which gets
> that row and does a put so that the  endkey could be changed , now when
> I try to view the record in .META. table using shell I get a
>

Tell me how you did this?  You removed the original row and replaced
it with another that has different end key?  Did you update the
info:regioninfo cell so it has a new hregioninfo with same start and
end row?   You know this makes a new region, rather than extend the
range of the previous region? (So the old region will be in the
filesystem still with the old data).  You should also make sure the
original region is closed before you delete the row from .META.


>
>
> ERROR: org.apache.hadoop.io.VersionMismatchException: null
>

Where did you get this from?


> 2011-08-15 14:32:34,639 FATAL [master-doop10.dt.sv4.decarta.com:60000]
> master.HMaster(948): Unhandled exception. Starting shutdown.
>
> A record version mismatch occured. Expecting v0, found v116
>

What you doing when this happens?  It looks like we are deserializing
the wrong content?  Is that possible.

St.Ack

Mime
View raw message