hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Rawson" <ryano...@gmail.com>
Subject Re: Review Request: HBASE-2514 RegionServer should refuse to be assigned a region that use LZO when LZO isn't available
Date Wed, 20 Oct 2010 23:28:07 GMT


> On 2010-10-20 12:41:06, stack wrote:
> > trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line
318
> > <http://review.cloudera.org/r/1046/diff/1/?file=14924#file14924line318>
> >
> >     So this String is not defined so we don't do any compression test with this
patch, right?
> >     
> >     Is this patch what we want?  Issue is about not loading a region that needs
lzo.  Thats one of two things: a new table that has lzo specified though we don't have lzo
loaded; or its an HRS being asked to open a region that was written by another HRS that had
lzo installed  -- and this does not.  In the issue we thought the region should just not be
opened.  If this HRS is errant being the only one w/ no LZO installed, then he'll open non-lzo'd
regions.  As is, HRSs will go down... a whole cluster could go down just because a silly admin
enabled lzo on a table that had a region per HRS in a cluster.

i think this is the patch we want.  If you look at JD's patch, he points out that it only
checks the schema, not the store files.  If any of the store files are compressed with LZO
but the schema is something else, then his patch will give it a 'pass' but it will fail to
read.  Checking all the store files is pretty time consuming and expensive.

Instead, we can give operators tools to keep themselves out of trouble.  If they intend to
run LZO, but a single machine is misconfigured, refuse to run instead of logging some difficult
to find error.

I dont think it's a common case to run uneven configured machines and expect the LZO-ed regions
to 'flow' to the 'correct' regionservers.  If we really want to do that, we should add capabilities
to regionservers and requirements to regions and stop the assignment in the master. Seems
excessive though.


- Ryan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/1046/#review1574
-----------------------------------------------------------


On 2010-10-18 19:30:09, Ryan Rawson wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://review.cloudera.org/r/1046/
> -----------------------------------------------------------
> 
> (Updated 2010-10-18 19:30:09)
> 
> 
> Review request for hbase.
> 
> 
> Summary
> -------
> 
> RegionServer should refuse to be assigned a region that use LZO when LZO isn't available
> 
> - instead of checking regions during open, i do a check and throw during the construction
of the RS that is is able to instantiate the listed codecs.
> 
> 
> This addresses bug HBASE-2514.
>     http://issues.apache.org/jira/browse/HBASE-2514
> 
> 
> Diffs
> -----
> 
>   trunk/CHANGES.txt 1024074 
>   trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1024074

>   trunk/src/main/java/org/apache/hadoop/hbase/util/CompressionTest.java 1024073 
>   trunk/src/test/java/org/apache/hadoop/hbase/util/TestCompressionTest.java PRE-CREATION

> 
> Diff: http://review.cloudera.org/r/1046/diff
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Ryan
> 
>


Mime
View raw message