hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "George P. Stathis" <gstat...@traackr.com>
Subject Re: Latency related configs for 0.90
Date Thu, 28 Apr 2011 19:54:05 GMT
On Thu, Apr 28, 2011 at 3:31 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> I'll try to repro here.
>
> Also I was thinking that in order to slow you down that much, you
> would need to have a query pattern that requires reading and writing a
> LOT to .META., so would it be possible for you to verify this indeed
> your case?
>

I doubt that's the case. The unit tests we are using were doing simple CRUD
operations on regular data tables. Granted they attempted to do them
quickly, but still: not that much data added or removed at one time. Just
quickly added and removed repeatedly for each unit test method using the
setup/teardown methods in our unit tests.


>
> And finally, just to make sure, you're saying that you didn't see that
> sort of performance issue with 0.89 right? Which one exactly? The only
> difference I can think of between 0924 and 0.90 is that the latter
> does .META. warming by fetching more rows after every lookup. I can't
> think of any other cause at the moment.
>

Yes, the same exact unit tests were doing just fine under 0.89. BUT: we were
not running trunk 0.89. We were running this one:
https://github.com/jameskennedy/hbase/tree/HLogSplit_0.89.20100726 because
we were using the hbase-trx contrib. This whole migration was intended to
put us back to a supported 0.90 trunk.


>
> Thank you so much for keeping us updated,
>
> J-D
>
> On Wed, Apr 27, 2011 at 5:51 PM, George P. Stathis <gstathis@traackr.com>
> wrote:
> > On Wed, Apr 27, 2011 at 8:13 PM, Jean-Daniel Cryans <jdcryans@apache.org
> >wrote:
> >
> >> I have a hard time digesting this... You ran the script, didn't change
> >> anything else, ran the test and everything was back to normal, right?
> >> Did you restart HBase or moved .META. around? The reason I'm asking is
> >> that this script doesn't have any effect until .META. is reopened so I
> >> would be quite flabbergasted to learn that _just_ running the script
> >> makes things faster.
> >>
> >
> > No, I did indeed restart hbase as the instructions suggest we do after
> the
> > script is executed.
> >
> >
> >>
> >> Another thing that's weird is that that setting for .META. has been
> >> there since 0.20.0 so if it's the cause then it should have been the
> >> same with 0.89
> >>
> >
> > The setting were there in the legacy dev hbase instance we have. It was
> set
> > to 16K and the script set it to 64K. So it seems this was expected. Where
> I
> > don't understand is the effect the script had on a brand new 0.90
> install.
> > Before we ran the script, the setting was not there. After we ran it, it
> was
> > there with the 64K value.
> >
> > For both legacy and new instances, running the script and re-starting
> hbase
> > solved our issue.
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message