hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject Re: Rowkey design question
Date Fri, 17 Apr 2015 18:03:08 GMT
Sorry, but … 

We are in violent agreement. 
If done wrong it can and will kill you. 
Murphy’s law. If there’s more than one way to do something … the wrong way will be chosen,
so where does that leave you? 

And then what hasn’t been said is the security concern which is odd because with XASecure
now Ranger (until they try a new and different name), you need to use coprocessors on a trigger
to see if you have permission to write to, or read data in a table. 

If you’re new to HBase… don’t use a coprocessor. That’s just asking for trouble. 
(And I know everyone here knows that to be the truth.) 

> On Apr 12, 2015, at 1:45 AM, lars hofhansl <larsh@apache.org> wrote:
> 
> After the fun interlude (sorry about that) let me get back to the issue.
> 
> There a multiple consideration:
> 
> 1. row vs column. If in doubt err on the side of more rows. Only use many columns in
a row when you need transaction over the data in the columns.
> 2. Value sizes. HBase is good at dealing with many small things. 1-5mb values here and
there are OK, but most rows should be < a few dozen KBs. Otherwise you'll see too much
write amplification.
> 3. Column families. Place columns you typically access together in the same column family,
and try to keep columns you don't access together mostly in different families.
> HBase can than efficiently rule out a large body of data to scan, by avoiding scanning
families that are not needed.
> 4. Coprocessors and filters let you transform/filter things where the data is. The benefit
can be huge.  With coprocessors you can "trap" scan requests (next() calls) and inject your
own logic.
> Thats what Phoenix does for example, and it's pretty efficient if done right (if you
do it wrong you can kill your region server).
> 
> On #2. You might want to invent a scheme where you store smaller values by value (i.e.
in HBase) and larger ones by reference.
> 
> I would put the column with the large value in its own family so that you could scan
the rest of the metadata without requiring HBase to read the large value.
> You can follow a simple protocol:
> A. If the value is small (pick some notion of small between 1 and 10mb), store it in
HBase, in a separate familY.
> B. Otherwise:
> 1. Write a row with the intended location of the file holding the value in HDFS.
> 2. Write the value into the HDFS file. Make sure the file location has a random element
to avoid races.
> 3. Update the row created in #1 with a commit column (just a column you set to true),
this is like a commit.
> (only when a writer reaches this point should the value be considered written)
> 
> Note the everything is idempotent. The worst that can happen is that the process fails
between #2 and #3. Now you have orphaned data in HDFS. Since the HDFS location has a random
element in it, you can just retry.
> You can either leave orphaned data (since the commit bit is not set, it's not visible
to a client), or you periodically look for those and clean them up.
> 
> Hope this helps. Please let us know how it goes.
> 
> -- Lars
> 
> 
> ________________________________
> From: Kristoffer Sjögren <stoffe@gmail.com>
> To: user@hbase.apache.org 
> Sent: Wednesday, April 8, 2015 6:41 AM
> Subject: Re: Rowkey design question
> 
> 
> Yes, I think you're right. Adding one or more dimensions to the rowkey
> would indeed make the table narrower.
> 
> And I guess it also make sense to store actual values (bigger qualifiers)
> outside HBase. Keeping them in Hadoop why not? Pulling hot ones out on SSD
> caches would be an interesting solution. And quite a bit simpler.
> 
> Good call and thanks for the tip! :-)
> 
> 
> 
> 
> On Wed, Apr 8, 2015 at 1:45 PM, Michael Segel <michael_segel@hotmail.com>
> wrote:
> 
>> Ok…
>> 
>> First, I’d suggest you rethink your schema by adding an additional
>> dimension.
>> You’ll end up with more rows, but a narrower table.
>> 
>> In terms of compaction… if the data is relatively static, you won’t have
>> compactions because nothing changed.
>> But if your data is that static… why not put the data in sequence files
>> and use HBase as the index. Could be faster.
>> 
>> HTH
>> 
>> -Mike
>> 
>>> On Apr 8, 2015, at 3:26 AM, Kristoffer Sjögren <stoffe@gmail.com> wrote:
>>> 
>>> I just read through HBase MOB design document and one thing that caught
>> my
>>> attention was the following statement.
>>> 
>>> "When HBase deals with large numbers of values > 100kb and up to ~10MB of
>>> data, it encounters performance degradations due to write amplification
>>> caused by splits and compactions."
>>> 
>>> Is there any chance to run into this problem in the read path for data
>> that
>>> is written infrequently and never changed?
>>> 
>>> On Wed, Apr 8, 2015 at 9:30 AM, Kristoffer Sjögren <stoffe@gmail.com>
>> wrote:
>>> 
>>>> A small set of qualifiers will be accessed frequently so keeping them in
>>>> block cache would be very beneficial. Some very seldom. So this sounds
>> very
>>>> promising!
>>>> 
>>>> The reason why i'm considering a coprocessor is that I need to provide
>>>> very specific information in the query request. Same thing with the
>>>> response. Queries are also highly parallelizable across rows and each
>>>> individual query produce a valid result that may or may not be
>> aggregated
>>>> with other results in the client, maybe even inside the region if it
>>>> contained multiple rows targeted by the query.
>>>> 
>>>> So it's a bit like Phoenix but with a different storage format and query
>>>> engine.
>>>> 
>>>> On Wed, Apr 8, 2015 at 12:46 AM, Nick Dimiduk <ndimiduk@gmail.com>
>> wrote:
>>>> 
>>>>> Those rows are written out into HBase blocks on cell boundaries. Your
>>>>> column family has a BLOCK_SIZE attribute, which you may or may have no
>>>>> overridden the default of 64k. Cells are written into a block until is
>> it
>>>>>> = the target block size. So your single 500mb row will be broken
down
>>>>> into
>>>>> thousands of HFile blocks in some number of HFiles. Some of those
>> blocks
>>>>> may contain just a cell or two and be a couple MB in size, to hold the
>>>>> largest of your cells. Those blocks will be loaded into the Block
>> Cache as
>>>>> they're accessed. If your careful with your access patterns and only
>>>>> request cells that you need to evaluate, you'll only ever load the
>> blocks
>>>>> containing those cells into the cache.
>>>>> 
>>>>>> Will the entire row be loaded or only the qualifiers I ask for?
>>>>> 
>>>>> So then, the answer to your question is: it depends on how you're
>>>>> interacting with the row from your coprocessor. The read path will only
>>>>> load blocks that your scanner requests. If your coprocessor is
>> producing
>>>>> scanner with to seek to specific qualifiers, you'll only load those
>>>>> blocks.
>>>>> 
>>>>> Related question: Is there a reason you're using a coprocessor instead
>> of
>>>>> a
>>>>> regular filter, or a simple qualified get/scan to access data from
>> these
>>>>> rows? The "default stuff" is already tuned to load data sparsely, as
>> would
>>>>> be desirable for your schema.
>>>>> 
>>>>> -n
>>>>> 
>>>>> On Tue, Apr 7, 2015 at 2:22 PM, Kristoffer Sjögren <stoffe@gmail.com>
>>>>> wrote:
>>>>> 
>>>>>> Sorry I should have explained my use case a bit more.
>>>>>> 
>>>>>> Yes, it's a pretty big row and it's "close" to worst case. Normally
>>>>> there
>>>>>> would be fewer qualifiers and the largest qualifiers would be smaller.
>>>>>> 
>>>>>> The reason why these rows gets big is because they stores aggregated
>>>>> data
>>>>>> in indexed compressed form. This format allow for extremely fast
>> queries
>>>>>> (on local disk format) over billions of rows (not rows in HBase
>> speak),
>>>>>> when touching smaller areas of the data. If would store the data
as
>>>>> regular
>>>>>> HBase rows things would get very slow unless I had many many region
>>>>>> servers.
>>>>>> 
>>>>>> The coprocessor is used for doing custom queries on the indexed data
>>>>> inside
>>>>>> the region servers. These queries are not like a regular row scan,
but
>>>>> very
>>>>>> specific as to how the data is formatted withing each column
>> qualifier.
>>>>>> 
>>>>>> Yes, this is not possible if HBase loads the whole 500MB each time
i
>>>>> want
>>>>>> to perform this custom query on a row. Hence my question :-)
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Tue, Apr 7, 2015 at 11:03 PM, Michael Segel <
>>>>> michael_segel@hotmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> Sorry, but your initial problem statement doesn’t seem to parse
…
>>>>>>> 
>>>>>>> Are you saying that you a single row with approximately 100,000
>>>>> elements
>>>>>>> where each element is roughly 1-5KB in size and in addition there
are
>>>>> ~5
>>>>>>> elements which will be between one and five MB in size?
>>>>>>> 
>>>>>>> And you then mention a coprocessor?
>>>>>>> 
>>>>>>> Just looking at the numbers… 100K * 5KB means that each row
would end
>>>>> up
>>>>>>> being 500MB in size.
>>>>>>> 
>>>>>>> That’s a pretty fat row.
>>>>>>> 
>>>>>>> I would suggest rethinking your strategy.
>>>>>>> 
>>>>>>>> On Apr 7, 2015, at 11:13 AM, Kristoffer Sjögren <stoffe@gmail.com>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Hi
>>>>>>>> 
>>>>>>>> I have a row with around 100.000 qualifiers with mostly small
values
>>>>>>> around
>>>>>>>> 1-5KB and maybe 5 largers ones around 1-5 MB. A coprocessor
do
>>>>> random
>>>>>>>> access of 1-10 qualifiers per row.
>>>>>>>> 
>>>>>>>> I would like to understand how HBase loads the data into
memory.
>>>>> Will
>>>>>> the
>>>>>>>> entire row be loaded or only the qualifiers I ask for (like
pointer
>>>>>>> access
>>>>>>>> into a direct ByteBuffer) ?
>>>>>>>> 
>>>>>>>> Cheers,
>>>>>>>> -Kristoffer
>>>>>>> 
>>>>>>> The opinions expressed here are mine, while they may reflect
a
>>>>> cognitive
>>>>>>> thought, that is purely accidental.
>>>>>>> Use at your own risk.
>>>>>>> Michael Segel
>>>>>>> michael_segel (AT) hotmail.com
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>>> 
>> 
>> The opinions expressed here are mine, while they may reflect a cognitive
>> thought, that is purely accidental.
>> Use at your own risk.
>> Michael Segel
>> michael_segel (AT) hotmail.com
>> 
>> 
>> 
>> 
>> 
>> 
> 

The opinions expressed here are mine, while they may reflect a cognitive thought, that is
purely accidental. 
Use at your own risk. 
Michael Segel
michael_segel (AT) hotmail.com






Mime
View raw message