hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From schausson <schaus...@softera.fr>
Subject Re: Writting bottleneck in HBase ?
Date Thu, 24 Nov 2016 16:13:07 GMT
Hi, thanks for your answer.

About your question related to thread management : yes, I have several
threads (up to 4) that may call my persistence method.

When I wrote the post, I had not configured anything special about regions
for my table so it basically used default splitting policy I guess. 
Next to your answer, I gave a try to this :
/byte[][] splits = new
RegionSplitter.HexStringSplit().split(numberOfRegionServers);
/
Which lead to 12 regions at table creation time.

It slightly improved performances : persistance drops from 2min to 1min40s
approximately.

I tried with 24 regions but nothing changed then...

About how parameters IDs are distributed : to make it simple, I read 5
values per parameter (*2000) and call persistence, and repeat this sequence
a thousand times. So they should distribute accross all my region servers,
right ?
One additional clue : parameters ID are alphanumeric, evenly distributed
between AAAAA and ZZZZZ, but I add a prefix to them which is long string
(about 25 characters). To save storage space (because rowId is dupplicated
for each cell), I use hashing mechanism to transform this long string into
Long value (and I ahev a mapping table next to the main table), so I dont
really know how these Long values "distribute"...

Not sure I'm clear...









--
View this message in context: http://apache-hbase.679495.n3.nabble.com/Writting-bottleneck-in-HBase-tp4084656p4084678.html
Sent from the HBase User mailing list archive at Nabble.com.

Mime
View raw message