accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From z11373 <z11...@outlook.com>
Subject Re: offload request to another tserver
Date Mon, 11 Jan 2016 18:53:46 GMT
Thanks Josh!
Before I change the config number, I am trying to figure out what triggered
it.

There are 3 tablet servers running, let say they are:
TS01 (this got the least scan requests)
TS02
TS03 (this got the most scan requests)

When the test clients are sending the scan requests, from Accumulo Monitor
page I see:
TS01: 1 tablet (of the table) with size 21.54M entries.
TS02: 3 tablets with size 13.96M, 6.95M, and 30.55M entries.
TS03: 2 tablets with 12.35 and 12.33M entries.

I ran the tests multiple times with different number of clients, and the
behavior is pretty consistent.
The dataset has about 100M entries, and the sum up of those number of
entries above is close to 100M. Therefore, the fact that TS03 got the
highest number of requests, makes me think likely hotspot on those 2 tablets
on third server, which could be just pure coincident. Unfortunately, the
nature of the data I ingesting is hard to pre-split them.

Some questions from observation above:
1. I'd assume Accumulo will split the table into tablets with evenly size,
or at least they'd be pretty close, but from the data above it doesn't seem
so. Is that because of compression?

2. The client is sending a lot of small scan requests (i.e. thousands of
scan requests in short period of time), which is not surprising to see one
of the tablet server got swamped if the data being queried reside there. I
am thinking to use BatchScanner, do you think that will help in this case?
Is there limit on number of Ranges we can add to the BatchScanner?

3. Is there an easier way to see or count all scan requests coming from the
client, other than looking it from tserver log files? The output on Monitor
page only gave numbers periodically (i.e. auto-refresh enabled), and looking
from the log is pretty overwhelming, not to mention other clients also
talking to Accumulo


Thanks,
Z




--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/offload-request-to-another-tserver-tp15879p15883.html
Sent from the Developers mailing list archive at Nabble.com.

Mime
View raw message