accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kepner, Jeremy - 1010 - MITLL" <>
Subject ingest performance oscillations and Xceivers
Date Wed, 02 Jan 2013 02:19:22 GMT
Accumulo Colleagues,
  I am trying to optimize my ingest into a single node Accumulo instance running on a 32 core
node with 96 GB of RAM.  I am seeing the follow ingest variations as a I change the number
of ingest processes (see attached):

Ingestors, Ingest rate
1, 60K inserts/sec (stable)
2, 120K inserts/sec (stable)
3, 60K to 180K inserts/sec
4, 90K to 220K inserts/sec
8, 80K to 280K inserts/sec
12, 80K to 280K inserts/sec

The only thing I can see that correlates with the ingest rate is the number of Xceivers. 
When the ingest rate is high the number of Xceivers is usually low.  Likewise, when the ingest
rate drops, the number of Xceivers usually increases significantly.

Question: What role to Xceivers play in ingest?

Request: It would be great to add a plot showing the number of Xceivers over time to the diagnostics.

Regards.  -Jeremy

View raw message