accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Marion <dlmar...@hotmail.com>
Subject Re: Review Request 43957: ACCUMULO-1755: removed synchronized modifier from TabletServerBatchWriterstartProcessing()
Date Thu, 25 Feb 2016 19:04:02 GMT


> On Feb. 25, 2016, 3:07 p.m., Adam Fuchs wrote:
> > core/src/main/java/org/apache/accumulo/core/client/impl/TabletServerBatchWriter.java,
line 243
> > <https://reviews.apache.org/r/43957/diff/3/?file=1268425#file1268425line243>
> >
> >     Removing this synchronized will make it tough to keep the MutationSet limited
to a given size. There's now the possibility that every thread can now add a mutation before
the aggregate size of mutations gets checked. Also, startProcessing() can get called once
per thread every time the buffer fills instead of just once. Synchronizing this method is
probably not a huge performance problem if we limit the operations to adding references to
a list and a quick size check. Maybe we can have startProcessing() return a Runnable that
then gets executed outside of a synchronized block in addMutation?

I thought about this a little more today and what I am doing here is trying to remove the
throttle from the client side of the TSBW, but this might queue up MutationSet objects to
be written to the backend. I wonder if having the MutationSet be a thread local might fix
this. Each client thread will have its own buffer and will throttle itself, but hopefully
not other threads. I should be able to determine the number of concurrent threads and divide
the max memory by that to flush each threads buffer independently.


- Dave


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43957/#review120584
-----------------------------------------------------------


On Feb. 24, 2016, 11:10 p.m., Dave Marion wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43957/
> -----------------------------------------------------------
> 
> (Updated Feb. 24, 2016, 11:10 p.m.)
> 
> 
> Review request for accumulo and Josh Elser.
> 
> 
> Bugs: ACCUMULO-1755
>     https://issues.apache.org/jira/browse/ACCUMULO-1755
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> ACCUMULO-1755: removed synchronized modifier from TabletServerBatchWriterstartProcessing()
> 
> 
> Diffs
> -----
> 
>   core/src/main/java/org/apache/accumulo/core/client/impl/TabletServerBatchWriter.java
bc90d00 
> 
> Diff: https://reviews.apache.org/r/43957/diff/
> 
> 
> Testing
> -------
> 
> unit tests in core pass
> 
> 
> Thanks,
> 
> Dave Marion
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message