phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <la...@apache.org>
Subject Question about memory chunk and waiting for memory to become free
Date Sun, 03 Sep 2017 18:28:12 GMT
Hi,
just looking at MemoryChunk for various reasons.
What I did not realize is that when no memory is currently available the implementation waits
for memory to become available.The default wait time is 10s.

This seems very dangerous to me. This can hang HBase handler threads at various points, especially
when Phoenix is "busy" (I realize we're not talking about real memory here, but the effect
is still that the HBase server threads would stall).
HBase was designed with the assumption that individual RPCs do not tie up a handler thread
for long. (The worst offender is a scan RPC that could scan up to a region's worth if a highly
selective filter was passed). It's also possible that this is mostly used in asynchronous
processing, although a quick scan of the code suggests that's not the case.

It's possible that I'm missing something (in which case someone enlighten me please). I feel
with wait time should be set to 0. I.e. we check whether there's memory available right now,
if not we just fail the request and free the handler thread. I'd even argue we can remove
the wait/synchronization code completely from MemoryChunk.

Comments?
-- Lars

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message