drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Westin <chriswesti...@gmail.com>
Subject Re: [DISCUSSION] should BufferAllocator.buffer() throw an OutOfMemoryException ?
Date Tue, 30 Jun 2015 19:13:30 GMT
No, but we should do something close to that.

There are cases where the caller can handle the inability to get more
memory, and may be able to go to disk. However, you are correct that there
are many that can't handle an OOM, and that fail to check.

Instead of unconditionally throwing OutOfMemoryRuntimeException, I would
suggest that the buffer() call take a flag that indicates whether or not it
should throw if it is unable to fulfill the request. This way, the call
sites that can handle an OOM can pass in the flag to return null, and the
rest can pass in the flag value to throw, and not have to have any checking
code.


On Tue, Jun 30, 2015 at 12:06 PM, Abdel Hakim Deneche <adeneche@maprtech.com
> wrote:

> our current implementations of BufferAllocator.buffer(int, int) returns
> null when it cannot allocate the buffer.
>
> But looking through the code, there are many places that don't check if the
> allocated buffer is null before trying to access it which will throw a
> NullPointerException.
>
> ValueVectors' allocateNewSafe() seem to be the only place that handle the
> null in a specific manner.
>
> Should we update the allocators' implementation to throw an
> OutOfMemoryRuntimeException instead of returning null ? this has the added
> benefit of displaying a proper out of memory error message to the user.
>
> Thanks!
>
> --
>
> Abdelhakim Deneche
>
> Software Engineer
>
>   <http://www.mapr.com/>
>
>
> Now Available - Free Hadoop On-Demand Training
> <
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message