phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-2405) Improve performance and stability of server side sort for ORDER BY
Date Fri, 18 Mar 2016 18:44:33 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201951#comment-15201951
] 

stack commented on PHOENIX-2405:
--------------------------------

bq. Thanks stack, how about fallback to Mnemonic only if OutOfMemoryError occurred ?

Triggering on an OOME seems a bit dodgy. Could trap on 'OutOfMemoryError: Map failed' but
would have to read the code to ensure no other damaging 'side-effects'. OOME is post-Full-GC
too, no? Would be good if could avoid a Full GC as trigger.

Better if we were just all mnemonic, all the time (caveat good perf, etc). Just trying to
learn what build and runtime need to look like if mnemonic route. Thanks.



> Improve performance and stability of server side sort for ORDER BY
> ------------------------------------------------------------------
>
>                 Key: PHOENIX-2405
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-2405
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: James Taylor
>            Assignee: Maryann Xue
>              Labels: gsoc2016
>             Fix For: 4.8.0
>
>
> We currently use memory mapped files to buffer data as it's being sorted in an ORDER
BY (see MappedByteBufferQueue). The following types of exceptions have been seen to occur:
> {code}
> Caused by: java.lang.OutOfMemoryError: Map failed
>         at sun.nio.ch.FileChannelImpl.map0(Native Method)
>         at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:904)
> {code}
> [~apurtell] has read that memory mapped files are not cleaned up after very well in Java:
> {quote}
> "Map failed" means the JVM ran out of virtual address space. If you search around stack
overflow for suggestions on what to do when your app (in this case Phoenix) encounters this
issue when using mapped buffers, the answers tend toward manually cleaning up the mapped buffers
or explicitly triggering a full GC. See http://stackoverflow.com/questions/8553158/prevent-outofmemory-when-using-java-nio-mappedbytebuffer
for example. There are apparently long standing JVM/JRE problems with reclamation of mapped
buffers. I think we may want to explore in Phoenix a different way to achieve what the current
code is doing.
> {quote}
> Instead of using memory mapped files, we could use heap memory, or perhaps there are
other mechanisms too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message