phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-5120) Avoid using MappedByteBuffers for server side sorting.
Date Sat, 02 Feb 2019 21:55:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-5120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16759217#comment-16759217
] 

Lars Hofhansl commented on PHOENIX-5120:
----------------------------------------

We just ran into this in production. I'm going to file a fresh issue.

My main observation here is:
# We're only spilling when the sort buffer is over some threshold, at which point we'd expect
this to get a bit slower.
# We do not need random access. The sorted part is written, than read back purely sequential.
Simple file IO would do the trick as well.



> Avoid using MappedByteBuffers for server side sorting.
> ------------------------------------------------------
>
>                 Key: PHOENIX-5120
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5120
>             Project: Phoenix
>          Issue Type: Task
>            Reporter: Lars Hofhansl
>            Priority: Major
>
> We should had a production outage due to this.
> MappedByteBuffer may leave files around, on top they use direct memory, which is not
cleared until the JVM executes a full GC.
> See last comment on PHOENIX-2405.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message