phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-5120) Avoid using MappedByteBuffers for server side sorting.
Date Sat, 02 Feb 2019 22:58:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-5120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16759230#comment-16759230
] 

Lars Hofhansl edited comment on PHOENIX-5120 at 2/2/19 10:57 PM:
-----------------------------------------------------------------

GROUP BY spilling also uses MappedByteBuffers. So there are three ways (at least) of spilling
in Phoenix: (1) Client side spooling iterators, (2) OrderedResultIterator, and (3) the GroupByCache.

The GROUP BY spilling is very different and writes/reads pages, so access patterns there are
not clear to me. I won't touch that part here.


was (Author: lhofhansl):
GROUP BY spilling also uses MappedByteBuffers. So there are three ways (at least) of spilling
in Phoenix: (1) Client side spooling iterators, (2) OrderedResultIterator, and (3) the GroupByCache.


> Avoid using MappedByteBuffers for server side sorting.
> ------------------------------------------------------
>
>                 Key: PHOENIX-5120
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5120
>             Project: Phoenix
>          Issue Type: Task
>            Reporter: Lars Hofhansl
>            Assignee: Lars Hofhansl
>            Priority: Major
>         Attachments: 5120-1.4-wip.txt, 5120-1.4.txt
>
>
> We should had a production outage due to this.
> MappedByteBuffer may leave files around, on top they use direct memory, which is not
cleared until the JVM executes a full GC.
> See last comment on PHOENIX-2405.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message