flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-7310) always use HybridMemorySegment
Date Fri, 04 Aug 2017 12:50:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16114322#comment-16114322
] 

ASF GitHub Bot commented on FLINK-7310:
---------------------------------------

Github user NicoK commented on the issue:

    https://github.com/apache/flink/pull/4445
  
    in a non-exhaustive mini benchmark, I ran `HashVsSortMiniBenchmark` and got the following
results:
    
    # Best out of 5 (in ms)
    
    Test | `master` | `Flink-7310`
    ---- | ------ | ----------
    Hash Build First | 5541 | 5629
    Sort-Merge | 6194 | 6816
    Hash Build | 3587 | 3629
    
    # All results
    
    ## `master`
    
    Test | 1 | 2 | 3 | 4 | 5
    ---- | - | - | - | - | -
    Hash Build First | 5772.0 | 5541.0 | 5707.0 | 5733.0 | 5751.0
    Sort-Merge | 6704.0 | 7146.0 | 6194.0 | 6915.0 | 6445.0
    Hash Build Second | 3834.0 | 3805.0 | 3811.0 | 3587.0 | 3563.0
    
    ## `FLINK-7310`
    
    Test | 1 | 2 | 3 | 4 | 5
    ---- | - | - | - | - | -
    Hash Build First | 5816.0 | 5770.0 | 5629.0 | 5656.0 | 5745.0
    Sort-Merge | 7284.0 | 7233.0 | 6816.0 | 6861.0 | 7218.0
    Hash Build Second | 3802.0 | 3836.0 | 3629.0 | 3782.0 | 3804.0


> always use HybridMemorySegment
> ------------------------------
>
>                 Key: FLINK-7310
>                 URL: https://issues.apache.org/jira/browse/FLINK-7310
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Core
>    Affects Versions: 1.4.0
>            Reporter: Nico Kruber
>            Assignee: Nico Kruber
>
> For future changes to the network buffers (sending our own off-heap buffers through to
netty), we cannot use {{HeapMemorySegment}} anymore and need to rely on {{HybridMemorySegment}}
instead.
> We should thus drop any code that loads the {{HeapMemorySegment}} (it is still available
if needed) in favour of the {{HybridMemorySegment}} which is able to work on both heap and
off-heap memory.
> FYI: For the performance penalty of this change compared to using {{HeapMemorySegment}}
alone, see this interesting blob article (from 2015):
> https://flink.apache.org/news/2015/09/16/off-heap-memory.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message