Mailing-List: contact issues-help@tez.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@tez.apache.org
Date: Wed, 22 Oct 2014 07:48:33 +0000 (UTC)
From: "Gopal V (JIRA)" <jira@apache.org>
To: issues@tez.apache.org
Message-ID: <JIRA.12742520.1411060538000.312759.1413964113621@Atlassian.JIRA>
In-Reply-To: <JIRA.12742520.1411060538000@Atlassian.JIRA>
References: <JIRA.12742520.1411060538000@Atlassian.JIRA>
 <JIRA.12742520.1411060538899@arcas>
Subject: [jira] [Updated] (TEZ-1593) Refactor PipelinedSorter to remove all
 MMAP based ByteBuffer references
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/TEZ-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gopal V updated TEZ-1593:
-------------------------
    Attachment: TEZ-1593.1.patch

> Refactor PipelinedSorter to remove all MMAP based ByteBuffer references
> -----------------------------------------------------------------------
>
>                 Key: TEZ-1593
>                 URL: https://issues.apache.org/jira/browse/TEZ-1593
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.6.0
>            Reporter: Gopal V
>            Assignee: Gopal V
>              Labels: Performance
>         Attachments: TEZ-1593.1.patch
>
>
> The current implementation of PipelinedSorter has a slow section which revolves around key comparisons - this was relevant when the implementation used direct byte buffers to back the kvbuffer.
> {code}
>       kvbuffer.position(istart);
>       kvbuffer.get(ki, 0, ilen);
>       kvbuffer.position(jstart);
>       kvbuffer.get(kj, 0, jlen);
>       // sort by key
>       final int cmp = comparator.compare(ki, 0, ilen, kj, 0, jlen);
> {code}
> The kvbuffer.get into the arrays ki and kj are the slowest part of the comparator operation.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)