crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chao Shi (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CRUNCH-368) TupleWritable.Comparator
Date Sat, 22 Mar 2014 02:59:42 GMT
Chao Shi created CRUNCH-368:
-------------------------------

             Summary: TupleWritable.Comparator
                 Key: CRUNCH-368
                 URL: https://issues.apache.org/jira/browse/CRUNCH-368
             Project: Crunch
          Issue Type: Improvement
          Components: Core
    Affects Versions: 0.8.3
            Reporter: Chao Shi
            Assignee: Chao Shi


This patch should improve comparison performance on TupleWritables. It saves the deserialization
overhead. It is particularly useful when the input tuple are large, e.g. contains long strings.

Please note that this changes the binary format of TupleWritable. It adds a var-int indicating
size of field after each type code. This is a limitation of the writable system. We do not
know the size of each field until fully desalinizing it. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message