Return-Path: X-Original-To: apmail-crunch-dev-archive@www.apache.org Delivered-To: apmail-crunch-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id F05EA10BFE for ; Fri, 24 Jan 2014 15:58:47 +0000 (UTC) Received: (qmail 81494 invoked by uid 500); 24 Jan 2014 15:58:39 -0000 Delivered-To: apmail-crunch-dev-archive@crunch.apache.org Received: (qmail 81453 invoked by uid 500); 24 Jan 2014 15:58:39 -0000 Mailing-List: contact dev-help@crunch.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@crunch.apache.org Delivered-To: mailing list dev@crunch.apache.org Received: (qmail 81441 invoked by uid 500); 24 Jan 2014 15:58:38 -0000 Delivered-To: apmail-incubator-crunch-dev@incubator.apache.org Received: (qmail 81435 invoked by uid 99); 24 Jan 2014 15:58:38 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 24 Jan 2014 15:58:38 +0000 Date: Fri, 24 Jan 2014 15:58:38 +0000 (UTC) From: "Josh Wills (JIRA)" To: crunch-dev@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CRUNCH-329) Re-add type info to TupleWritable to make fields sort correctly MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CRUNCH-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881076#comment-13881076 ] Josh Wills commented on CRUNCH-329: ----------------------------------- Or do you mean, on a WritableComparable that is used in a Tuple? > Re-add type info to TupleWritable to make fields sort correctly > --------------------------------------------------------------- > > Key: CRUNCH-329 > URL: https://issues.apache.org/jira/browse/CRUNCH-329 > Project: Crunch > Issue Type: Bug > Components: Core > Affects Versions: 0.10.0, 0.8.3 > Reporter: Josh Wills > Assignee: Josh Wills > Fix For: 0.10.0, 0.8.3 > > Attachments: fix-ss-writables.patch > > > Secondary sorts aren't currently working correctly for Writable types after we hacked the TupleWritable impl to make all of the fields BytesWritables (e.g., secondary IntWritable values will no longer be sorted correctly, even though everything is still grouped correctly.) > The least-bad way that I came up with to fix this is to use integer codes for each possible WritableComparable type in a pipeline that we can use to decode what Writable type each tuple field corresponds to. This allows us to keep the various fields sortable while still doing a reasonable job of minimizing the serialization required to pass the type information along. -- This message was sent by Atlassian JIRA (v6.1.5#6160)