Return-Path: X-Original-To: apmail-crunch-dev-archive@www.apache.org Delivered-To: apmail-crunch-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7C4EF17DBB for ; Mon, 2 Mar 2015 18:06:18 +0000 (UTC) Received: (qmail 181 invoked by uid 500); 2 Mar 2015 18:06:05 -0000 Delivered-To: apmail-crunch-dev-archive@crunch.apache.org Received: (qmail 107 invoked by uid 500); 2 Mar 2015 18:06:05 -0000 Mailing-List: contact dev-help@crunch.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@crunch.apache.org Delivered-To: mailing list dev@crunch.apache.org Received: (qmail 99810 invoked by uid 500); 2 Mar 2015 18:06:05 -0000 Delivered-To: apmail-incubator-crunch-dev@incubator.apache.org Received: (qmail 99782 invoked by uid 99); 2 Mar 2015 18:06:05 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Mar 2015 18:06:05 +0000 Date: Mon, 2 Mar 2015 18:06:04 +0000 (UTC) From: "Josh Wills (JIRA)" To: crunch-dev@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CRUNCH-502) OutputFormat has inconsistent context state in interface functions MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CRUNCH-502?page=3Dcom.atlassian= .jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1434= 3496#comment-14343496 ]=20 Josh Wills commented on CRUNCH-502: ----------------------------------- Okay, those seem fixable. I'll dig in. > OutputFormat has inconsistent context state in interface functions > ------------------------------------------------------------------ > > Key: CRUNCH-502 > URL: https://issues.apache.org/jira/browse/CRUNCH-502 > Project: Crunch > Issue Type: Bug > Components: IO > Affects Versions: 0.12.0 > Reporter: M=C4=81rti=C5=86=C5=A1 Kalv=C4=81ns > > I created example project to demonstrate problematic behaviour: > https://github.com/sisidra/crunch-ofb > 1. FormatBundle config is not populated to Configuration in checkOutputSp= ecs: > https://github.com/sisidra/crunch-ofb/blob/master/src/main/java/com/spoti= fy/crunch/bugreport/MyOutputFormat.java#L39 > {code} > 15/03/02 15:40:24 INFO bugreport.MyOutputFormat: my.config.key (checkOutp= utSpecs): null > 15/03/02 15:40:24 ERROR bugreport.MyOutputFormat: Wrong my.config.key val= ue in checkOutputSpecs! > {code} > 2. TaskAttemptContext. getTaskAttemptID().toString() is different in getR= ecordWriter and getOutputCommitter: > {code} > 2015-03-02 15:40:38,960 INFO [main] com.spotify.crunch.bugreport.MyOutput= Format: TaskAttemptID (getOutputCommitter): attempt_1422406067005_0121_m_00= 0000_0 > ... > 2015-03-02 15:40:39,789 INFO [main] com.spotify.crunch.bugreport.MyOutput= Format: TaskAttemptID (getRecordWriter): attempt_1422406067005_out0_0121_m_= 000000_0 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)