Return-Path: 
 <dev-return-3729-apmail-crunch-dev-archive=crunch.apache.org@crunch.apache.org>
X-Original-To: apmail-crunch-dev-archive@www.apache.org
Delivered-To: apmail-crunch-dev-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 4034010898
	for <apmail-crunch-dev-archive@www.apache.org>;
 Thu, 16 Jan 2014 17:40:38 +0000 (UTC)
Received: (qmail 63286 invoked by uid 500); 16 Jan 2014 17:40:37 -0000
Delivered-To: apmail-crunch-dev-archive@crunch.apache.org
Received: (qmail 63219 invoked by uid 500); 16 Jan 2014 17:40:37 -0000
Mailing-List: contact dev-help@crunch.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:dev-help@crunch.apache.org>
List-Unsubscribe: <mailto:dev-unsubscribe@crunch.apache.org>
List-Post: <mailto:dev@crunch.apache.org>
List-Id: <dev.crunch.apache.org>
Reply-To: dev@crunch.apache.org
Delivered-To: mailing list dev@crunch.apache.org
Delivered-To: moderator for dev@crunch.apache.org
Received: (qmail 50019 invoked by uid 99); 16 Jan 2014 16:48:30 -0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of d.dierickx@gmail.com designates
 209.85.214.180 as permitted sender)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type;
        bh=NQmBPQvoRdiBemnWmqMAhC2AJQUq0FoxgF8rCBUbWsY=;
        b=kpLH6MFwoLkOByPj2j/Nzx8yVyFFDfk1HcQSzQ6ttoX3crn0g/IfjuJ8biunqcMWuA
         R3hFtGKguQ/iFvBrs6iF+w3cVop6wg+jg2U/afc7iuwKCyTD3q4jPAyQ32whSffMzBSN
         2656HYpxAixb+qoDA7Mb2EfOz+aus+siurUus4YdyvRSx0xD8Ma4YEVPLGu085HV1M9n
         QkjBBqvXWrNrm0W6igfE5f6FbBXcPov8b8nAiSz3agIkESQB/1l7qEpZXxLAclIdnTJo
         vbd7S1RhabFQZ/QSP/+aVN9exshcZVyLQ7ME3BFLnG1bX56NgBsrnKYl4Y9x13xyQVH3
         eAWA==
X-Received: by 10.182.129.201 with SMTP id ny9mr8037317obb.0.1389890440127;
 Thu, 16 Jan 2014 08:40:40 -0800 (PST)
MIME-Version: 1.0
In-Reply-To: 
 <CA+3fhE7wspKBjYLxeAOGLO1fWjCOpmPgwpDrF=7Z7rseDM6g3w@mail.gmail.com>
References: 
 <CA+3fhE7wspKBjYLxeAOGLO1fWjCOpmPgwpDrF=7Z7rseDM6g3w@mail.gmail.com>
From: Dierickx Dominique <d.dierickx@gmail.com>
Date: Thu, 16 Jan 2014 17:40:19 +0100
Message-ID: 
 <CA+3fhE7KtFJ5qEuVo+yd+emQxx8L86owKWd++HoaP6eyhhxvNQ@mail.gmail.com>
Subject: Fwd: Too many Crunch output counters
To: dev@crunch.apache.org
Content-Type: multipart/mixed; boundary=089e015366b4f6444104f01915db
X-Virus-Checked: Checked by ClamAV on apache.org

--089e015366b4f6444104f01915db
Content-Type: multipart/alternative; boundary=089e015366b4f6443c04f01915d9

--089e015366b4f6443c04f01915d9
Content-Type: text/plain; charset=ISO-8859-1

We're having some trouble with the amount of counters that Crunch creates
when writing to a lot of different output files (slightly more than 120).
This wouldn't be an issue if we were able to configure the maximum number
of allowed counters but unfortunately, because we are running an older
version of Hadoop, doing this is not an option and we are required to patch
Crunch locally when using a new release to leave out the counters. The
required patch (one line...) can be found in the attachment.

I'm not saying the counters should be removed but maybe it is an option to
make them configurable without paying too much of a performance penalty?

Regards,
Dominique Dierickx

--089e015366b4f6443c04f01915d9
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_quote"><div dir=3D"ltr">We&#39;re havi=
ng some trouble with the amount of counters that Crunch creates when writin=
g to a lot of different output files (slightly more than 120). This wouldn&=
#39;t be an issue if we were able to configure the maximum number of allowe=
d counters but unfortunately, because we are running an older version of Ha=
doop, doing this is not an option and we are required to patch Crunch local=
ly when using a new release to leave out the counters. The required patch (=
one line...) can be found in the attachment.<div>


<br></div><div>I&#39;m not saying the counters should be removed but maybe =
it is an option to make them configurable without paying too much of a perf=
ormance penalty?</div><div><br></div><div>Regards,</div><div>Dominique Dier=
ickx</div>


</div>
</div><br></div>

--089e015366b4f6443c04f01915d9--

--089e015366b4f6444104f01915db
Content-Type: text/plain; charset=US-ASCII; name="no-counters.diff"
Content-Disposition: attachment; filename="no-counters.diff"
Content-Transfer-Encoding: base64
X-Attachment-Id: f_hqi8jpoc0

ZGlmZiAtLWdpdCBhL2NydW5jaC1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9jcnVuY2gv
aW8vQ3J1bmNoT3V0cHV0cy5qYXZhIGIvY3J1bmNoLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBh
Y2hlL2NydW5jaC9pby9DcnVuY2hPdXRwdXRzLmphdmEKaW5kZXggY2NmNGZiNS4uNzFlM2JiZSAx
MDA2NDQKLS0tIGEvY3J1bmNoLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2NydW5jaC9p
by9DcnVuY2hPdXRwdXRzLmphdmEKKysrIGIvY3J1bmNoLWNvcmUvc3JjL21haW4vamF2YS9vcmcv
YXBhY2hlL2NydW5jaC9pby9DcnVuY2hPdXRwdXRzLmphdmEKQEAgLTEyNSw3ICsxMjUsNyBAQCBw
dWJsaWMgY2xhc3MgQ3J1bmNoT3V0cHV0czxLLCBWPiB7CiAgICAgICAgIG5hbWVkT3V0cHV0ICsg
IiciKTsKICAgICB9CiAgICAgVGFza0F0dGVtcHRDb250ZXh0IHRhc2tDb250ZXh0ID0gZ2V0Q29u
dGV4dChuYW1lZE91dHB1dCk7Ci0gICAgYmFzZUNvbnRleHQuZ2V0Q291bnRlcihDT1VOVEVSU19H
Uk9VUCwgbmFtZWRPdXRwdXQpLmluY3JlbWVudCgxKTsKKyAgICAvL2Jhc2VDb250ZXh0LmdldENv
dW50ZXIoQ09VTlRFUlNfR1JPVVAsIG5hbWVkT3V0cHV0KS5pbmNyZW1lbnQoMSk7CiAgICAgZ2V0
UmVjb3JkV3JpdGVyKHRhc2tDb250ZXh0LCBuYW1lZE91dHB1dCkud3JpdGUoa2V5LCB2YWx1ZSk7
CiAgIH0KICAgCg==
--089e015366b4f6444104f01915db--