Return-Path: X-Original-To: apmail-pig-dev-archive@www.apache.org Delivered-To: apmail-pig-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D008F104FB for ; Sat, 18 Jan 2014 03:45:35 +0000 (UTC) Received: (qmail 70862 invoked by uid 500); 18 Jan 2014 03:45:30 -0000 Delivered-To: apmail-pig-dev-archive@pig.apache.org Received: (qmail 70810 invoked by uid 500); 18 Jan 2014 03:45:23 -0000 Mailing-List: contact dev-help@pig.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@pig.apache.org Delivered-To: mailing list dev@pig.apache.org Received: (qmail 70796 invoked by uid 500); 18 Jan 2014 03:45:20 -0000 Delivered-To: apmail-hadoop-pig-dev@hadoop.apache.org Received: (qmail 70793 invoked by uid 99); 18 Jan 2014 03:45:20 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 18 Jan 2014 03:45:20 +0000 Date: Sat, 18 Jan 2014 03:45:20 +0000 (UTC) From: "Aniket Mokashi (JIRA)" To: pig-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (PIG-2207) Support custom counters for aggregating warnings from different udfs MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/PIG-2207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-2207: -------------------------------- Status: Patch Available (was: Open) > Support custom counters for aggregating warnings from different udfs > -------------------------------------------------------------------- > > Key: PIG-2207 > URL: https://issues.apache.org/jira/browse/PIG-2207 > Project: Pig > Issue Type: Improvement > Reporter: Thejas M Nair > Assignee: Aniket Mokashi > Labels: newbie > Attachments: PIG-2207-1.patch > > > Pig allows udfs to aggregate warning messages instead of writing out a separate warning message each time. Udfs can do this by logging the warning using EvalFunc.warn(String msg, Enum) call. But the udfs are forced to use PigWarning class if the warning needs to be printed at the end of the pig script . > For example, with the changes in PIG-2191, some of the builtin udfs are using PigWarning.UDF_WARNING_1 as argument in calls to EvalFunc.warn. This will result in the warning count being printed on STDERR - > {code} > 2011-08-05 22:10:29,285 [main] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Encountered Warning UDF_WARNING_1 2 time(s). > 2011-08-05 22:10:29,285 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success! > {code} > But it would be better if a udf such as the LOWER udf could use a custom warning counter, and the STDERR is like - > {code} > 2011-08-05 22:10:29,285 [main] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Encountered Warning LOWER_FUNC_INPUT_WARNING 2 time(s). > 2011-08-05 22:10:29,285 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success! > {code} > A new function could be added to support this - (something like) EvalFunc.warn(String warnName, String warnMsg); A specific counter group could be used for udf warnings (see org.apache.hadoop.mapred.Counters), and counters for that group could be done during final warning aggregation in done in MapReduceLauncher.computeWarningAggregate(). -- This message was sent by Atlassian JIRA (v6.1.5#6160)