hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitriy V. Ryaboy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1427) Monitor and kill runaway UDFs
Date Thu, 27 May 2010 06:59:38 GMT

    [ https://issues.apache.org/jira/browse/PIG-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12872120#action_12872120

Dmitriy V. Ryaboy commented on PIG-1427:

Attaching a new version, featuring:
* default values (for primitives and strings only)
* custom timeout and error handlers
* apache headers
* ivy changes to include the guava jar

Guava is mavenized, but the mavenization is broken -- they appear to have used the wrong md5
or something, so it ivy refuses to pull it down. Easy enough to stick the jar (from the above
url) into lib/.

I should probably move the MonitoredUDF annotation to some more public place, and add the
stability/visibility labels. Anything other than that you guys want to see? 


> Monitor and kill runaway UDFs
> -----------------------------
>                 Key: PIG-1427
>                 URL: https://issues.apache.org/jira/browse/PIG-1427
>             Project: Pig
>          Issue Type: New Feature
>    Affects Versions: 0.8.0
>            Reporter: Dmitriy V. Ryaboy
>            Assignee: Dmitriy V. Ryaboy
>         Attachments: monitoredUdf.patch, monitoredUdf.patch
> As a safety measure, it is sometimes useful to monitor UDFs as they execute. It is often
preferable to return null or some other default value instead of timing out a runaway evaluation
and killing a job. We have in the past seen complex regular expressions lead to job failures
due to just half a dozen (out of millions) particularly obnoxious strings.
> It would be great to give Pig users a lightweight way of enabling UDF monitoring.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message