hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-476) given a date that can match a SimpleDateFormat want to be able to extract arbitrary SimpleDateFormat data, like day or year
Date Fri, 10 Oct 2008 17:17:44 GMT

    [ https://issues.apache.org/jira/browse/PIG-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638594#action_12638594
] 

Alan Gates commented on PIG-476:
--------------------------------

Pig Latin has a way that you can define constructors for a UDF:

{code}
define MyDateExtractor org.apache.pig.piggybank.evaluation.util.apachelogparser.DateExtractor("MM-dd-yyyy");
...
A = FOREACH raw GENERATE DateExtractor(dayTime);
{code}

Whatever you pass as an argument in the define method is passed to the constructor of the
UDF.  In your case, this would allow you to pass the date format up front, parse it once,
and avoid parsing it on every tuple passed to the UDF.  This should give you a significant
performance boost.  The downside of this is that if you want to use the same UDF with different
date formats in the same query you'd have to alias it different ways.  

It's up to you whether to choose flexibility or performance here.

> given a date that can match a SimpleDateFormat want to be able to extract arbitrary SimpleDateFormat
data, like day or year
> ---------------------------------------------------------------------------------------------------------------------------
>
>                 Key: PIG-476
>                 URL: https://issues.apache.org/jira/browse/PIG-476
>             Project: Pig
>          Issue Type: New Feature
>            Reporter: Earl Cahill
>         Attachments: DateExtractor-PIG-476
>
>
> Want to be able to do something like
> A = FOREACH raw GENERATE org.apache.pig.piggybank.evaluation.util.apachelogparser.DateExtractor(dayTime,
"yyyy", "dd/MMM/yyyy:HH:mm:ss");
> to extract the year, or if your date is formatted as
> dd/MMM/yyyy:HH:mm:ss Z
> you could do something like
> A = FOREACH raw GENERATE org.apache.pig.piggybank.evaluation.util.apachelogparser.DateExtractor(dayTime,
"MM-dd-yyyy");
> to grab out the day

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message