pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Niels Basjes (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-4639) Add better parser for Apache HTTPD access log.
Date Sun, 09 Aug 2015 20:03:46 GMT

    [ https://issues.apache.org/jira/browse/PIG-4639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14679311#comment-14679311

Niels Basjes commented on PIG-4639:

It is quite a bit of code and I am currently adding a few new useful features (like transparently
being able to handle multiple LogFormat specifications and support for nginx logfiles).
So for now leaving it where it is seems best for the short term.

I'll look into Apache Commons and have a discussion if it can be added there.

> Add better parser for Apache HTTPD access log.
> ----------------------------------------------
>                 Key: PIG-4639
>                 URL: https://issues.apache.org/jira/browse/PIG-4639
>             Project: Pig
>          Issue Type: New Feature
>          Components: piggybank
>    Affects Versions: 0.15.0
>            Reporter: Niels Basjes
>            Assignee: Niels Basjes
>             Fix For: 0.16.0
>         Attachments: PIG-4639-20150723-classnotfound.patch, PIG-4639-20150725.patch,
> Currently there are two parsers for Apache HTTPD acces log files in piggybank that only
allow parsing the 'combined' and 'common' logformats. These two also only parse the 'basics'.
> This is proposed patch to add the existing https://github.com/nielsbasjes/logparser (Apache
2.0 license) as an 'out of the box' parser to piggybank. 
> This parser parses the logfile using the LogFormat specification used to writte it. Almost
all LogFormat specifiers are supported and as such adds easy parsing capabilities for (almost)
all custom logformats used in production scenarios. 
> This parser also goes much deeper in the sense that it allows extracting things like
the value of a cookie or the value of a  query string parameter.

This message was sent by Atlassian JIRA

View raw message