drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacques Nadeau (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-3423) Add New HTTPD format plugin
Date Thu, 05 Nov 2015 23:25:27 GMT

    [ https://issues.apache.org/jira/browse/DRILL-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992699#comment-14992699

Jacques Nadeau commented on DRILL-3423:

I think everyone is focusing too much on what the parser is capable of doing. That should
be the last thing we focus on. We should start with the user API. Let's take an example log
file format and decide what the output table should look like. Then let's talk about how we
could vary things to provide more flexibility. 

I proposed a particular format. When you guys saw it, you thought that we needed more flexibility.
I then proposed a modification to provide flexibility around mapping between log file fields
and table fields. 

[~nielsbasjes], I appreciate your statements about the flexibility of the plugin and agree
it is very powerful. What we need to figure out is what is the right way to expose that power
in a SQL context. It doesn't make sense for Drill to support custom dissectors. If someone
wanted to provide that capability, they would implement a Drill UDF (a similarly easy thing
to implement).

[~jscott@maprtech.com], with regards to your comment "This model makes it extremely difficult
to support mapping of data types", my whole suggestion there was to expose more flexibility
by using the mapping suggestion above. I'm thinking that maybe I wasn't clear enough in my
recommendation and you misunderstood what I was suggesting.

So let's start with what a user would want. Then figure out how to implement that. I think
that will make this discussion substantially less conceptual.

> Add New HTTPD format plugin
> ---------------------------
>                 Key: DRILL-3423
>                 URL: https://issues.apache.org/jira/browse/DRILL-3423
>             Project: Apache Drill
>          Issue Type: New Feature
>          Components: Storage - Other
>            Reporter: Jacques Nadeau
>            Assignee: Jim Scott
>             Fix For: 1.4.0
> Add an HTTPD logparser based format plugin.  The author has been kind enough to move
the logparser project to be released under the Apache License.  Can find it here:
> <dependency>
>     <groupId>nl.basjes.parse.httpdlog</groupId>
>     <artifactId>httpdlog-parser</artifactId>
>     <version>2.0</version>
> </dependency>

This message was sent by Atlassian JIRA

View raw message