drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacques Nadeau (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-3423) Add New HTTPD format plugin
Date Sun, 18 Oct 2015 23:38:05 GMT

    [ https://issues.apache.org/jira/browse/DRILL-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14962689#comment-14962689
] 

Jacques Nadeau commented on DRILL-3423:
---------------------------------------

Based on [~nielsbasjes] comments, I think it makes sense to add an additional property as
part of the HttpdLogFormatConfig that provides a set of mappings.

{code}
@JsonTypeName("httpd")
public static class HttpdLogFormatConfig implements FormatPluginConfig {
  public String format;

  @JsonTypeName("text") @JsonInclude(Include.NON_DEFAULT)
  public List<Mapping> mapping
}

public static class HttpdLogMapping {
  public String httpdPath;
  public String outputPath;
}

{code}

Then we can set it to a default mapping (like I originally did) unless the user has special
needs. Drill's primary mechanism for remapping is SQL projections and views as opposed to
exposing in a format plugin level. We would need to parse the outputPath according to a drill
map path and use the type detection as Niels suggested. I'm probably not going to make much
progress on this for some time so if someone else wants to move it forward, they should feel
free.




> Add New HTTPD format plugin
> ---------------------------
>
>                 Key: DRILL-3423
>                 URL: https://issues.apache.org/jira/browse/DRILL-3423
>             Project: Apache Drill
>          Issue Type: New Feature
>          Components: Storage - Other
>            Reporter: Jacques Nadeau
>            Assignee: Jacques Nadeau
>             Fix For: 1.3.0
>
>
> Add an HTTPD logparser based format plugin.  The author has been kind enough to move
the logparser project to be released under the Apache License.  Can find it here:
> <dependency>
>     <groupId>nl.basjes.parse.httpdlog</groupId>
>     <artifactId>httpdlog-parser</artifactId>
>     <version>2.0</version>
> </dependency>
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message