drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-951) CSV header row should be parsed
Date Tue, 03 Nov 2015 04:56:27 GMT

    [ https://issues.apache.org/jira/browse/DRILL-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14986688#comment-14986688
] 

ASF GitHub Bot commented on DRILL-951:
--------------------------------------

Github user jacques-n commented on a diff in the pull request:

    https://github.com/apache/drill/pull/232#discussion_r43716119
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/TextParsingSettings.java
---
    @@ -57,6 +56,14 @@ public void set(TextFormatConfig config){
         this.delimiter = bSafe(config.getFieldDelimiter(), "fieldDelimiter");
         this.comment = bSafe(config.getComment(), "comment");
         this.skipFirstLine = config.isSkipFirstLine();
    +    //TODO: avoid specific extension based check
    --- End diff --
    
    You can make this work correctly by using a format plugin config. You can add csvh here:
    
    https://github.com/apache/drill/blob/master/exec/java-exec/src/main/resources/bootstrap-storage-plugins.json
    
    Basically, create a new item in the map. Note how the text formats are a subset of the
TextFormatConfig class. You can add a new property that tells you to perform header extraction.
See the properties here [1] that correspond to the format properties: [1] https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/TextFormatPlugin.java#L134


> CSV header row should be parsed
> -------------------------------
>
>                 Key: DRILL-951
>                 URL: https://issues.apache.org/jira/browse/DRILL-951
>             Project: Apache Drill
>          Issue Type: New Feature
>          Components: Storage - Text & CSV
>            Reporter: Tomer Shiran
>            Assignee: Abhijit Pol
>             Fix For: Future
>
>
> CSV reader is currently treating header names like regular rows. There should be a way
to treat the header row as the column names (optional?).
> I exported this dataset to a CSV: https://data.sfgov.org/Public-Safety/SFPD-Incidents-Previous-Three-Months/tmnf-yvry



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message