drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bhallamudi Venkata Siva Kamesh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4120) dir0 does not work when the directory structure contains Avro files
Date Wed, 09 Dec 2015 05:28:11 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15048070#comment-15048070
] 

Bhallamudi Venkata Siva Kamesh commented on DRILL-4120:
-------------------------------------------------------

Attached patch does not contain schema support for directory. It uses, as of now, *DynamicDrillTable*.

To add schema support for directory, I think, we need to cache schema at every directory level
(which is union schema of all the files under a directory recursively). Otherwise, constructing
this for every query may be very costly operation. If so, will implement it as a separate
JIRA.
Any comments?

> dir0 does not work when the directory structure contains Avro files
> -------------------------------------------------------------------
>
>                 Key: DRILL-4120
>                 URL: https://issues.apache.org/jira/browse/DRILL-4120
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning & Optimization
>    Affects Versions: 1.3.0
>            Reporter: Stefán Baxter
>            Assignee: Bhallamudi Venkata Siva Kamesh
>             Fix For: 1.5.0
>
>         Attachments: 0001-DRILL-4120-Support-reading-directories-having-avro-f.patch
>
>
> Any select statment containing dirN fails if the target directory structure contains
Avro files.
> Steps to test:
> 1. create a simple directory structure
> 2. copy an avro file into each directory
> 3. execute a query containing dir0
> outcome:
> Error: VALIDATION ERROR: From line 1, column 117 to line 1, column 120: Column 'dir0'
not found in any table



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message