drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Westin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-1131) Drill should ignore files in starting with . _
Date Wed, 01 Apr 2015 16:38:55 GMT

    [ https://issues.apache.org/jira/browse/DRILL-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14390951#comment-14390951
] 

Chris Westin commented on DRILL-1131:
-------------------------------------

DRILL-2577 covers this for Hadoop output files; this includes a few more options for the file
filter that [~sphillips] describes there.

> Drill should ignore files in starting with . _
> ----------------------------------------------
>
>                 Key: DRILL-1131
>                 URL: https://issues.apache.org/jira/browse/DRILL-1131
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - Parquet
>            Reporter: Ramana Inukonda Nagaraj
>            Assignee: Steven Phillips
>             Fix For: 1.0.0
>
>
> Files containing . and _ as the first characters are ignored by hive and others are these
are typically logs and status files written out by tools like mapreduce. Drill should not
read them when querying a directory containing a list of parquet files.
> Currently it fails with the error:
> message: "Failure while setting up Foreman. < AssertionError:[ Internal error: Error
while applying rule DrillPushProjIntoScan, args [rel#78:ProjectRel.NONE.ANY([]).[](child=rel#15:Subset#1.ENUMERABLE.ANY([]).[],p_partkey=$1,p_type=$2),
rel#8:EnumerableTableAccessRel.ENUMERABLE.ANY([]).[](table=[dfs, drillTestDirDencTpchSF100,
part])] ] < DrillRuntimeException:[ java.io.IOException: Could not read footer: java.io.IOException:
Could not read footer for file com.mapr.fs.MapRFileStatus@99c9d45e ] < IOException:[ Could
not read footer: java.io.IOException: Could not read footer for file com.mapr.fs.MapRFileStatus@99c9d45e
] < IOException:[ Could not read footer for file com.mapr.fs.MapRFileStatus@99c9d45e ]
< IOException:[ Open failed for file: /drill/testdata/dencSF100/part/.impala_insert_staging,
error: Invalid argument (22) ]"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message