hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Russell Melick (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-1644) use filter pushdown for automatically accessing indexes
Date Sat, 30 Apr 2011 05:52:03 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027294#comment-13027294
] 

Russell Melick commented on HIVE-1644:
--------------------------------------

Unfortunately, having an empty array gives out of bounds exceptions.

{{
java.lang.ArrayIndexOutOfBoundsException: 0
    [junit] 	at org.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:309)
}}

I tried looking through the code at other places we call setInputPaths. In a few places, we
create a directory by calling

{{
Path dir = new Path(System.getProperty("test.data.dir", "."));
FileInputFormat.setInputPaths(job, dir);
}} 


I tried using this, but unfortunately, this also gives an exception.
{{
java.io.IOException: cannot find dir = file:/Users/rmelick/hive/ql/mapred in pathToPartitionInfo:
[pfile:/Users/rmelick/hive/build/ql/test/data/warehouse/temp]
}}

The ql/mapred directory does not exist on my computer, but I also tried changing it to just
hive/ql, and it also failed with the same exception.  I'm not sure if TestFlatFileInputFormat:146
is creating the temp file like you were thinking.  I haven't tried doing it like that.

> use filter pushdown for automatically accessing indexes
> -------------------------------------------------------
>
>                 Key: HIVE-1644
>                 URL: https://issues.apache.org/jira/browse/HIVE-1644
>             Project: Hive
>          Issue Type: Improvement
>          Components: Indexing
>    Affects Versions: 0.8.0
>            Reporter: John Sichi
>            Assignee: Russell Melick
>         Attachments: HIVE-1644.1.patch, HIVE-1644.10.patch, HIVE-1644.11.patch, HIVE-1644.12.patch,
HIVE-1644.13.patch, HIVE-1644.14.patch, HIVE-1644.15.patch, HIVE-1644.16.patch, HIVE-1644.17.patch,
HIVE-1644.18.patch, HIVE-1644.2.patch, HIVE-1644.3.patch, HIVE-1644.4.patch, HIVE-1644.5.patch,
HIVE-1644.6.patch, HIVE-1644.7.patch, HIVE-1644.8.patch, HIVE-1644.9.patch, hive.log
>
>
> HIVE-1226 provides utilities for analyzing filters which have been pushed down to a table
scan.  The next step is to use these for selecting available indexes and generating access
plans for those indexes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message