hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Yang (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HIVE-1377) getPartitionDescFromPath() in CombeHiveInputFormat should handle matching by path
Date Fri, 28 May 2010 21:58:39 GMT

     [ https://issues.apache.org/jira/browse/HIVE-1377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Paul Yang updated HIVE-1377:
----------------------------

    Description: 
The use case is:
{code}
dir = hdfs://host:9000/user/warehouse/tableName/abc
pathToPartitionInfo = {/user/warehouse/tableName : myPart}
{code}

Then calling 
{code} 
getPartitionDescFromPath(dir, pathToPartitionInfo)
{code}

will throw an IOException because /user/warehouse/tableName is not a prefix of hdfs://host:9000/user/warehouse/tableName/abc.
Currently, this is not an issue but will come up if CombineFileInputFormat is modified so
what the scheme and authority are not stripped out  when generating splits (see MAPREDUCE-1806).

The proposed solution is add a case where matching is done by just the path component of the
URI's.


  was:
The use case is:
{code}
dir = hdfs://host:9000/user/warehouse/tableName/abc
pathToPartitionInfo = {/user/warehouse/tableName : myPart}
{code}

Then calling 
{code} 
getPartitionDescFromPath(dir, pathToPartitionInfo)
{code}

will throw an IOException. Currently, this is not an issue but will come up if CombineFileInputFormat
is modified to so what the scheme and authority are not stripped out in MAPREDUCE-1806.

The proposed solution is add a case where matching is done by just the path component of the
URI's.



> getPartitionDescFromPath() in CombeHiveInputFormat should handle matching by path
> ---------------------------------------------------------------------------------
>
>                 Key: HIVE-1377
>                 URL: https://issues.apache.org/jira/browse/HIVE-1377
>             Project: Hadoop Hive
>          Issue Type: Bug
>    Affects Versions: 0.6.0
>            Reporter: Paul Yang
>            Assignee: Paul Yang
>
> The use case is:
> {code}
> dir = hdfs://host:9000/user/warehouse/tableName/abc
> pathToPartitionInfo = {/user/warehouse/tableName : myPart}
> {code}
> Then calling 
> {code} 
> getPartitionDescFromPath(dir, pathToPartitionInfo)
> {code}
> will throw an IOException because /user/warehouse/tableName is not a prefix of hdfs://host:9000/user/warehouse/tableName/abc.
Currently, this is not an issue but will come up if CombineFileInputFormat is modified so
what the scheme and authority are not stripped out  when generating splits (see MAPREDUCE-1806).
> The proposed solution is add a case where matching is done by just the path component
of the URI's.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message