drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Barclay (Drill) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-1805) view not found if view file directory contains child with colon in simple name
Date Wed, 30 Sep 2015 21:19:04 GMT

     [ https://issues.apache.org/jira/browse/DRILL-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daniel Barclay (Drill) updated DRILL-1805:
------------------------------------------
    Description: 
Scanning the file system for view files fails (resulting in "Table 'vv' not found" errors)
if the directory being scanned for view files contains a file whose simple name (last pathname
segment) contains a colon.

For example, the unit test method {{testDRILL_811View}} in Drill's {{./exec/java-exec/src/test/java/org/apache/drill/TestExampleQueries.java}}
fails if {{/tmp}} contains a file named like "{{aptitude-root.1528:JIsVaZ}}".

The cause is that Hadoop filesystem glob-pattern-matching code ({{org.apache.hadoop.fs.Globber}}'s
{{glob()}}, calling {{org.apache.hadoop.fs.Path}}'s {{Path(Path,String)}}) mixes up relative
file pathname strings and relative URI-style {{Path}} strings.  [Note:  Changes to isolate
tests from each other might su

The problem seems to be where {{glob()}} calls {{child.getPath().getName()}} to get the raw
final segment of the pathname and passes that as the second argument to {{Path.Path(Path,
String)}} (which takes URI/{{Path}} syntax) without encoding the raw segment into a relative
URI/{{Path}} string by prepending "{{./}}" because of the colon (e.g., as {{Path.Path(String,
String, String)}} does internally).

Action items:
1) Report Hadoop bug to Hadoop.
2) Review Drill's handling and propagation of the error.  


  was:
Scanning the file system for view files fails (resulting in "Table 'vv' not found" errors)
if the directory being scanned for view files contains a file whose simple name (last pathname
segment) contains a colon.

For example, the unit test method {{testDRILL_811View}} in Drill's {{./exec/java-exec/src/test/java/org/apache/drill/TestExampleQueries.java}}
fails if {{/tmp}} contains a file named like "{{aptitude-root.1528:JIsVaZ}}".

The cause is that Hadoop filesystem glob-pattern-matching code ({{org.apache.hadoop.fs.Globber}}'s
{{glob()}}, calling {{org.apache.hadoop.fs.Path}}'s {{Path(Path,String)}}) mixes up relative
file pathname strings and relative URI-style {{Path}} strings.  [Note:  Changes to isolate
tests from each other might su

The problem seems to be where {{glob()}} calls {{child.getPath().getName()}} to get the raw
final segment of the pathname and passes that as the second argument to {{Path.Path(Path,
String)}} (which (undocumentedly) takes URI/{{Path}} syntax) without encoding the raw segment
into a relative URI/{{Path}} string by prepending "{{./}}" because of the colon (e.g., as
{{Path.Path(String, String, String)}} does internally).

Action items:
1) Report Hadoop bug to Hadoop.
2) Review Drill's handling and propagation of the error.  



> view not found if view file directory contains child with colon in simple name
> ------------------------------------------------------------------------------
>
>                 Key: DRILL-1805
>                 URL: https://issues.apache.org/jira/browse/DRILL-1805
>             Project: Apache Drill
>          Issue Type: Bug
>            Reporter: Daniel Barclay (Drill)
>            Priority: Minor
>             Fix For: Future
>
>
> Scanning the file system for view files fails (resulting in "Table 'vv' not found" errors)
if the directory being scanned for view files contains a file whose simple name (last pathname
segment) contains a colon.
> For example, the unit test method {{testDRILL_811View}} in Drill's {{./exec/java-exec/src/test/java/org/apache/drill/TestExampleQueries.java}}
fails if {{/tmp}} contains a file named like "{{aptitude-root.1528:JIsVaZ}}".
> The cause is that Hadoop filesystem glob-pattern-matching code ({{org.apache.hadoop.fs.Globber}}'s
{{glob()}}, calling {{org.apache.hadoop.fs.Path}}'s {{Path(Path,String)}}) mixes up relative
file pathname strings and relative URI-style {{Path}} strings.  [Note:  Changes to isolate
tests from each other might su
> The problem seems to be where {{glob()}} calls {{child.getPath().getName()}} to get the
raw final segment of the pathname and passes that as the second argument to {{Path.Path(Path,
String)}} (which takes URI/{{Path}} syntax) without encoding the raw segment into a relative
URI/{{Path}} string by prepending "{{./}}" because of the colon (e.g., as {{Path.Path(String,
String, String)}} does internally).
> Action items:
> 1) Report Hadoop bug to Hadoop.
> 2) Review Drill's handling and propagation of the error.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message