drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venki Korukanti (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-2733) Make dfs_test.tmp schema location on local fs exclusive to test JVM fork.
Date Thu, 09 Apr 2015 05:51:12 GMT
Venki Korukanti created DRILL-2733:

             Summary: Make dfs_test.tmp schema location on local fs exclusive to test JVM
                 Key: DRILL-2733
                 URL: https://issues.apache.org/jira/browse/DRILL-2733
             Project: Apache Drill
          Issue Type: Improvement
          Components: Tools, Build & Test
    Affects Versions: 0.8.0
            Reporter: Venki Korukanti
            Assignee: Venki Korukanti
             Fix For: 0.9.0

Currently {{dfs_test.tmp}} workspace location on local filesystem is hardcoded to {{/tmp/drilltest}}.
If a test creates a view or new table, it is created under this location. Problem is when
two or more test forks are running in parallel, sharing the {{dfs_test.tmp}} workspace location
causes synchronization issues.

For example: {{TestViewSupport#view1}} creates a view {{testView1}} in workspace {{dfs_test.tmp}}.
This causes a new view file under {{/tmp/drilltest}} directory. At this point parallel running
test {{TestInfoSchema#showTables}} makes a call to list tables in {{dfs_test.tmp}} workspace.
Show tables returns {{testView1}} as one of the tables in {{dfs_test.tmp}} workspace which
it is not expecting causing the {{TestInfoSchema#showTables}} to fail. 

Proposed solution is:
When setting up Drill test cluster in {{BaseTestQuery}} (root class for most tests), modify
workspace {{dfs_test.tmp}} location to point to a temp directory created using {{Files.createTempDir}}.
If two or more processes call {{Files.createTempDir}} at the same time each one guarateed
to get a exclusive directory as long as there are no more than 1000 calls per millisecond.

For JDBC, we rely on property {{drillJDBCUnitTests}} in connection properties to setup exclusive
directory for {{dfs_test.tmp}} workspace. 

This message was sent by Atlassian JIRA

View raw message