hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joydeep Sen Sarma (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1464) improve test query performance
Date Wed, 21 Jul 2010 15:45:49 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12890753#action_12890753

Joydeep Sen Sarma commented on HIVE-1464:

i think i found why it's happening.

these lines in build-common.xml:

    <copy todir="${test.data.dir}">
      <fileset dir="${test.src.data.dir}">
        <exclude name="**/.svn"/>

are copying .gitignore to test/data/warehouse/src

it remains there after this. the reason this is happening now is that 'cleanup()' happens
via metadata commands - and there are no tables in metadata at the beginning of the test.
so nothing gets deleted. then a whole bunch of 'load data' commands are executed - which also
don't delete old stuff.

after this - earlier each test would call cleanup() again - which would delete the entire
directory. but now this is not done anymore. so the .gitignore stays there.

lots of ways of fixing this - but this whole data/warehouse dir and then copying it recursively
is totally unnecessary i think. i will file a separate jira.

> improve test query performance
> ------------------------------
>                 Key: HIVE-1464
>                 URL: https://issues.apache.org/jira/browse/HIVE-1464
>             Project: Hadoop Hive
>          Issue Type: Test
>          Components: Testing Infrastructure
>            Reporter: Joydeep Sen Sarma
>            Assignee: Joydeep Sen Sarma
>             Fix For: 0.7.0
>         Attachments: 1464.1.patch
> clientpositive/negative tests are extremely slow.
> one major problem seems to be that all the test warehouse tables are deleted and created/re-populated
for each test. most of the times this is not required and if we can fix this the tests will
run much faster.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message