hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gunther Hagleitner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7271) Speed up unit tests
Date Wed, 25 Jun 2014 09:44:24 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14043251#comment-14043251
] 

Gunther Hagleitner commented on HIVE-7271:
------------------------------------------

[~leftylev] Just a thought: This parameter is useful to run unit tests quicker, but in general
not something i would recommend anyone to set to true outside of that. It basically runs the
hashtable generation for mapjoins in the CLI process - something that could give you memory
issues. I'll put that in the release notes, but in general I think parameters like this needn't
be exposed to users. In these cases should we just leave it out of the xml?

> Speed up unit tests
> -------------------
>
>                 Key: HIVE-7271
>                 URL: https://issues.apache.org/jira/browse/HIVE-7271
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Gunther Hagleitner
>            Assignee: Gunther Hagleitner
>              Labels: TODOC14
>             Fix For: 0.14.0
>
>         Attachments: HIVE-7271.1.patch, HIVE-7271.2.patch, HIVE-7271.3.patch, HIVE-7271.4.patch,
HIVE-7271.5.patch, HIVE-7271.6.patch, HIVE-7271.7.patch
>
>
> Did some experiments to see if there's a way to speed up unit tests. TestCliDriver seemed
to take a lot of time just spinning up/tearing down JVMs. I was also curious to see if running
everything on a ram disk would help.
> Results (I ran tests up to authorization_2):
> - Current setup: 40 minutes
> - Single JVM (not using child JVM to run all queries): 8 minutes
> - Single JVM + ram disk: 7 minutes
> So the ram disk didn't help that much. But running tests in single JVM seems worthwhile
doing.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message