hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashish Thusoo (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-204) Provide option to run hadoop via TestMiniMR
Date Tue, 05 May 2009 13:42:30 GMT

    [ https://issues.apache.org/jira/browse/HIVE-204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12706024#action_12706024
] 

Ashish Thusoo commented on HIVE-204:
------------------------------------

The following are my comments:

1. In QTestUtil - you probably don't need reflection to call getFileSystem. You should just
use reflection for getting the MiniDFSCluster.
2. In build.xml and build-common.xml - can you move the new pathelements from the common-classpath
to the test classpath in ql/build.xml. The tests for other
modules I presume don't need all these jars.
3. In QTestGenTask.java - what do you need the clusterQueryFiles for, can't you just use the
the split extensions that you have made to the queryFile member variable?


> Provide option to run hadoop via TestMiniMR
> -------------------------------------------
>
>                 Key: HIVE-204
>                 URL: https://issues.apache.org/jira/browse/HIVE-204
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Testing Infrastructure
>            Reporter: Raghotham Murthy
>            Assignee: Namit Jain
>         Attachments: hive.204.1.patch, hive.204.2.patch, hive.204.3.patch, hive.204.4.patch,
hive.204.5.patch
>
>
> Right now MapRedTask does an exec on the hadoop command line. This prevents us from stepping
through query execution code. If there were an option to run hadoop via MiniMR, this will
create tasks within the same VM and so allows stepping into the execution code. See, src/test/org/apache/hadoop/mapred/TestMiniMRWithDFS.java
in the hadoop source code for an example.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message