hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashish Thusoo (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-204) Provide option to run hadoop via TestMiniMR
Date Tue, 19 May 2009 00:10:45 GMT

    [ https://issues.apache.org/jira/browse/HIVE-204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710550#action_12710550
] 

Ashish Thusoo commented on HIVE-204:
------------------------------------

getting some conflicts in build-common.xml. Can you regenerate the patch?

Also some review comments are as follows:

1. Instead of minimr.query.files is it better to just introduce an option -Dmode=miniMR so
that this can be run on any of the files. In order to run it in that mode on the two files,
we could just call TestCliDriver with that mode and pass only those files in the query.files
variable.
2. In QTestUtil.java I did not quite understand the following code:
  +      // hive.metastore.warehouse.dir needs to be set relative to the jobtracker
  +      String fsName = conf.get("fs.default.name");
  +      assert fsName != null;
  +      conf.set("hive.metastore.warehouse.dir", fsName.concat("/build/ql/test/data/warehouse/"));
  +      
  +      conf.set("mapred.job.tracker", "localhost:" + mr.getJobTrackerPort());
Where do you specify the root of the filesystem?


> Provide option to run hadoop via TestMiniMR
> -------------------------------------------
>
>                 Key: HIVE-204
>                 URL: https://issues.apache.org/jira/browse/HIVE-204
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Testing Infrastructure
>            Reporter: Raghotham Murthy
>            Assignee: Namit Jain
>         Attachments: hive.204.1.patch, hive.204.2.patch, hive.204.3.patch, hive.204.4.patch,
hive.204.5.patch, hive.204.6.patch
>
>
> Right now MapRedTask does an exec on the hadoop command line. This prevents us from stepping
through query execution code. If there were an option to run hadoop via MiniMR, this will
create tasks within the same VM and so allows stepping into the execution code. See, src/test/org/apache/hadoop/mapred/TestMiniMRWithDFS.java
in the hadoop source code for an example.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message