hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brock Noland <br...@cloudera.com>
Subject Re: MR logs during .q file tests
Date Wed, 06 Nov 2013 16:16:10 GMT
Hi,

The old build did not have MR log by default.  The new maven build does and
the logs should be in ${java.io.tmpdir}/${user.name} or /tmp/brock for me.

The ptest build overrides this to put them in the log directory, see the
hiveptest-* files here:

http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-150/succeeded/TestCliDriver-alter3.q-smb_mapjoin_2.q-exim_02_00_part_empty.q-and-12-more/

Ideally we'd put the mr logs in ${test.tmp.dir}/log in the local maven
builds as well. We'd just need to change some items.

Brock


On Tue, Nov 5, 2013 at 8:40 PM, Sergey Shelukhin <sergey@hortonworks.com>wrote:

> Hi.
>
> Is there any way to get MR logs during .q tests? Ideally, without changing
> code.
>
> MR logs by default don't go into hive.log, even though the logging from
> caller alleges that's where they are to be found.
> Previously, a workaround existed where you could force MR to local by
> changing code in MapRedTask, but I had problems with it after mavenization
> - it fails w/out of memory. That can probably be fixed by setting some
> magic variable somewhere (I did set MAVEN_OPTS from FAQ), but this is a
> hack, requires rebuild and later revert/checkout, and doesn't catch all the
> bugs - e.g. serialization issues in operators are not visible w/local MR.
>
> So I wonder if there's good way to obtain these logs. Or if there should be
> a JIRA to add it :)
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>



-- 
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message