bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Harsch <thar...@yarcdata.com>
Subject Re: smoke tests in 0.7.0
Date Tue, 23 Sep 2014 23:49:50 GMT
Hi Jay,
I looked at the README in master, and it has since been updated.  It was the README in the
0.7.0 distro that was the issue.

I tried running the gradle smokes in master and I am getting this:
:pig:test
ENV VARIABLE: PIG_HOME = /usr/lib/pig

org.apache.bigtop.itest.hadoop.mapreduce.TestPigSmoke > test FAILED
    java.lang.AssertionError at TestPigSmoke.groovy:56

1 test completed, 1 failed
:pig:test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':pig:test'.
> There were failing tests. See the report at: file:///root/git/bigtop/bigtop-tests/smoke-tests/pig/build/reports/tests/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to
get more log output.

BUILD FAILED

Total time: 32.62 secs


From: jay vyas <jayunit100.apache@gmail.com<mailto:jayunit100.apache@gmail.com>>
Reply-To: "user@bigtop.apache.org<mailto:user@bigtop.apache.org>" <user@bigtop.apache.org<mailto:user@bigtop.apache.org>>
Date: Tuesday, September 23, 2014 1:43 PM
To: "user@bigtop.apache.org<mailto:user@bigtop.apache.org>" <user@bigtop.apache.org<mailto:user@bigtop.apache.org>>
Subject: Re: smoke tests in 0.7.0

Hi tim ,

1) thanks for noting.  looks like time for a README update.  the "package" part needs to be
modified .   could you create a JIRA for that?

2) Also, iirc , is there a /user/root/ directory in hdfs?  There needs to be one so that it
can write the local files.

weve also completed a simpler way of running the tests recently in the "smoke-tests" package
which you can try (see the README).


On Tue, Sep 23, 2014 at 4:31 PM, Tim Harsch <tharsch@yarcdata.com<mailto:tharsch@yarcdata.com>>
wrote:
Hi all,
I am having trouble getting the smoke tests to run in 0.7.0.  I guess my first question is:
which systems are these smokes being validated against at release time?  I'm wondering if
it should work for CentOS6

For installing bigtop, I followed instructions in the Hadoop for Dummies book.  I install
a CentOS6 vmWare player.  And installed bigtop via the following:
"yum install hadoop\* mahout\* oozie\* hbase\* hive\* hue\* pig\* zookeeper\* giraph\*"

I believe I now have a working hadoop system, "jps" confirms running services.  But, almost
none of the tests run:

I've read the top level README, and followed the wiki https://cwiki.apache.org/confluence/display/BIGTOP/Running+integration+and+system+tests.

README:
Following the first step for testing yields:

[root@localhost bigtop-0.7.0]# mvn install -DskipTests -DskipITs -DperformRelease -f bigtop-tests/test-execution/smokes/package/pom.xml
[INFO] Scanning for projects...
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR]   The project  (/root/bigtop/bigtop-0.7.0/bigtop-tests/test-execution/smokes/package/pom.xml)
has 1 error
[ERROR]     Non-readable POM /root/bigtop/bigtop-0.7.0/bigtop-tests/test-execution/smokes/package/pom.xml:
/root/bigtop/bigtop-0.7.0/bigtop-tests/test-execution/smokes/package/pom.xml (No such file
or directory)
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following
articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException

>From the error we see there is no package module.

WIKI:
Basically advises to run a submodule…  hadoop seems like a good one.  If I run it though
I get 8 of 8 tests failing.  Here is the first sign of trouble:

Running org.apache.bigtop.itest.hadoop.mapreduce.TestHadoopSmoke
Failed command: hadoop fs -rmr test.hadoopsmoke.1411495413183/cachefile/out
hadoop jar /usr/lib/hadoop-mapreduce/hadoop-streaming.jar -D mapred.map.tasks=1 -D mapred.reduce.tasks=1
-D mapred.job.name<http://mapred.job.name>=Experiment -cacheArchive file:////user/root/test.hadoopsmoke.1411495413183/cachefile/cachedir.jar#testlink
-input test.hadoopsmoke.1411495413183/cachefile/input.txt -mapper map.sh -file map.sh -reducer
cat -output test.hadoopsmoke.1411495413183/cachefile/out -verbose
error code: 2
stdout: [2014-09-23 20:03:38,522 WARN  [main] streaming.StreamJob (StreamJob.java:parseArgv(290))
- -file option is deprecated, please use generic option -files instead., 2014-09-23 20:03:38,913
WARN  [main] streaming.StreamJob (StreamJob.java:parseArgv(337)) - -cacheArchive option is
deprecated, please use -archives instead., STREAM: addTaskEnvironment=HADOOP_ROOT_LOGGER=,
STREAM: shippedCanonFiles_=[/root/bigtop/bigtop-0.7.0/bigtop-tests/test-execution/smokes/hadoop/target/map.sh],
STREAM: shipped: true /root/bigtop/bigtop-0.7.0/bigtop-tests/test-execution/smokes/hadoop/target/map.sh,
STREAM: cmd=map.sh, STREAM: cmd=null, STREAM: shipped:

Seems like I must have something wrong.

Any help appreciated.
Thanks,
Tim





--
jay vyas

Mime
View raw message