flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: IT Test
Date Wed, 29 Jun 2016 09:39:59 GMT
Hi Vijay,

Glad we solved the problem.

Good catch with the FlinkYarnSessionCli. We should clean up the
properties file after running the Yarn tests. If you want to fix that,
you could open a pull request.

Cheers,
Max

On Wed, Jun 29, 2016 at 1:34 AM, Vijay Srinivasaraghavan
<vijikarthi@yahoo.com.invalid> wrote:
> Hi Max,
> You are right. The uber jar is built in "intsall" or "package" stage and Yarn test assumes
the flink uber jar to pre-exist. We may have to document the steps so that it will not be
missed when someone else tries in the future.
> I have also noticed errors with the "CliFrontendAddressConfigurationTest" (happens when
you execute mvn clean install/verify after running Yarn test) and the reason is because the
temporary file (https://github.com/apache/flink/blob/f2e9c521aa4e0cbc14b71c561b8c5dd46201a45a/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java#L79)
is not deleted and the test code picks up wrong job manager address. I had to manually delete
the /tmp/.yarn-properties-<USER> file before running the test code again.
> RegardsVijay
>
>
>     On Tuesday, June 28, 2016 1:53 AM, Maximilian Michels <mxm@apache.org> wrote:
>
>
>  Hi Vijay,
>
> Hasn't been failing for me whenever I pulled in the latest master.
> Neither when I run "mvn clean verify" nor when I execute the test from
> within IntelliJ. That said, there are some unstable tests that fail on
> Travis.
>
>>YARNSessionFIFOITCase.setup:71->YarnTestBase.startYARNWithConfig:343 Flink uberjar
not found
>
> Your test run fails because it can't find the Flink jar. Most likely,
> you have not run 'mvn clean install' before running the Yarn tests.
> You need to 'package' or 'install' at least flink-yarn in order to
> execute flink-yarn-tests.
>
> Cheers,
> Max
>
> On Mon, Jun 27, 2016 at 6:07 PM, Vijay Srinivasaraghavan
> <vijikarthi@yahoo.com.invalid> wrote:
>> I just pulled the latest code and ran below commands. Both of them failed and it
is consistent.
>> 1) "mvn clean install"
>> Results :
>>
>> Failed tests:  LocalFlinkMiniClusterITCase.testLocalFlinkMiniClusterWithMultipleTaskManagers:166
Thread Thread[initialSeedUniquifierGenerator,5,main] was started by the mini cluster, but
not shut down
>> Tests run: 1269, Failures: 1, Errors: 0, Skipped: 9
>> [INFO] ------------------------------------------------------------------------[INFO]
Reactor Summary:[INFO] [INFO] force-shading ...................................... SUCCESS
[  1.038 s][INFO] flink .............................................. SUCCESS [  2.097 s][INFO]
flink-annotations .................................. SUCCESS [  1.045 s][INFO] flink-shaded-hadoop
................................ SUCCESS [  0.107 s][INFO] flink-shaded-hadoop2 ...............................
SUCCESS [  5.412 s][INFO] flink-shaded-include-yarn-tests .................... SUCCESS [ 
6.747 s][INFO] flink-shaded-curator ............................... SUCCESS [  0.101 s][INFO]
flink-shaded-curator-recipes ....................... SUCCESS [  0.843 s][INFO] flink-shaded-curator-test
.......................... SUCCESS [  0.148 s][INFO] flink-core .........................................
SUCCESS [ 35.891 s][INFO] flink-java ......................................... SUCCESS [ 14.604
s][INFO] flink-runtime ...................................... SUCCESS [03:34 min][INFO] flink-optimizer
.................................... SUCCESS [  6.584 s][INFO] flink-clients ......................................
SUCCESS [ 18.831 s][INFO] flink-test-utils ................................... SUCCESS [ 
4.042 s][INFO] flink-scala ........................................ SUCCESS [ 21.669 s][INFO]
flink-runtime-web .................................. SUCCESS [ 11.744 s][INFO] flink-streaming-java
............................... SUCCESS [ 53.149 s][INFO] flink-examples .....................................
SUCCESS [  0.826 s][INFO] flink-examples-batch ............................... SUCCESS [ 
9.919 s][INFO] flink-contrib ...................................... SUCCESS [  0.118 s][INFO]
flink-statebackend-rocksdb ......................... SUCCESS [  7.223 s][INFO] flink-tests
........................................ FAILURE [05:27 min][INFO] flink-streaming-scala ..............................
SKIPPED[INFO] flink-streaming-connectors ......................... SKIPPED[INFO] flink-connector-flume
.............................. SKIPPED[INFO] flink-libraries ....................................
SKIPPED[INFO] flink-table ........................................ SKIPPED[INFO] flink-connector-kafka-base
......................... SKIPPED[INFO] flink-connector-kafka-0.8 ..........................
SKIPPED[INFO] flink-connector-kafka-0.9 .......................... SKIPPED[INFO] flink-connector-elasticsearch
...................... SKIPPED[INFO] flink-connector-elasticsearch2 .....................
SKIPPED[INFO] flink-connector-rabbitmq ........................... SKIPPED[INFO] flink-connector-twitter
............................ SKIPPED[INFO] flink-connector-nifi ...............................
SKIPPED[INFO] flink-connector-cassandra .......................... SKIPPED[INFO] flink-connector-filesystem
......................... SKIPPED[INFO] flink-batch-connectors .............................
SKIPPED[INFO] flink-avro ......................................... SKIPPED[INFO] flink-jdbc
......................................... SKIPPED[INFO] flink-hadoop-compatibility .........................
SKIPPED[INFO] flink-hbase ........................................ SKIPPED[INFO] flink-hcatalog
..................................... SKIPPED[INFO] flink-examples-streaming ...........................
SKIPPED[INFO] flink-gelly ........................................ SKIPPED[INFO] flink-gelly-scala
.................................. SKIPPED[INFO] flink-gelly-examples ...............................
SKIPPED[INFO] flink-python ....................................... SKIPPED[INFO] flink-ml
........................................... SKIPPED[INFO] flink-cep ..........................................
SKIPPED[INFO] flink-cep-scala .................................... SKIPPED[INFO] flink-scala-shell
.................................. SKIPPED[INFO] flink-quickstart ...................................
SKIPPED[INFO] flink-quickstart-java .............................. SKIPPED[INFO] flink-quickstart-scala
............................. SKIPPED[INFO] flink-storm ........................................
SKIPPED[INFO] flink-storm-examples ............................... SKIPPED[INFO] flink-streaming-contrib
............................ SKIPPED[INFO] flink-tweet-inputformat ............................
SKIPPED[INFO] flink-operator-stats ............................... SKIPPED[INFO] flink-connector-wikiedits
.......................... SKIPPED[INFO] flink-yarn .........................................
SKIPPED[INFO] flink-dist ......................................... SKIPPED[INFO] flink-metrics
...................................... SKIPPED[INFO] flink-metrics-dropwizard ...........................
SKIPPED[INFO] flink-metrics-ganglia .............................. SKIPPED[INFO] flink-metrics-graphite
............................. SKIPPED[INFO] flink-metrics-statsd ...............................
SKIPPED[INFO] flink-fs-tests ..................................... SKIPPED[INFO] flink-java8
........................................ SKIPPED[INFO] ------------------------------------------------------------------------[INFO]
BUILD FAILURE[INFO] ------------------------------------------------------------------------[INFO]
Total time: 12:24 min[INFO] Finished at: 2016-06-27T08:19:18-07:00[INFO] Final Memory: 147M/4711M[INFO]
------------------------------------------------------------------------[ERROR] Failed to
execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (integration-tests)
on project flink-tests_2.10: There are test failures.[ERROR] [ERROR] Please refer to /workspace/git-projects/junk/flink/flink-tests/target/surefire-reports
for the individual test results.[ERROR] -> [Help 1][ERROR] [ERROR] To see the full stack
trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven using the -X switch
to enable full debug logging.[ERROR] [ERROR] For more information about the errors and possible
solutions, please read the following articles:[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command[ERROR]  mvn
<goals> -rf :flink-tests_2.10
>> 2) mvn clean verify -pl flink-yarn-tests -Pinclude-yarn-tests -Dtest=YARNSessionFIFOITCase#testJavaAPI
-DfailIfNoTests=false
>> ------------------------------------------------------- T E S T S-------------------------------------------------------Running
org.apache.flink.yarn.YARNSessionFIFOITCase08:28:32,368 INFO  org.apache.flink.yarn.YarnTestBase
                           - Trying to locate uberjar in ..Tests run: 1, Failures: 1, Errors:
0, Skipped: 0, Time elapsed: 0.366 sec <<< FAILURE! - in org.apache.flink.yarn.YARNSessionFIFOITCaseorg.apache.flink.yarn.YARNSessionFIFOITCase
 Time elapsed: 0.366 sec  <<< FAILURE!java.lang.AssertionError: Flink uberjar not
found at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertNotNull(Assert.java:621) at org.apache.flink.yarn.YarnTestBase.startYARNWithConfig(YarnTestBase.java:343)
at org.apache.flink.yarn.YARNSessionFIFOITCase.setup(YARNSessionFIFOITCase.java:71)
>>
>> Results :
>> Failed tests:  YARNSessionFIFOITCase.setup:71->YarnTestBase.startYARNWithConfig:343
Flink uberjar not found
>> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
>> [INFO] ------------------------------------------------------------------------[INFO]
BUILD FAILURE[INFO] ------------------------------------------------------------------------[INFO]
Total time: 23.960 s[INFO] Finished at: 2016-06-27T08:28:32-07:00[INFO] Final Memory: 135M/3439M[INFO]
------------------------------------------------------------------------[ERROR] Failed to
execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (integration-tests)
on project flink-yarn-tests_2.10: There are test failures.[ERROR] [ERROR] Please refer to
/workspace/git-projects/junk/flink/flink-yarn-tests/target/surefire-reports for the individual
test results.[ERROR] -> [Help 1][ERROR] [ERROR] To see the full stack trace of the errors,
re-run Maven with the -e switch.[ERROR] Re-run Maven using the -X switch to enable full debug
logging.[ERROR] [ERROR] For more information about the errors and possible solutions, please
read the following articles:[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
>>
>>
>>    On Monday, June 27, 2016 7:16 AM, Aljoscha Krettek <aljoscha@apache.org>
wrote:
>>
>>
>>  Hi,the first command you mentioned doesn't work because the Yarn tests are not executed
in the traditional phases and therefore maven will complain. If you do it like this it will
work (I tried it):
>> mvn clean verify -pl flink-yarn-tests -Pinclude-yarn-tests -Dtest=YARNSessionFIFOITCase#testJavaAPI
-DfailIfNoTests=false
>>
>> (notice the added -DfailIfNoTests=false).
>> I also ran a "mvn clean install" afterwards and it worked as well. Could you try
with the extended command from above?
>> Cheers,Aljoscha
>>
>>
>> On Mon, 27 Jun 2016 at 14:57 Vijay Srinivasaraghavan <vijikarthi@yahoo.com>
wrote:
>>
>>
>> I am on Ubuntu 16.x, Java OpenJDK 1.8.0_91.
>>
>> Can you try below commands and see if its working with the latest trunk code.
>> mvn clean verify -pl flink-yarn-tests -Pinclude-yarn-tests -Dtest=YARNSessionFIFOITCase#testJavaAPI
>>
>> mvn clean install
>> RegardsVijay
>>
>>    On Monday, June 27, 2016 5:44 AM, Aljoscha Krettek <aljoscha@apache.org>
wrote:
>>
>>
>>  Hi,I just ran a "mvn clean verify" and it passed on my machine (latest master, OS
X El Capitan, Java 1.8.0_40, Maven 3.3.9). What's your environment?
>> Cheers,Aljoscha
>> On Fri, 24 Jun 2016 at 16:47 Vijay Srinivasaraghavan <vijikarthi@yahoo.com.invalid>
wrote:
>>
>> I am seeing below failure consistently with the latest trunk code when I run "mvn
clean verify". Is anyone seeing similar error in your environment?
>> Failed tests:  LocalFlinkMiniClusterITCase.testLocalFlinkMiniClusterWithMultipleTaskManagers:166
Thread Thread[initialSeedUniquifierGenerator,5,main] was started by the mini cluster, but
not shut down
>> RegardsVijay
>>
>>
>>
>>
>>
>>
>
>

Mime
View raw message