polygene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Merlin <paulmer...@apache.org>
Subject Re: Docker based integration testing
Date Mon, 27 Feb 2017 18:17:26 GMT
We currently have several failures related to docker testing on CI:

https://builds.apache.org/view/P/view/Polygene/job/Polygene(JavaEdition)-develop-java8-check/272/testReport/

All seems related to a docker/system/networking issue.

Locally, on a mac, only the Indexing-SQL tests fail as expected since it
is broken.

I'll add a skipDocker property to the build and use it on CI to silent
those failures on our Java 8 & 9 checks jobs. I'll leave the docker
tests enabled on our checkDistributions job that doesn't run on each git
push.




Niclas Hedhman a écrit :
> Yeah, I did
>   ./gradlew clean build assemble install
>
> and got the Riak test failures locally
>
> On Sun, Feb 26, 2017 at 5:15 PM, Paul Merlin <paul@nosphere.org> wrote:
>
>> Le 26 février 2017 05:37:00 GMT+01:00, Niclas Hedhman <niclas@hedhman.org>
>> a écrit :
>>> So only the Docker daemon is needed?
>>>
>>> And I run what target for ./gradlew ?
>>>
>>> Cheers
>>> Niclas
>>>
>>> On Sat, Feb 25, 2017 at 10:33 PM, Paul Merlin <paulmerlin@apache.org> wrote:
>>>
>>>  Gang,
>>>>  I finally found time to take a stab at fixing Docker based integration
>>>>  testing on Linux.
>>>>
>>>>  It should now work out of the box on Linux, given that a Docker service
>>>>  is running locally and that your user has the correct permissions.
>>>>
>>>>  Docker based integration testing is now disabled if unable to contact a
>>>>  Docker service. No need to set DOCKER_HOST.
>>>>
>>>>  Niclas, could you confirm it works for you?
>>>>
>>>>  Cheers
>>>>
>>>>  /Paul
>>>>
>>>>
>>>>
>>>>  Paul Merlin a écrit :
>>>>
>>>>>  Thanks for your feedback Niclas!
>>>>>
>>>>>  Using the unix:///path/to/docker.sock URL on Linux should do.
>>>>>  I'll do another round on that later this week.
>>>>>
>>>>>  Cheers
>>>>>
>>>>>
>>>>>  Niclas Hedhman a écrit :
>>>>>
>>>>>>  DOCKER_* may not be the best way to determine if Docker is present.
I
>>>>>>
>>>>>  am on
>>>>>  a Debian system, and no env variable named DOCKER* is set by default,
>>>>>  yet
>>>>>  docker is operational. IIUIC, these variables are needed for Docker
>>>>>  Swarm
>>>>>  and other specialized setup where one is not running the container on
>>>>>  the
>>>>>  same system as issuing the `docker` command.
>>>>>>  That said, running
>>>>>>  ./gradlew :ext:entity-redis:test
>>>>>>
>>>>>>  without changing my defaults gives me some errors;
>>>>>>
>>>>>>  :extensions:entitystore-redis:test
>>>>>>
>>>>>>  org.apache.polygene.entitystore.redis.RedisMapEntityStoreWithCacheTest
>>>>>>
>>>>>  classMethod FAILED
>>>>>>      java.lang.ExceptionInInitializerError
>>>>>>          Caused by: java.lang.IllegalStateException at
>>>>>>  RedisMapEntityStoreWithCacheTest.java:38
>>>>>>              Caused by: com.spotify.docker.client.
>>>>>>
>>>>>  exceptions.DockerException
>>>>>  at RedisMapEntityStoreWithCacheTest.java:38
>>>>>>                  Caused by: java.util.concurrent.ExecutionException
at
>>>>>>  RedisMapEntityStoreWithCacheTest.java:38
>>>>>>                      Caused by: javax.ws.rs.ProcessingException
>>>>>>                          Caused by:
>>>>>>  org.apache.http.conn.HttpHostConnectException
>>>>>>                              Caused by: java.net.ConnectException
>>>>>>
>>>>>>  org.apache.polygene.entitystore.redis.RedisMapEntityStoreTest >
>>>>>>
>>>>>  classMethod
>>>>>  FAILED
>>>>>>      java.lang.ExceptionInInitializerError
>>>>>>          Caused by: java.lang.IllegalStateException at
>>>>>>  RedisMapEntityStoreTest.java:38
>>>>>>              Caused by: com.spotify.docker.client.
>>>>>>
>>>>>  exceptions.DockerException
>>>>>  at RedisMapEntityStoreTest.java:38
>>>>>>                  Caused by: java.util.concurrent.ExecutionException
at
>>>>>>  RedisMapEntityStoreTest.java:38
>>>>>>                      Caused by: javax.ws.rs.ProcessingException
>>>>>>                          Caused by:
>>>>>>  org.apache.http.conn.HttpHostConnectException
>>>>>>                              Caused by: java.net.ConnectException
>>>>>>
>>>>>>  2 tests completed, 2 failed
>>>>>>
>>>>>>
>>>>>>  And the underlying exception is;
>>>>>>
>>>>>>  Caused by: org.apache.http.conn.HttpHostConnectException: Connect
to
>>>>>>  localhost:2375 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1]
failed:
>>>>>>  Connection refused (Connection refused)
>>>>>>
>>>>>>
>>>>>>  Which I presume is due to the gradle-docker-plugin going on the
network
>>>>>>  protocol instead of the command-line interface. The default docker
>>>>>>  interface on Linux seems to be Unix sockets, which most Java apps
tend
>>>>>>
>>>>>  to
>>>>>  ignore and not support.
>>>>>>   1059 ?        Ssl    0:02 docker-containerd -l
>>>>>>  unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim
>>>>>>  docker-containerd-shim --metrics-interval=0 --start-timeout 2m
>>>>>>
>>>>>  --state-dir
>>>>>  /var/run/docker/libcontainerd/containerd --runtime docker-runc
>>>>>>  Hope that helps. I am sending you login separately to a clean/unused
>>>>>>
>>>>>  server
>>>>>  in Germany, in case you want to run experiments on it.
>>>>>>  Cheers
>>>>>>
>>>>>>  On Sun, Jan 15, 2017 at 8:08 PM, Paul Merlin <paulmerlin@apache.org>
>>>>>>
>>>>>  wrote:
>>>>>>  Could anybody confirm it works for them?
>>>>>>>  Paul Merlin a écrit :
>>>>>>>
>>>>>>>>  Gang,
>>>>>>>>
>>>>>>>>  I pushed changes to the build system and tests to automate
integration
>>>>>>>>  testing.
>>>>>>>>
>>>>>>>>  It uses docker to build the necessary images and run docker
containers
>>>>>>>>  as a JUnit rule.
>>>>>>>>  All this is done only if the DOCKER_HOST environment variable
is set,
>>>>>>>>  otherwise it's skipped.
>>>>>>>>
>>>>>>>>  On linux with docker installed it should work out of the
box given you
>>>>>>>>  have DOCKER_HOST and DOCKER_CERT_PATH set.
>>>>>>>>
>>>>>>>>  On mac you need to:
>>>>>>>>  - docker-machine create someFancyNameYouChoose
>>>>>>>>  - eval $(docker-machine env someFancyNameYouChoose)
>>>>>>>>
>>>>>>>>  On windows, see https://docs.docker.com/machine/reference/env/
>>>>>>>>
>>>>>>>>  With that done, when you now run e.g:
>>>>>>>>
>>>>>>>>    ./gradlew :ext:entity-redis:test
>>>>>>>>
>>>>>>>>  it will build the docker images and run Redis in a container
around
>>>>>>>>  the EntityStore tests.
>>>>>>>>
>>>>>>>>  The same goes for these extensions:
>>>>>>>>
>>>>>>>>  - Memcached Cache
>>>>>>>>  - SQL EntityStore using both MySQL and PostgreSQL
>>>>>>>>  - SQL Index/Query using PostgreSQL
>>>>>>>>  - Riak EntityStore
>>>>>>>>
>>>>>>>>  All the others extensions are tested using embedded solutions,
>>>>>>>>  including MongoDB.
>>>>>>>>
>>>>>>>>  Docker images are built by the :internals:testsupport-internal
>>>>>>>>  project, you'll find the Dockerfiles there.
>>>>>>>>
>>>>>>>>  Thanks to this I was able to fix a bug I introduced in the
Riak ES in
>>>>>>>>  recent changes.
>>>>>>>>  The only thing actually broken is SQL Index/Query, see
>>>>>>>>  https://issues.apache.org/jira/browse/POLYGENE-222
>>>>>>>>
>>>>>>>>  For now these test do not run on CI.
>>>>>>>>  builds.apache.org supports docker so we should be able to
do that too
>>>>>>>>  at some point.
>>>>>>>>  In the meantime some of us should run the *whole* test suite
from time
>>>>>>>>  to time.
>>>>>>>>  At least now it should be damn easy to do!
>>>>>>>>
>>>>>>>>  Before calling this automation done I'd like to get someone
to confirm
>>>>>>>>  it 'works for them'.
>>>>>>>>  I only tested on a mac.
>>>>>>>>
>>>>>>>>  Cheers
>>>>>>>>
>>>>>>>>  /Paul
>> ./gradlew test
>> for the whole test suite
>>
>> --
>> Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma
>> brièveté.
>>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message