ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (IGNITE-11842) clients fails to connect
Date Tue, 14 May 2019 10:42:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-11842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16838523#comment-16838523
] 

James edited comment on IGNITE-11842 at 5/14/19 10:41 AM:
----------------------------------------------------------

The problem has been tracked down to a problem with a Kubernetes container using docker alpine
image.

The use of "/lib/ld-musl-x86_64.so.1"  by alpine java openjdk 8 is the problem.

Using java openjdk 8 that uses glibc works fine.

BAD:

Alpine stack:
 /usr/lib # ldd /usr/lib/jvm/java-1.8-openjdk/jre/bin/java
 /lib/ld-musl-x86_64.so.1 (0x7fdc4616f000)
 libjli.so => /usr/lib/jvm/java-1.8-openjdk/jre/bin/../lib/amd64/jli/libjli.so (0x7fdc45d5f000)
 libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7fdc4616f000)
 libz.so.1 => /lib/libz.so.1 (0x7fdc45b48000)

 

GOOD:

CentOS stack:
 ldd /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/jre/bin/java
 linux-vdso.so.1 => (0x00007ffdf01cb000)
 libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f433c237000)
 libz.so.1 => /lib64/libz.so.1 (0x00007f433c021000)
 libjli.so => /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/jre/bin/../lib/amd64/jli/libjli.so
(0x00007f433be12000)
 libdl.so.2 => /lib64/libdl.so.2 (0x00007f433bc0e000)
 libc.so.6 => /lib64/libc.so.6 (0x00007f433b841000)
 /lib64/ld-linux-x86-64.so.2 (0x00007f433c453000)

 


was (Author: jcdutton):
The problem has been tracked does to a problem with a kubenetes container using docker alpine
image.

The use of "/lib/ld-musl-x86_64.so.1"  by alpine java openjdk 8 is the problem.

Using java openjdk 8 that uses glibc works fine.

BAD:

Alpine stack:
/usr/lib # ldd /usr/lib/jvm/java-1.8-openjdk/jre/bin/java
/lib/ld-musl-x86_64.so.1 (0x7fdc4616f000)
libjli.so => /usr/lib/jvm/java-1.8-openjdk/jre/bin/../lib/amd64/jli/libjli.so (0x7fdc45d5f000)
libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7fdc4616f000)
libz.so.1 => /lib/libz.so.1 (0x7fdc45b48000)

 

GOOD:

CentOS stack:
ldd /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/jre/bin/java
linux-vdso.so.1 => (0x00007ffdf01cb000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f433c237000)
libz.so.1 => /lib64/libz.so.1 (0x00007f433c021000)
libjli.so => /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/jre/bin/../lib/amd64/jli/libjli.so
(0x00007f433be12000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f433bc0e000)
libc.so.6 => /lib64/libc.so.6 (0x00007f433b841000)
/lib64/ld-linux-x86-64.so.2 (0x00007f433c453000)

 

> clients fails to connect
> ------------------------
>
>                 Key: IGNITE-11842
>                 URL: https://issues.apache.org/jira/browse/IGNITE-11842
>             Project: Ignite
>          Issue Type: Bug
>          Components: cache
>    Affects Versions: 2.7
>         Environment: kubernetes
>  
>            Reporter: James
>            Priority: Major
>
> The main symptom is that clients are failing to connect to the ignite cluster, with reported
timeouts in the logs.
> The main fact we have is this (from within the client within a kubernetes container on
Linux):
> / # netstat -ntp
> Active Internet connections (w/o servers)
> Proto Recv-Q Send-Q Local Address           Foreign Address        
State       PID/Program name
> tcp   215796      0 ::ffff:10.42.2.97:43666 ::ffff:10.42.3.170:47500 ESTABLISHED
13/java
>  
> Namely, the application is failing to read data from the tcp socket. Notice the “Recv-Q”
of 215796.
>  
> This could be an client application, but also the same thing happens with ignitevisor.sh
> Downgrading to Apache Ignite 2.3 resolves the problem.
> Testes so far:
> 2.7  intermittently fails to connect to the ignite cluster.
> 2.3 seems OK.
> 2.6 also fails after a number of clients have connected successfully.
>  
> Has anyone else seen this?
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message