ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Roman Guseinov (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (IGNITE-9826) Ignite node with TcpDiscoveryS3IpFinder can hang while stopping
Date Tue, 09 Oct 2018 12:52:00 GMT

     [ https://issues.apache.org/jira/browse/IGNITE-9826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Roman Guseinov updated IGNITE-9826:
-----------------------------------
    Description: 
Here is a stack trace:


{code:java}
[15:46:11,081][SEVERE][tcp-disco-ip-finder-cleaner-#4%node1%][TcpDiscoverySpi] Failed to clean
IP finder up.
class org.apache.ignite.spi.IgniteSpiException: Failed to list objects in the bucket: rguseinov-ipfinder
	at org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder.getRegisteredAddresses(TcpDiscoveryS3IpFinder.java:200)
	at org.apache.ignite.spi.discovery.tcp.ipfinder.s3.S3IpFinderReproducer$CustomS3IpFinder.getRegisteredAddresses(S3IpFinderReproducer.java:103)
	at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.registeredAddresses(TcpDiscoverySpi.java:1900)
	at org.apache.ignite.spi.discovery.tcp.ServerImpl$IpFinderCleaner.cleanIpFinder(ServerImpl.java:1998)
	at org.apache.ignite.spi.discovery.tcp.ServerImpl$IpFinderCleaner.body(ServerImpl.java:1973)
	at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
Caused by: com.amazonaws.SdkClientException: Failed to sanitize XML document destined for
handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$ListBucketHandler
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.sanitizeXmlDocument(XmlResponsesSaxParser.java:214)
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseListBucketObjectsResponse(XmlResponsesSaxParser.java:298)
	at com.amazonaws.services.s3.model.transform.Unmarshallers$ListObjectsUnmarshaller.unmarshall(Unmarshallers.java:70)
	at com.amazonaws.services.s3.model.transform.Unmarshallers$ListObjectsUnmarshaller.unmarshall(Unmarshallers.java:59)
	at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:62)
	at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:31)
	at com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:70)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1501)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1222)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1035)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:747)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:721)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:704)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:672)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:654)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:518)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4137)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4079)
	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:819)
	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:791)
	at org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder.getRegisteredAddresses(TcpDiscoveryS3IpFinder.java:148)
	... 5 more
Caused by: com.amazonaws.AbortedException: 
	at com.amazonaws.internal.SdkFilterInputStream.abortIfNeeded(SdkFilterInputStream.java:51)
	at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:71)
	at com.amazonaws.event.ProgressInputStream.read(ProgressInputStream.java:181)
	at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
	at java.io.InputStreamReader.read(InputStreamReader.java:184)
	at java.io.BufferedReader.read1(BufferedReader.java:210)
	at java.io.BufferedReader.read(BufferedReader.java:286)
	at java.io.Reader.read(Reader.java:140)
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.sanitizeXmlDocument(XmlResponsesSaxParser.java:186)
	... 25 more
{code}


Reproducer is attached [^S3IpFinderReproducer.java] . 

  was:
Here is stack trace:


{code:java}
[15:46:11,081][SEVERE][tcp-disco-ip-finder-cleaner-#4%node1%][TcpDiscoverySpi] Failed to clean
IP finder up.
class org.apache.ignite.spi.IgniteSpiException: Failed to list objects in the bucket: rguseinov-ipfinder
	at org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder.getRegisteredAddresses(TcpDiscoveryS3IpFinder.java:200)
	at org.apache.ignite.spi.discovery.tcp.ipfinder.s3.S3IpFinderReproducer$CustomS3IpFinder.getRegisteredAddresses(S3IpFinderReproducer.java:103)
	at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.registeredAddresses(TcpDiscoverySpi.java:1900)
	at org.apache.ignite.spi.discovery.tcp.ServerImpl$IpFinderCleaner.cleanIpFinder(ServerImpl.java:1998)
	at org.apache.ignite.spi.discovery.tcp.ServerImpl$IpFinderCleaner.body(ServerImpl.java:1973)
	at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
Caused by: com.amazonaws.SdkClientException: Failed to sanitize XML document destined for
handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$ListBucketHandler
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.sanitizeXmlDocument(XmlResponsesSaxParser.java:214)
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseListBucketObjectsResponse(XmlResponsesSaxParser.java:298)
	at com.amazonaws.services.s3.model.transform.Unmarshallers$ListObjectsUnmarshaller.unmarshall(Unmarshallers.java:70)
	at com.amazonaws.services.s3.model.transform.Unmarshallers$ListObjectsUnmarshaller.unmarshall(Unmarshallers.java:59)
	at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:62)
	at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:31)
	at com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:70)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1501)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1222)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1035)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:747)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:721)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:704)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:672)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:654)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:518)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4137)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4079)
	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:819)
	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:791)
	at org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder.getRegisteredAddresses(TcpDiscoveryS3IpFinder.java:148)
	... 5 more
Caused by: com.amazonaws.AbortedException: 
	at com.amazonaws.internal.SdkFilterInputStream.abortIfNeeded(SdkFilterInputStream.java:51)
	at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:71)
	at com.amazonaws.event.ProgressInputStream.read(ProgressInputStream.java:181)
	at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
	at java.io.InputStreamReader.read(InputStreamReader.java:184)
	at java.io.BufferedReader.read1(BufferedReader.java:210)
	at java.io.BufferedReader.read(BufferedReader.java:286)
	at java.io.Reader.read(Reader.java:140)
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.sanitizeXmlDocument(XmlResponsesSaxParser.java:186)
	... 25 more
{code}


Reproducer is attached [^S3IpFinderReproducer.java] . 


> Ignite node with TcpDiscoveryS3IpFinder can hang while stopping
> ---------------------------------------------------------------
>
>                 Key: IGNITE-9826
>                 URL: https://issues.apache.org/jira/browse/IGNITE-9826
>             Project: Ignite
>          Issue Type: Bug
>          Components: aws
>    Affects Versions: 2.6
>            Reporter: Roman Guseinov
>            Assignee: Roman Guseinov
>            Priority: Major
>         Attachments: S3IpFinderReproducer.java
>
>
> Here is a stack trace:
> {code:java}
> [15:46:11,081][SEVERE][tcp-disco-ip-finder-cleaner-#4%node1%][TcpDiscoverySpi] Failed
to clean IP finder up.
> class org.apache.ignite.spi.IgniteSpiException: Failed to list objects in the bucket:
rguseinov-ipfinder
> 	at org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder.getRegisteredAddresses(TcpDiscoveryS3IpFinder.java:200)
> 	at org.apache.ignite.spi.discovery.tcp.ipfinder.s3.S3IpFinderReproducer$CustomS3IpFinder.getRegisteredAddresses(S3IpFinderReproducer.java:103)
> 	at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.registeredAddresses(TcpDiscoverySpi.java:1900)
> 	at org.apache.ignite.spi.discovery.tcp.ServerImpl$IpFinderCleaner.cleanIpFinder(ServerImpl.java:1998)
> 	at org.apache.ignite.spi.discovery.tcp.ServerImpl$IpFinderCleaner.body(ServerImpl.java:1973)
> 	at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
> Caused by: com.amazonaws.SdkClientException: Failed to sanitize XML document destined
for handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$ListBucketHandler
> 	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.sanitizeXmlDocument(XmlResponsesSaxParser.java:214)
> 	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseListBucketObjectsResponse(XmlResponsesSaxParser.java:298)
> 	at com.amazonaws.services.s3.model.transform.Unmarshallers$ListObjectsUnmarshaller.unmarshall(Unmarshallers.java:70)
> 	at com.amazonaws.services.s3.model.transform.Unmarshallers$ListObjectsUnmarshaller.unmarshall(Unmarshallers.java:59)
> 	at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:62)
> 	at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:31)
> 	at com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:70)
> 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1501)
> 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1222)
> 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1035)
> 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:747)
> 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:721)
> 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:704)
> 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:672)
> 	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:654)
> 	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:518)
> 	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4137)
> 	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4079)
> 	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:819)
> 	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:791)
> 	at org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder.getRegisteredAddresses(TcpDiscoveryS3IpFinder.java:148)
> 	... 5 more
> Caused by: com.amazonaws.AbortedException: 
> 	at com.amazonaws.internal.SdkFilterInputStream.abortIfNeeded(SdkFilterInputStream.java:51)
> 	at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:71)
> 	at com.amazonaws.event.ProgressInputStream.read(ProgressInputStream.java:181)
> 	at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
> 	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
> 	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
> 	at java.io.InputStreamReader.read(InputStreamReader.java:184)
> 	at java.io.BufferedReader.read1(BufferedReader.java:210)
> 	at java.io.BufferedReader.read(BufferedReader.java:286)
> 	at java.io.Reader.read(Reader.java:140)
> 	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.sanitizeXmlDocument(XmlResponsesSaxParser.java:186)
> 	... 25 more
> {code}
> Reproducer is attached [^S3IpFinderReproducer.java] . 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message