hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14906) ITestAzureConcurrentOutOfBandIo failing with checksum errors on write
Date Mon, 25 Sep 2017 13:50:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16179039#comment-16179039
] 

Steve Loughran commented on HADOOP-14906:
-----------------------------------------

And run with azure log at debug
{code}
2017-09-25 14:47:16,936 INFO  [JUnit-testReadOOBWrites]: impl.MetricsConfig (MetricsConfig.java:loadFirst(115))
- loaded properties from hadoop-metrics2-azure-file-system.properties
2017-09-25 14:47:16,959 INFO  [JUnit-testReadOOBWrites]: impl.MetricsSinkAdapter (MetricsSinkAdapter.java:start(207))
- Sink azuretestcollector started
2017-09-25 14:47:17,448 INFO  [JUnit-testReadOOBWrites]: impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(374))
- Scheduled Metric snapshot period at 10 second(s).
2017-09-25 14:47:17,449 INFO  [JUnit-testReadOOBWrites]: impl.MetricsSystemImpl (MetricsSystemImpl.java:start(191))
- azure-file-system metrics system started
2017-09-25 14:47:17,481 DEBUG [JUnit-testReadOOBWrites]: azure.AzureNativeFileSystemStore
(AzureNativeFileSystemStore.java:configureAzureStorageSession(813)) - AzureNativeFileSystemStore
init. Settings=8,true,90,{3000,3000,30000,30},{true,1.0,1.0}
2017-09-25 14:47:17,484 DEBUG [JUnit-testReadOOBWrites]: azure.AzureNativeFileSystemStore
(AzureNativeFileSystemStore.java:initialize(542)) - Page blob directories:  
2017-09-25 14:47:17,484 DEBUG [JUnit-testReadOOBWrites]: azure.AzureNativeFileSystemStore
(AzureNativeFileSystemStore.java:initialize(550)) - Block blobs with compaction directories:
 
2017-09-25 14:47:17,484 DEBUG [JUnit-testReadOOBWrites]: azure.AzureNativeFileSystemStore
(AzureNativeFileSystemStore.java:initialize(567)) - Atomic rename directories: /hbase 
2017-09-25 14:47:17,501 DEBUG [JUnit-testReadOOBWrites]: azure.SelfThrottlingIntercept (SelfThrottlingIntercept.java:sendingRequest(167))
-  SelfThrottlingIntercept:: SendingRequest:   threadId=11, requestType=read , isFirstRequest=true,
sleepDuration=0
2017-09-25 14:47:17,536 DEBUG [JUnit-testReadOOBWrites]: azure.SelfThrottlingIntercept (SelfThrottlingIntercept.java:responseReceived(115))
- SelfThrottlingIntercept:: ResponseReceived: threadId=11, Status=200, Elapsed(ms)=34, ETAG="0x8D5041BEF5D2DAF",
contentLength=-1, requestMethod=HEAD
2017-09-25 14:47:17,538 DEBUG [JUnit-testReadOOBWrites]: azure.SelfThrottlingIntercept (SelfThrottlingIntercept.java:sendingRequest(167))
-  SelfThrottlingIntercept:: SendingRequest:   threadId=11, requestType=write, isFirstRequest=true,
sleepDuration=0
2017-09-25 14:47:17,569 DEBUG [JUnit-testReadOOBWrites]: azure.SelfThrottlingIntercept (SelfThrottlingIntercept.java:responseReceived(115))
- SelfThrottlingIntercept:: ResponseReceived: threadId=11, Status=200, Elapsed(ms)=30, ETAG="0x8D5041BEF87544D",
contentLength=-1, requestMethod=PUT
2017-09-25 14:47:17,637 DEBUG [pool-1-thread-2]: azure.SelfThrottlingIntercept (SelfThrottlingIntercept.java:sendingRequest(167))
-  SelfThrottlingIntercept:: SendingRequest:   threadId=18, requestType=write, isFirstRequest=true,
sleepDuration=0
2017-09-25 14:47:17,637 DEBUG [pool-1-thread-1]: azure.SelfThrottlingIntercept (SelfThrottlingIntercept.java:sendingRequest(167))
-  SelfThrottlingIntercept:: SendingRequest:   threadId=17, requestType=write, isFirstRequest=true,
sleepDuration=0
2017-09-25 14:47:22,869 DEBUG [pool-1-thread-2]: azure.SelfThrottlingIntercept (SelfThrottlingIntercept.java:responseReceived(115))
- SelfThrottlingIntercept:: ResponseReceived: threadId=18, Status=400, Elapsed(ms)=5229, ETAG=null,
contentLength=405, requestMethod=PUT
2017-09-25 14:47:22,892 INFO  [JUnit-testReadOOBWrites]: azure.AbstractWasbTestBase (AbstractWasbTestBase.java:describe(172))
- 

testReadOOBWrites: closing test account and filesystem


java.io.IOException
	at com.microsoft.azure.storage.core.Utility.initIOException(Utility.java:770)
	at com.microsoft.azure.storage.blob.BlobOutputStreamInternal.writeBlock(BlobOutputStreamInternal.java:443)
	at com.microsoft.azure.storage.blob.BlobOutputStreamInternal.access$000(BlobOutputStreamInternal.java:52)
	at com.microsoft.azure.storage.blob.BlobOutputStreamInternal$1.call(BlobOutputStreamInternal.java:387)
	at com.microsoft.azure.storage.blob.BlobOutputStreamInternal$1.call(BlobOutputStreamInternal.java:384)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: com.microsoft.azure.storage.StorageException: The MD5 value specified in the request
did not match with the MD5 value calculated by the server.
	at com.microsoft.azure.storage.StorageException.translateException(StorageException.java:89)
	at com.microsoft.azure.storage.core.StorageRequest.materializeException(StorageRequest.java:315)
	at com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:175)
	at com.microsoft.azure.storage.blob.CloudBlockBlob.uploadBlockInternal(CloudBlockBlob.java:1078)
	at com.microsoft.azure.storage.blob.CloudBlockBlob.uploadBlock(CloudBlockBlob.java:1050)
	at com.microsoft.azure.storage.blob.BlobOutputStreamInternal.writeBlock(BlobOutputStreamInternal.java:437)
	... 9 more
{code}

> ITestAzureConcurrentOutOfBandIo failing with checksum errors on write
> ---------------------------------------------------------------------
>
>                 Key: HADOOP-14906
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14906
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/azure
>    Affects Versions: 2.9.0, 3.1.0
>         Environment: UK BT ASDL connection, 1.8.0_121-b13, azure storage ireland
>            Reporter: Steve Loughran
>
> {{ITestAzureConcurrentOutOfBandIo}} is consistently raising an IOE with the text "The
MD5 value specified in the request did not match with the MD5 value calculated by the server"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message