hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Georgi Chalakov (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-14906) ITestAzureConcurrentOutOfBandIo failing with checksum errors on write
Date Mon, 25 Sep 2017 18:29:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16179511#comment-16179511
] 

Georgi Chalakov edited comment on HADOOP-14906 at 9/25/17 6:28 PM:
-------------------------------------------------------------------

I am not sure that this is related to the block compaction change. The debug message shows
no directories in the list for block blobs with compaction. I posted the code where we check
whether the file is in one of those directories and if it is not we skip BlockBlobAppendStream.

{quote}
2017-09-25 14:47:17,484 DEBUG [JUnit-testReadOOBWrites]: azure.AzureNativeFileSystemStore
(AzureNativeFileSystemStore.java:initialize(550)) - Block blobs with compaction directories:
 
{quote}

{code:title=hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java|borderStyle=solid}
      if (isBlockBlobWithCompactionKey(key)) {
        BlockBlobAppendStream blockBlobOutputStream = new BlockBlobAppendStream(
            (CloudBlockBlobWrapper) blob,
            keyEncoded,
            this.uploadBlockSizeBytes,
            true,
            getInstrumentedContext());
        outputStream = blockBlobOutputStream;
      } else {
        outputStream = openOutputStream(blob);
      }
{code}


was (Author: georgi):
I am not sure that this is related to the block compaction change. The debug message shows
no directories in the list for block blobs with compaction. I posted the code where we check
whether the file is in one of those directories and if it is not we skip BlockBlobAppendStream.

{noformat}
2017-09-25 14:47:17,484 DEBUG [JUnit-testReadOOBWrites]: azure.AzureNativeFileSystemStore
(AzureNativeFileSystemStore.java:initialize(550)) - Block blobs with compaction directories:
 
{noformat}

{code:title=hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java|borderStyle=solid}
      if (isBlockBlobWithCompactionKey(key)) {
        BlockBlobAppendStream blockBlobOutputStream = new BlockBlobAppendStream(
            (CloudBlockBlobWrapper) blob,
            keyEncoded,
            this.uploadBlockSizeBytes,
            true,
            getInstrumentedContext());
        outputStream = blockBlobOutputStream;
      } else {
        outputStream = openOutputStream(blob);
      }
{code}

> ITestAzureConcurrentOutOfBandIo failing with checksum errors on write
> ---------------------------------------------------------------------
>
>                 Key: HADOOP-14906
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14906
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/azure
>    Affects Versions: 2.9.0, 3.1.0
>         Environment: UK BT ASDL connection, 1.8.0_121-b13, azure storage ireland
>            Reporter: Steve Loughran
>
> {{ITestAzureConcurrentOutOfBandIo}} is consistently raising an IOE with the text "The
MD5 value specified in the request did not match with the MD5 value calculated by the server"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message