hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dushyanth (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-12635) Adding Append API support for WASB
Date Mon, 11 Jan 2016 00:47:39 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-12635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Dushyanth updated HADOOP-12635:
    Attachment: HADOOP-12635.002.patch

[~cnauroth] Thank you very much for the code review. 

I have attached a new patch based on the review comments.

1)  You are right, there is a race condition. I have modified the updateBlobAppendMetadata
to be a test and set operation. In this way, the creation of Append stream would become atomic.
So now if two clients try to open the Append stream, one of them would be successful, the
other would fail as the metadata would be updated holding a lease on the blob.
2) Created an initialize() method for creation of thread pool and append lease renewer thread
other public API has a check to see if the append stream is initialized.
3) This was a mistake, I had to set the lease renewel period to 10 secs and the timeout to
30 secs. Keeping it to 1 sec for Azure storage I think would create bottleneck issues for
the blob, so I have kept the lease renewel to 10 secs, let me know if you feel otherwise.
4) Fix to 4 should address it.
6) Fixed it.

> Adding Append API support for WASB
> ----------------------------------
>                 Key: HADOOP-12635
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12635
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: azure
>    Affects Versions: 2.7.1
>            Reporter: Dushyanth
>            Assignee: Dushyanth
>         Attachments: Append API.docx, HADOOP-12635.001.patch, HADOOP-12635.002.patch
> Currently the WASB implementation of the HDFS interface does not support Append API.
This JIRA is added to design and implement the Append API support to WASB. The intended support
for Append would only support a single writer.  

This message was sent by Atlassian JIRA

View raw message