hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jay Hankinson (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-13518) hdfsHSync()/hdfsFileClose() doesn't release lease with libhdfs on Azure ADL
Date Tue, 01 May 2018 17:41:01 GMT
Jay Hankinson created HDFS-13518:
------------------------------------

             Summary: hdfsHSync()/hdfsFileClose() doesn't release lease with libhdfs on Azure
ADL
                 Key: HDFS-13518
                 URL: https://issues.apache.org/jira/browse/HDFS-13518
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: libhdfs
    Affects Versions: 2.7.3
         Environment: HDInsight in Azure:

actian@hn0-vector:~$ hadoop version

Hadoop 2.7.3.2.6.2.25-1

Subversion git@github.com:hortonworks/hadoop.git -r 1ceeb58bb3bb5904df0cbb7983389bcaf2ffd0b6

Compiled by jenkins on 2017-11-29T15:28Z

Compiled with protoc 2.5.0

>From source with checksum 90b73c4c185645c1f47b61f942230

This command was run using /usr/hdp/2.6.2.25-1/hadoop/hadoop-common-2.7.3.2.6.2.25-1.jar

actian@hn0-vector:~$ cat /etc/lsb-release 

DISTRIB_ID=Ubuntu

DISTRIB_RELEASE=16.04

DISTRIB_CODENAME=xenial

DISTRIB_DESCRIPTION="Ubuntu 16.04.4 LTS"
            Reporter: Jay Hankinson


If hdfsFileClose() is called on a file opened for write/append directly after hdfsHSync(),
without further data being written to the file, the file doesn't close and the lease isn't
released. In the ADL
output trace, with hdfsHSync() immediately before hdfsFileClose() I see:
 
18/04/26 23:31:11 DEBUG store.HttpTransport: HTTPRequest,Succeeded,cReqId:607c6051-2ef3-4e57-9b20-caea70f22a31.0,lat:43,err:,Reqlen:14,Resplen:0,token_ns:3700,sReqId:40b2a8df-c9b5-4eff-b84e-c1346f4fdb88,path:/tmp/testfile.txt,qp:op=APPEND&append=true&syncFlag=METADATA&filesessionid=76b19d5b-f3ad-4400-a742-2f7996450859&leaseid=76b19d5b-f3ad-4400-a742-2f7996450859&offset=28&api-version=2016-11-01
 
but nothing else. I also get an error if I try to re-open the file:
 
18/04/26 23:37:28 DEBUG store.HttpTransport: HTTPRequest,Failed,cReqId:f216b9d6-3e35-4166-92cf-a7c4e2389dcd.0,lat:9,err:HTTP400(IllegalArgumentException),Reqlen:0,Resplen:357,token_ns:13001,sReqId:4b9e1e11-3f61-46a6-b461-3e5b180266c5,path:/tmp/testfile.txt,qp:op=APPEND&append=true&syncFlag=DATA&filesessionid=3197c37c-356d-491a-a902-aa19b6d8a374&leaseid=3197c37c-356d-491a-a902-aa19b6d8a374&api-version=2016-11-01
hdfsOpenFile(/tmp/testfile.txt): FileSystem#append((Lorg/apache/hadoop/fs/Path;)Lorg/apache/hadoop/fs/FSDataOutputStream;)
error:
java.io.IOException: APPEND failed with error 0x83090a16 (Failed to perform the requested
operation because the file is currently open in write mode by another user or process.). [4b9e1e11-3f61-46a6-b461-3e5b180266c5][2018-04-26T16:37:28.6549201-07:00]
at com.microsoft.azure.datalake.store.ADLStoreClient.getRemoteException(ADLStoreClient.java:1142)
at com.microsoft.azure.datalake.store.ADLStoreClient.getExceptionFromResponse(ADLStoreClient.java:1106)
at com.microsoft.azure.datalake.store.ADLStoreClient.getAppendStream(ADLStoreClient.java:294)
at org.apache.hadoop.fs.adl.AdlFileSystem.append(AdlFileSystem.java:398)
at org.apache.hadoop.fs.FileSystem.append(FileSystem.java:1187)
 
If hdfsWrite()/flush() is called after hdfsHSync() and before hdfsFileClose() I see the APPEND()
with syncFlag=CLOSE as expected.
 
18/04/26 23:37:53 DEBUG store.HttpTransport: HTTPRequest,Succeeded,cReqId:ecd19d36-b535-4dd5-8c49-1588a2a60947.0,lat:124,err:,Reqlen:14,Resplen:0,token_ns:4700,sReqId:4a6a2c94-ab31-47b2-aae8-1e4250974d27,path:/tmp/testfile.txt,qp:op=APPEND&append=true&syncFlag=METADATA&filesessionid=21adf26e-3208-40fc-803a-1c18e1914090&leaseid=21adf26e-3208-40fc-803a-1c18e1914090&offset=0&api-version=2016-11-01
18/04/26 23:37:53 DEBUG store.HttpTransport: HTTPRequest,Succeeded,cReqId:08708059-6396-4af9-a4da-d7d60c44041b.0,lat:43,err:,Reqlen:14,Resplen:0,token_ns:3900,sReqId:a4b5f419-f87f-4bed-99ed-8ad1d24c9ab7,path:/tmp/testfile.txt,qp:op=APPEND&append=true&syncFlag=CLOSE&filesessionid=21adf26e-3208-40fc-803a-1c18e1914090&leaseid=21adf26e-3208-40fc-803a-1c18e1914090&offset=14&api-version=2016-11-01
 
Appending to the file is then prevented for the next 8 or so minutes until the lease expires.
libhdfs repro case attached.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org


Mime
View raw message