hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Xu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-12239) StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB
Date Wed, 15 Jul 2015 18:07:04 GMT
Duo Xu created HADOOP-12239:
-------------------------------

             Summary: StorageException complaining " no lease ID" when updating FolderLastModifiedTime
in WASB
                 Key: HADOOP-12239
                 URL: https://issues.apache.org/jira/browse/HADOOP-12239
             Project: Hadoop Common
          Issue Type: Bug
          Components: tools
    Affects Versions: 2.7.0
            Reporter: Duo Xu
            Assignee: Duo Xu


This is a similar issue as HADOOP-11523 and HADOOP-12089, which I found in a customer's HBase
cluster logs, but the piece of code is in a different place.
{code}
2015-07-09 13:38:57,388 INFO org.apache.hadoop.hbase.master.SplitLogManager: dead splitlog
workers [workernode3.skypedataprdhbaeus00.b6.internal.cloudapp.net,60020,1436448555180]
2015-07-09 13:38:57,466 ERROR org.apache.hadoop.hbase.executor.EventHandler: Caught throwable
while processing event M_SERVER_SHUTDOWN
java.io.IOException: failed log splitting for workernode12.skypedataprdhbaeus00.b6.internal.cloudapp.net,60020,1436448566374,
will retry
	at org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.resubmit(ServerShutdownHandler.java:343)
	at org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.process(ServerShutdownHandler.java:211)
	at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Unable to write RenamePending file for folder rename from
hbase/WALs/workernode12.skypedataprdhbaeus00.b6.internal.cloudapp.net,60020,1436448566374
to hbase/WALs/workernode12.skypedataprdhbaeus00.b6.internal.cloudapp.net,60020,1436448566374-splitting
	at org.apache.hadoop.fs.azure.NativeAzureFileSystem$FolderRenamePending.writeFile(NativeAzureFileSystem.java:258)
	at org.apache.hadoop.fs.azure.NativeAzureFileSystem.prepareAtomicFolderRename(NativeAzureFileSystem.java:2110)
	at org.apache.hadoop.fs.azure.NativeAzureFileSystem.rename(NativeAzureFileSystem.java:1998)
	at org.apache.hadoop.hbase.master.MasterFileSystem.getLogDirs(MasterFileSystem.java:325)
	at org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:412)
	at org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:390)
	at org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:288)
	at org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.process(ServerShutdownHandler.java:204)
	... 4 more
Caused by: org.apache.hadoop.fs.azure.AzureException: com.microsoft.azure.storage.StorageException:
There is currently a lease on the blob and no lease ID was specified in the request.
	at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.updateFolderLastModifiedTime(AzureNativeFileSystemStore.java:2598)
	at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.updateFolderLastModifiedTime(AzureNativeFileSystemStore.java:2609)
	at org.apache.hadoop.fs.azure.NativeAzureFileSystem.create(NativeAzureFileSystem.java:1366)
	at org.apache.hadoop.fs.azure.NativeAzureFileSystem.create(NativeAzureFileSystem.java:1195)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:908)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:889)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:786)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:775)
	at org.apache.hadoop.fs.azure.NativeAzureFileSystem$FolderRenamePending.writeFile(NativeAzureFileSystem.java:255)
	... 11 more
Caused by: com.microsoft.azure.storage.StorageException: There is currently a lease on the
blob and no lease ID was specified in the request.
	at com.microsoft.azure.storage.StorageException.translateException(StorageException.java:89)
	at com.microsoft.azure.storage.core.StorageRequest.materializeException(StorageRequest.java:307)
	at com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:182)
	at com.microsoft.azure.storage.blob.CloudBlob.uploadProperties(CloudBlob.java:2892)
	at org.apache.hadoop.fs.azure.StorageInterfaceImpl$CloudBlobWrapperImpl.uploadProperties(StorageInterfaceImpl.java:372)
	at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.updateFolderLastModifiedTime(AzureNativeFileSystemStore.java:2593)
	... 19 more
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message