hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mukul Kumar Singh (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-12337) Ozone: Concurrent RocksDB open calls fail because of "No locks available"
Date Tue, 22 Aug 2017 12:50:00 GMT
Mukul Kumar Singh created HDFS-12337:
----------------------------------------

             Summary: Ozone: Concurrent RocksDB open calls fail because of "No locks available"
                 Key: HDFS-12337
                 URL: https://issues.apache.org/jira/browse/HDFS-12337
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: ozone
    Affects Versions: HDFS-7240
            Reporter: Mukul Kumar Singh
            Assignee: Mukul Kumar Singh
             Fix For: HDFS-7240


HDFS-12216 fixes the issue with the static container cache and re-using the same container
port on datanode restart. However TestKeys still fails after HDFS-12216 is fixed.

The test is now failing because Concurrent RocksDB open calls fail. In the current code BlockDeleting
service and Dispatcher tries to open the db concurrently.

This jira will also fix the keepPort property for Ratis Container port and set the reuse address
property for XceiverServerRatis correctly.

{code}
2017-08-22 16:51:34,453 [BlockDeletingService#1] INFO  utils.RocksDBStore (RocksDBStore.java:<init>(64))
- opening db file /Users/msingh/code/work/apache/cblock/ozone_review/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/dn0_data0/containers/621a3b15-b9fc-4d49-a6e6-29d4c40cc91f/metadata/container.db
2017-08-22 16:51:34,460 [nioEventLoopGroup-9-1] INFO  logging.LoggingHandler (Slf4JLogger.java:info(101))
- [id: 0x8822cd3d, /0.0.0.0:57044] RECEIVED: [id: 0x61367e6f, /127.0.0.1:57173 => /127.0.0
.1:57044]
2017-08-22 16:51:34,461 [nioEventLoopGroup-10-1] INFO  utils.RocksDBStore (RocksDBStore.java:<init>(64))
- opening db file /Users/msingh/code/work/apache/cblock/ozone_review/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/dn0_data0/containers/621a3b15-b9fc-4d49-a6e6-29d4c40cc91f/metadata/container.db
2017-08-22 16:51:34,465 [nioEventLoopGroup-10-1] INFO  utils.RocksDBStore (RocksDBStore.java:<init>(67))
- Failed init RocksDB, db path : /Users/msingh/code/work/apache/cblock/ozone_review/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/dn0_data0/containers/621a3b15-b9fc-4d49-a6e6-29d4c40cc91f/metadata/container.dbexception
org.rocksdb.RocksDBException: lock /Users/msingh/code/work/apache/cblock/ozone_review/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/dn0_data0/containers/621a3b15-b9fc-4d49-a6e6-29d4c40cc91f/metadata/container.db/LOCK:
No locks available
2017-08-22 16:51:34,465 [BlockDeletingService#1] INFO  background.BlockDeletingService (BlockDeletingService.java:call(203))
     - The elapsed time of task@70a576ee for deleting blocks: 12ms.
2017-08-22 16:51:34,474 [nioEventLoopGroup-10-1] INFO  impl.Dispatcher (ContainerUtils.java:logAndReturnError(129))
     - Operation: GetKey : Trace ID: 73f19131-f63b-459a-8f09-9a3db893a296 : Message: 621a3b15-b9fc-4d49-a6e6-29d4c40cc91f
: Result: UNABLE_TO_READ_METADATA_DB
2017-08-22 16:51:34,475 [Thread-382] INFO  exceptions.OzoneExceptionMapper (OzoneExceptionMapper.java:toResponse(39))
ozone  c2a23759-c76f-49ea-b574-f0802a4e5b75/c0df3a48-f75b-4b5e-b1bd-c189ce698056/13b3d486-3d7a-49e4-bc9d-1ef63e674548
hdfs 73f19131-f63b-459a-8f09-9a3db893a296 - Returning exception. ex: {"httpCode":500,"shortMessage":"internalServerError","resource":"hdfs","message":"621a3b15-b9fc-4d49-a6e6-29d4c40cc91f","requestID":"73f19131-f63b-459a-8f09-9a3db893a296","hostName":"hw13605.local"}
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org


Mime
View raw message