hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaoyu Yao (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-7066) LazyWriter#evictBlocks misses a null check for replicaState
Date Mon, 15 Sep 2014 22:03:34 GMT

     [ https://issues.apache.org/jira/browse/HDFS-7066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Xiaoyu Yao updated HDFS-7066:
-----------------------------
    Description: 
LazyWriter#evictBlocks (added for HDFS-6581) misses a null check for replicaState. As a result,
there are many NPEs in the debug log under certain conditions. 

2014-09-15 14:27:10,820 DEBUG impl.FsDatasetImpl (FsDatasetImpl.java:evictBlocks(2335)) -
Evicting block null
2014-09-15 14:27:10,821 WARN  impl.FsDatasetImpl (FsDatasetImpl.java:run(2409)) - Ignoring
exception in LazyWriter:
java.lang.NullPointerException
	at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl$LazyWriter.evictBlocks(FsDatasetImpl.java:2343)
	at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl$LazyWriter.run(FsDatasetImpl.java:2396)
	at java.lang.Thread.run(Thread.java:745)

The proposed fix is to break if there is no candidate available to evict.

{code}

	while (iterations++ < MAX_BLOCK_EVICTIONS_PER_ITERATION &&
            transientFreeSpaceBelowThreshold()) {
       LazyWriteReplicaTracker.ReplicaState replicaState =
           lazyWriteReplicaTracker.getNextCandidateForEviction();

       ++++++++++++++++++++
	if (replicaState == null) {
          break;
       }
       ++++++++++++++++++++

       if (LOG.isDebugEnabled()) {
         LOG.debug("Evicting block " + replicaState);
       }

{code}

  was:
LazyWriter#evictBlocks (added for HDFS-6581) misses a null check for replicaState. As a result,
there are many NPEs in the debug log. 

2014-09-15 14:27:10,820 DEBUG impl.FsDatasetImpl (FsDatasetImpl.java:evictBlocks(2335)) -
Evicting block null
2014-09-15 14:27:10,821 WARN  impl.FsDatasetImpl (FsDatasetImpl.java:run(2409)) - Ignoring
exception in LazyWriter:
java.lang.NullPointerException
	at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl$LazyWriter.evictBlocks(FsDatasetImpl.java:2343)
	at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl$LazyWriter.run(FsDatasetImpl.java:2396)
	at java.lang.Thread.run(Thread.java:745)

The proposed fix is to break if there is no candidate available to evict.

{code}

	while (iterations++ < MAX_BLOCK_EVICTIONS_PER_ITERATION &&
            transientFreeSpaceBelowThreshold()) {
       LazyWriteReplicaTracker.ReplicaState replicaState =
           lazyWriteReplicaTracker.getNextCandidateForEviction();

       ++++++++++++++++++++
	if (replicaState == null) {
          break;
       }
       ++++++++++++++++++++

       if (LOG.isDebugEnabled()) {
         LOG.debug("Evicting block " + replicaState);
       }

{code}


> LazyWriter#evictBlocks misses a null check for replicaState
> -----------------------------------------------------------
>
>                 Key: HDFS-7066
>                 URL: https://issues.apache.org/jira/browse/HDFS-7066
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: datanode
>    Affects Versions: 2.5.1
>            Reporter: Xiaoyu Yao
>            Assignee: Xiaoyu Yao
>            Priority: Minor
>
> LazyWriter#evictBlocks (added for HDFS-6581) misses a null check for replicaState. As
a result, there are many NPEs in the debug log under certain conditions. 
> 2014-09-15 14:27:10,820 DEBUG impl.FsDatasetImpl (FsDatasetImpl.java:evictBlocks(2335))
- Evicting block null
> 2014-09-15 14:27:10,821 WARN  impl.FsDatasetImpl (FsDatasetImpl.java:run(2409)) - Ignoring
exception in LazyWriter:
> java.lang.NullPointerException
> 	at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl$LazyWriter.evictBlocks(FsDatasetImpl.java:2343)
> 	at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl$LazyWriter.run(FsDatasetImpl.java:2396)
> 	at java.lang.Thread.run(Thread.java:745)
> The proposed fix is to break if there is no candidate available to evict.
> {code}
> 	while (iterations++ < MAX_BLOCK_EVICTIONS_PER_ITERATION &&
>             transientFreeSpaceBelowThreshold()) {
>        LazyWriteReplicaTracker.ReplicaState replicaState =
>            lazyWriteReplicaTracker.getNextCandidateForEviction();
>        ++++++++++++++++++++
> 	if (replicaState == null) {
>           break;
>        }
>        ++++++++++++++++++++
>        if (LOG.isDebugEnabled()) {
>          LOG.debug("Evicting block " + replicaState);
>        }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message