hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-13834) RBF: Connection creator thread should catch Throwable
Date Mon, 24 Jun 2019 18:03:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16871569#comment-16871569
] 

Hudson commented on HDFS-13834:
-------------------------------

FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #16813 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/16813/])
HDFS-13834. RBF: Connection creator thread should catch Throwable. (brahma: rev fa55eacd357f4c4524fb54598e989c5746921de2)
* (edit) hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestConnectionManager.java
* (edit) hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/ConnectionManager.java


> RBF: Connection creator thread should catch Throwable
> -----------------------------------------------------
>
>                 Key: HDFS-13834
>                 URL: https://issues.apache.org/jira/browse/HDFS-13834
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: CR Hota
>            Assignee: CR Hota
>            Priority: Critical
>             Fix For: HDFS-13891
>
>         Attachments: HDFS-13834-HDFS-13891.0.patch, HDFS-13834-HDFS-13891.1.patch, HDFS-13834-HDFS-13891.2.patch,
HDFS-13834-HDFS-13891.3.patch, HDFS-13834-HDFS-13891.4.patch, HDFS-13834-HDFS-13891.5.patch,
HDFS-13834.0.patch, HDFS-13834.1.patch
>
>
> Connection creator thread is a single thread thats responsible for creating all downstream
namenode connections.
> This is very critical thread and hence should not die understand exception/error scenarios.
> We saw this behavior in production systems where the thread died leaving the router process
in bad state.
> The thread should also catch a generic error/exception.
> {code}
>     @Override
>     public void run() {
>       while (this.running) {
>         try {
>           ConnectionPool pool = this.queue.take();
>           try {
>             int total = pool.getNumConnections();
>             int active = pool.getNumActiveConnections();
>             if (pool.getNumConnections() < pool.getMaxSize() &&
>                 active >= MIN_ACTIVE_RATIO * total) {
>               ConnectionContext conn = pool.newConnection();
>               pool.addConnection(conn);
>             } else {
>               LOG.debug("Cannot add more than {} connections to {}",
>                   pool.getMaxSize(), pool);
>             }
>           } catch (IOException e) {
>             LOG.error("Cannot create a new connection", e);
>           }
>         } catch (InterruptedException e) {
>           LOG.error("The connection creator was interrupted");
>           this.running = false;
>         }
>       }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message