Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 55C7F17BE7 for ; Wed, 8 Oct 2014 01:49:35 +0000 (UTC) Received: (qmail 32182 invoked by uid 500); 8 Oct 2014 01:49:35 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 32133 invoked by uid 500); 8 Oct 2014 01:49:35 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 32121 invoked by uid 99); 8 Oct 2014 01:49:35 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Oct 2014 01:49:35 +0000 Date: Wed, 8 Oct 2014 01:49:34 +0000 (UTC) From: "Jing Zhao (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HDFS-7185) The active NameNode will not accept an fsimage sent from the standby during rolling upgrade MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7185: ---------------------------- Status: Patch Available (was: Open) > The active NameNode will not accept an fsimage sent from the standby during rolling upgrade > ------------------------------------------------------------------------------------------- > > Key: HDFS-7185 > URL: https://issues.apache.org/jira/browse/HDFS-7185 > Project: Hadoop HDFS > Issue Type: Bug > Affects Versions: 2.6.0 > Reporter: Colin Patrick McCabe > Assignee: Jing Zhao > Attachments: HDFS-7185.000.patch > > > The active NameNode will not accept an fsimage sent from the standby during rolling upgrade. The active fails with the exception: > {code} > 18:25:07,620 WARN ImageServlet:198 - Received an invalid request file transfer request from a secondary with storage info -59:65195028:0:CID-385de4d7-64e4-4dde-9f5d-0a6e431987f6 > 18:25:07,620 WARN log:76 - Committed before 410 PutImage failed. java.io.IOException: This namenode has storage info -55:65195028:0:CID-385de4d7-64e4-4dde-9f5d-0a6e431987f6 but the secondary expected -59:65195028:0:CID-385de4d7-64e4-4dde-9f5d- > 0a6e431987f6 > at org.apache.hadoop.hdfs.server.namenode.ImageServlet.validateRequest(ImageServlet.java:200) > at org.apache.hadoop.hdfs.server.namenode.ImageServlet.doPut(ImageServlet.java:443) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:730) > {code} > On the standby, the exception is: > {code} > java.io.IOException: Exception during image upload: org.apache.hadoop.hdfs.server.namenode.TransferFsImage$HttpPutFailedException: This namenode has storage info -55:65195028:0:CID-385de4d7-64e4-4dde-9f5d-0a6e431987f6 but the secondary expected > -59:65195028:0:CID-385de4d7-64e4-4dde-9f5d-0a6e431987f6 > at org.apache.hadoop.hdfs.server.namenode.ha.StandbyCheckpointer.doCheckpoint(StandbyCheckpointer.java:218) > at org.apache.hadoop.hdfs.server.namenode.ha.StandbyCheckpointer.access$1400(StandbyCheckpointer.java:62) > {code} > This seems to be a consequence of the fact that the VERSION file still is at -55 (the old version) even after the rolling upgrade has started. When the rolling upgrade is finalized with {{hdfs dfsadmin -rollingUpgrade finalize}}, both VERSION files get set to the new version, and the problem goes away. -- This message was sent by Atlassian JIRA (v6.3.4#6332)