ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom Beerbower" <tbeerbo...@hortonworks.com>
Subject Re: Review Request 30648: RU - Add more debugging information to HDFS prepare and finalize steps
Date Fri, 06 Feb 2015 04:58:14 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30648/#review71375
-----------------------------------------------------------

Ship it!


Ship It!

- Tom Beerbower


On Feb. 5, 2015, 8:04 p.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30648/
> -----------------------------------------------------------
> 
> (Updated Feb. 5, 2015, 8:04 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, Tom Beerbower,
and Yurii Shylov.
> 
> 
> Bugs: AMBARI-9485
>     https://issues.apache.org/jira/browse/AMBARI-9485
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Add more debugging information for HDFS prepare and finalize steps.
> 
> Prepare:
> 1. With HA setup, the current code requires the NameNode NOT in safemode when starting
RU.
> 2. With non-HA setup, the current code requires the NameNode in safemode when start RU.
> 
> The reason is that for non-HA setup, we need to do an extra checkpoint in the single
NN (by calling saveNamespace), which requires the NN to be in safemode (to avoid namespace
modification during the checkpoint). But for HA-setup, the checkpoint is done by the standby
NN, thus we do not need to put the active NN into the safemode and in this way we can avoid
service downtime.
> 
> We need to log what the original state is.
> 
> Finalize:
> This has a confusing message related to https://issues.apache.org/jira/secure/attachment/12679263/HDFS-7340.001.patch
> 
> ```
> [root@c6404 ~]# su - hdfs -c 'hdfs dfsadmin -rollingUpgrade query'
> QUERY rolling upgrade ...
> Proceed with rolling upgrade:
>   Block Pool ID: BP-819970438-192.168.64.104-1423091248521
>      Start Time: Thu Feb 05 01:15:16 UTC 2015 (=1423098916798)
>   Finalize Time: <NOT FINALIZED>
> 
> [root@c6404 ~]# su - hdfs -c 'hdfs dfsadmin -rollingUpgrade finalize'
> FINALIZE rolling upgrade ...
> There is no rolling upgrade in progress or rolling upgrade has already been finalized.
> 
> [root@c6404 ~]# su - hdfs -c 'hdfs dfsadmin -rollingUpgrade query'
> QUERY rolling upgrade ...
> There is no rolling upgrade in progress or rolling upgrade has already been finalized.
> ```
> 
> This means that the finalize did happen.
> We should run another "hdfs dfsadmin -rollingUpgrade query" after finalize to ensure
this happened.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py
e11546c 
> 
> Diff: https://reviews.apache.org/r/30648/diff/
> 
> 
> Testing
> -------
> 
> Installed a 3-node cluster with HDFS and ZK, and Namenode HA, then verified that Namenode
prepare showed the additional output, and that HDFS finalize made it clear that it did happen.
> 
> Unit tests passed.
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 29:57.240s
> [INFO] Finished at: Wed Feb 04 18:47:14 PST 2015
> [INFO] Final Memory: 60M/496M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message