ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Di Li" <d...@ca.ibm.com>
Subject Re: Review Request 37437: AMBARI-12751: MapRed history server is shown as stopped after downgrade
Date Thu, 13 Aug 2015 17:53:10 GMT


> On Aug. 13, 2015, 5:31 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml, line 580
> > <https://reviews.apache.org/r/37437/diff/1/?file=1039334#file1039334line580>
> >
> >     Di, can you explain this some more?
> >     This is performing an RU from HDP 2.2 -> 2.3, and that property exists in
HDP 2.3, so due to config merging, it means that if the property does not exist or the user
has not modified it, it will be set to  *...HistoryServerLeveldbStateStoreService.
> >     Note: if the user does modify it, their value will be preserved.
> >     
> >     From your description, it sounds like the property does not work in HDP 2.2,
so why should there be a PreCheck for it at all?

hello Alejandro,

AMBARI-12276 added "MapReduce2JobHistoryStatePreservingCheck" class, a RU precheck class that
checks whether "mapreduce.jobhistory.recovery.enable", "mapreduce.jobhistory.recovery.store.class",
and "mapreduce.jobhistory.recovery.store.leveldb.path" are set in the mapred-site.xml.

In my tests, I updated an HDP 2.2 cluster to HDP 2.3, then I clicked downgrade instead of
Finalize. The downgrade kept the three properties in the configuration for the MapRed2 history
server. Because the HDP 2.2 version of the MapRed history server does not have the class,
it failed to start and shown as stopped post downgrade.

So my thinking was that I would modify the upgrade-2.3.xml to remove the three mapred properties
during the downgrade. If users want to perform another round of rolling upgrade, they will
need to manually add the three properties back to the mapred-site.xml


- Di


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37437/#review95291
-----------------------------------------------------------


On Aug. 13, 2015, 1:13 p.m., Di Li wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37437/
> -----------------------------------------------------------
> 
> (Updated Aug. 13, 2015, 1:13 p.m.)
> 
> 
> Review request for Ambari and Alejandro Fernandez.
> 
> 
> Bugs: AMBARI-12751
>     https://issues.apache.org/jira/browse/AMBARI-12751
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> RU precheck requires "mapreduce.jobhistory.recovery.store.class" property to be set in
the mapred-site.xml with value "org.apache.hadoop.mapreduce.v2.hs.HistoryServerLeveldbStateStoreService"
> 
> With this value set, the HDP 2.2 level MapRed history server can not start. It shows
an err about class org.apache.hadoop.mapreduce.v2.hs.HistoryServerLeveldbStateStoreService
does not exist.
> 
> When downgrading, the mapred properties added to pass the RU precheck needs to be removed.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml f6823c8 
> 
> Diff: https://reviews.apache.org/r/37437/diff/
> 
> 
> Testing
> -------
> 
> manually patch cluster to run rolling upgrade, after it passes the Core Master step (which
includes upgrading the MapRed2 History server), run downgrade, verify the MapRed2 history
server is still running once the downgrade is done, the three mapred properties added to satisfy
the ru precheck are removed.
> 
> 
> Thanks,
> 
> Di Li
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message