ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmytro Grinenko" <hapy.les...@gmail.com>
Subject Re: Review Request 36302: Ambari should not configure hbase.regionserver.global.memstore.upperLimit in HDP-2.3
Date Wed, 08 Jul 2015 16:10:32 GMT


> On July 8, 2015, 3:33 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/themes/theme.json,
line 28
> > <https://reviews.apache.org/r/36302/diff/1/?file=1002100#file1002100line28>
> >
> >     Shouldn't this be removed since the property was deleted in 2.3?

actuly the sense of this is to remove the property, themes.json is a thing which inherited
from stack to stack per section and if we do not re-define them to null, they will stay.


> On July 8, 2015, 3:33 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/themes/theme.json,
lines 28-29
> > <https://reviews.apache.org/r/36302/diff/1/?file=1002104#file1002104line28>
> >
> >     Wasn't this removed from 2.3 in the upgrade pack:
> >     
> >     ```
> >     <transfer operation="delete" delete-key="hbase.regionserver.global.memstore.upperLimit"/>
> >     ```

actuly the sense of this is to remove the property, themes.json is a thing which inherited
from stack to stack per section and if we do not re-define them to null, they will stay.


> On July 8, 2015, 3:33 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/themes/theme.json,
lines 28-29
> > <https://reviews.apache.org/r/36302/diff/1/?file=1002104#file1002104line28>
> >
> >     Wasn't this removed from 2.3 in the upgrade pack:
> >     
> >     ```
> >     <transfer operation="delete" delete-key="hbase.regionserver.global.memstore.upperLimit"/>
> >     ```

actuly the sense of this is to remove the property, themes.json is a thing which inherited
from stack to stack per section and if we do not re-define them to null, they will stay.


> On July 8, 2015, 3:33 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/HBaseConfigCalculation.java,
line 34
> > <https://reviews.apache.org/r/36302/diff/1/?file=1002095#file1002095line34>
> >
> >     Documentation?

sure


> On July 8, 2015, 3:33 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/HBaseConfigCalculation.java,
lines 53-55
> > <https://reviews.apache.org/r/36302/diff/1/?file=1002095#file1002095line53>
> >
> >     The only way that this would run is if HBase was installed, so how could the
config not be found? If you're worried about a null here, that's fine, but I don't think it
should be `HostRoleStatus.COMPELTED`

The presens of the service not mean that some config couldn't be absent(due different reasons
incl. inconsistent state of configs, other bugs), but if this happend we catch here NPE, right?
About the status, we can return fail here, ok.


> On July 8, 2015, 3:33 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/hbase-site.xml,
lines 43-52
> > <https://reviews.apache.org/r/36302/diff/1/?file=1002098#file1002098line43>
> >
> >     These are removed from 2.3, right?

yes, <deleted>true</deleted> is set.


> On July 8, 2015, 3:33 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/configuration/hbase-site.xml,
lines 43-52
> > <https://reviews.apache.org/r/36302/diff/1/?file=1002102#file1002102line43>
> >
> >     These were removed from 2.3, right?

yes, <deleted>true</deleted> is set.


- Dmytro


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36302/#review90909
-----------------------------------------------------------


On July 8, 2015, 3:10 p.m., Dmitro Lisnichenko wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36302/
> -----------------------------------------------------------
> 
> (Updated July 8, 2015, 3:10 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-12224
>     https://issues.apache.org/jira/browse/AMBARI-12224
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> HBase has deprecated hbase.regionserver.global.memstore.upperLimit and hbase.regionserver.global.memstore.lowerLimit
for HDP-2.3 in favor of hbase.regionserver.global.memstore.size and hbase.regionserver.global.memstore.size.lower.limit.
> The slider called "% of RegionServer Allocated to Write Buffers" sets hbase.regionserver.global.memstore.size,
while hbase.regionserver.global.memstore.upperLimit is in the advanced configuration section.
This leaves two bugs:
> In HDP-2.2, the slider has no affect since HBase basically ignores the parameter.
> In HDP-2.3 the value set by the slider also has no affect since the deprecated values
set take precedence.
> One solution is to get rid of hbase.regionserver.global.memstore.upperLimit and hbase.regionserver.global.memstore.lowerLimit
in advanced settings in Ambari, and make the slider configuration to use hbase.regionserver.global.memstore.upperLimit
in HDP-2.2 and hbase.regionserver.global.memstore.size in HDP-2.3. Rolling upgrade can convert
these settings.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/HBaseConfigCalculation.java
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/themes/theme.json 53da86d

>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml c5fb156 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/hbase-site.xml
12cd44b 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/metainfo.xml 921a225

>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/themes/theme.json PRE-CREATION

>   ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/themes/theme.json
1808809 
>   ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/configuration/hbase-site.xml
43f872a 
>   ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/metainfo.xml aa169df

>   ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/themes/theme.json
PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/36302/diff/
> 
> 
> Testing
> -------
> 
> manual checks
> Tests in error: 
>   UpgradeResourceProviderTest.testCreatePartialDowngrade:428 ? NullPointer
> 
> Tests run: 3111, Failures: 0, Errors: 1, Skipped: 28
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Ambari Views ...................................... SUCCESS [6.409s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [3.909s]
> [INFO] Ambari Server ..................................... FAILURE [1:16:42.534s]
> [INFO] Ambari Agent ...................................... SKIPPED
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 1:16:57.150s
> [INFO] Finished at: Wed Jul 08 10:34:40 UTC 2015
> [INFO] Final Memory: 39M/826M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message