ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmytro Sen" <d...@hortonworks.com>
Subject Re: Review Request 34846: Ooozie start takes too long
Date Sat, 30 May 2015 16:22:30 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34846/#review85869
-----------------------------------------------------------

Ship it!


Ship It!

- Dmytro Sen


On Май 30, 2015, 4:21 п.п., Andrew Onischuk wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34846/
> -----------------------------------------------------------
> 
> (Updated Май 30, 2015, 4:21 п.п.)
> 
> 
> Review request for Ambari and Dmytro Sen.
> 
> 
> Bugs: AMBARI-11558
>     https://issues.apache.org/jira/browse/AMBARI-11558
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Currently Oozie Start task takes ~3 minutes. Which includes some time-
> expensive actions like:  
> 1\. Extracting a big tar archive  
> 2\. Execute prepare-war fom it  
> 3\. Checking for hdfs directory via hadoop binary.
> 
> 1 and 2 can be avoid on the non-first starts, by saving checksum of archive
> which was extracted, and not re-extracting it unless the checksum changed, or
> the unextracted folder is gone.
> 
> 3 can benefit from using fast WebHDFS calls.
> 
> **As a result for me for non-first Oozie start this reduced time of start from 180 seconds
to 13-30 seconds.**
> 
> 
> Diffs
> -----
> 
>   ambari-common/src/main/python/resource_management/core/logger.py c060753 
>   ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py
25f0cd5 
>   ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
c773fca 
>   ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py
ef721ba 
>   ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
6963410 
>   ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 284f392 
>   ambari-server/src/test/python/stacks/utils/RMFTestCase.py 2f608c4 
> 
> Diff: https://reviews.apache.org/r/34846/diff/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message