Mailing-List: contact dev-help@ambari.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ambari.apache.org
Content-Type: multipart/alternative;
 boundary="===============8206994124810234279=="
MIME-Version: 1.0
Subject: Re: Review Request 40080: AMBARI-13748: Being able to clean
 repository cache before installing open source components
From: "Di Li" <dili@ca.ibm.com>
To: "Alejandro Fernandez" <afernandez@hortonworks.com>,
 "Andrew Onischuk" <aonishuk@hortonworks.com>,
 "Jayush Luniya" <jluniya@hortonworks.com>,
 "Sid Wagle" <swagle@hortonworks.com>
Cc: "Di Li" <dili@ca.ibm.com>, "Ambari" <dev@ambari.apache.org>
Date: Wed, 11 Nov 2015 17:53:40 -0000
Message-ID: <20151111175340.1680.63434@reviews.apache.org>
Auto-Submitted: auto-generated
Sender: "Di Li" <noreply@reviews.apache.org>
References: <20151111160315.1681.4212@reviews.apache.org>
In-Reply-To: <20151111160315.1681.4212@reviews.apache.org>
Reply-To: "Di Li" <dili@ca.ibm.com>

--===============8206994124810234279==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


> On Nov. 11, 2015, 4:03 p.m., Andrew Onischuk wrote:
> > Di, also one more question. As far as I know, new repositories are automatically added to cache on suse/rhel when first yum/zypper install is triggered. Do we target something else here?
> 
> Di Li wrote:
>     hello Andrew,
>     
>     Thanks for the explanation. I will just use this one for all your comments
>     
>     1. Repo vs package install in the basic case: I guess one way to deal with this is to create repo on each agent before installing packages. This require some serious changes to the way agent install packages, I suppose. I get it that trying to create the repo file per package install can make sure the repo file content is always up to date.
>     
>     2. Locks: it's not being handled right now. 
>     
>     3. Purpose of the jira: This is a very special case where the cache contains out of date info of HDP which may cause the install pacakge steps to fail when user tries to run rolling upgrade or do a fresh install. This may happen for a cluster previous installed with HDP (when user does a floppy job on clean up the cluster or clean up the repo cache properly). The logic here is try to do some cache cleaning when Ambari creates the repo file on each Agent node. 
>     
>     Just as you said, new repo logic in SLES/RHEL is perfectly fine, for a cluster that has never been installed with HDP before or when user properly cleans up repo cache.
> 
> Andrew Onischuk wrote:
>     Di, ok I get it but the problem is slowness. My proposal to fix this would be the way we have done this for apt-get intall vs apt-get update.
>     (check apt.py in re
>     
>     Please check apt.py. 
>           if code:
>             Logger.info("Execution of '%s' returned %d. %s" % (cmd, code, out))
>             Logger.info("Failed to install package %s. Executing `%s`" % (name, string_cmd_from_args_list(REPO_UPDATE_CMD)))
>             code, out = shell.call(REPO_UPDATE_CMD, sudo=True, logoutput=self.get_logoutput())
>             
>     My proposal to fix this in the same way. meaning:
>     
>     1. Try to install package
>     2. If it fails we do 'yum clean all'
>     3. Try again, if it fails, fail the task.
>     
>     
>     I think this is the way this issue should be handled
>     
>     regards,
>     Andrew.
> 
> Andrew Onischuk wrote:
>     Also please don't forget to add log messages about what is done at every point. Like in the example I gave you above.
> 
> Andrew Onischuk wrote:
>     Di Li can I get your skype? Or can you add me empower-666. To get this conversation faster on?

hello Andrew,

thanks for the pointers, I will work on your proposed approach and submit a new patch as soon as possible.

Thank you.

Di


- Di


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40080/#review106074
-----------------------------------------------------------


On Nov. 10, 2015, 9:02 p.m., Di Li wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40080/
> -----------------------------------------------------------
> 
> (Updated Nov. 10, 2015, 9:02 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Jayush Luniya, and Sid Wagle.
> 
> 
> Bugs: AMBARI-13748
>     https://issues.apache.org/jira/browse/AMBARI-13748
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Need a way for Ambari to trigger a repository clean (yum clean expire-cache on rhel, for example) when updating repository files during an Ambari update as well as fresh installation of a cluster.
> 
> Currently, this must be done occasionally through a manual step, and if not done will lead to problems during install.
> 
> 
> Diffs
> -----
> 
>   ambari-common/src/main/python/resource_management/libraries/providers/repository.py 11002cc 
>   ambari-common/src/main/python/resource_management/libraries/resources/repository.py 2484d67 
>   ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 1dfaa54 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py 05751fa 
>   ambari-server/src/test/python/custom_actions/TestInstallPackages.py 83b6bb5 
>   ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py f9d69a8 
> 
> Diff: https://reviews.apache.org/r/40080/diff/
> 
> 
> Testing
> -------
> 
> New unit test. Patch cluster with the changes and run fresh install and rolling upgrade to verify the repo clean up logic was run.
> 
> 
> Thanks,
> 
> Di Li
> 
>


--===============8206994124810234279==--