Return-Path: X-Original-To: apmail-incubator-allura-commits-archive@minotaur.apache.org Delivered-To: apmail-incubator-allura-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E670910C3E for ; Wed, 23 Oct 2013 12:57:27 +0000 (UTC) Received: (qmail 40822 invoked by uid 500); 23 Oct 2013 12:57:11 -0000 Delivered-To: apmail-incubator-allura-commits-archive@incubator.apache.org Received: (qmail 40174 invoked by uid 500); 23 Oct 2013 12:56:46 -0000 Mailing-List: contact allura-commits-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: allura-dev@incubator.apache.org Delivered-To: mailing list allura-commits@incubator.apache.org Received: (qmail 39707 invoked by uid 99); 23 Oct 2013 12:56:39 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Oct 2013 12:56:39 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id 2E5B631931D; Wed, 23 Oct 2013 12:56:39 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: tvansteenburgh@apache.org To: allura-commits@incubator.apache.org Date: Wed, 23 Oct 2013 12:57:06 -0000 Message-Id: <2f4af4bafdb449daa1276cf5097f3097@git.apache.org> In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [29/50] git commit: [#6759] Gracefully handle HTTP errors when importing GC tickets [#6759] Gracefully handle HTTP errors when importing GC tickets GC will sometimes list tickets that actually 404 when you try to view them. Nothing we can do about that but log it and keep going. Signed-off-by: Cory Johns Project: http://git-wip-us.apache.org/repos/asf/incubator-allura/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-allura/commit/0e8aa5e6 Tree: http://git-wip-us.apache.org/repos/asf/incubator-allura/tree/0e8aa5e6 Diff: http://git-wip-us.apache.org/repos/asf/incubator-allura/diff/0e8aa5e6 Branch: refs/heads/tv/6610 Commit: 0e8aa5e6a939c73b38a3ef5bc66a5f1e348aeb60 Parents: 658f68f Author: Cory Johns Authored: Wed Oct 16 18:01:54 2013 +0000 Committer: Cory Johns Committed: Wed Oct 16 18:03:11 2013 +0000 ---------------------------------------------------------------------- ForgeImporters/forgeimporters/google/__init__.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-allura/blob/0e8aa5e6/ForgeImporters/forgeimporters/google/__init__.py ---------------------------------------------------------------------- diff --git a/ForgeImporters/forgeimporters/google/__init__.py b/ForgeImporters/forgeimporters/google/__init__.py index 1a2c5c1..2f51aa0 100644 --- a/ForgeImporters/forgeimporters/google/__init__.py +++ b/ForgeImporters/forgeimporters/google/__init__.py @@ -17,6 +17,7 @@ import re import urllib +from urllib2 import HTTPError from urlparse import urlparse, urljoin, parse_qs from collections import defaultdict from contextlib import closing @@ -135,7 +136,11 @@ class GoogleCodeProjectExtractor(ProjectExtractor): if len(extractor.page) <= 0: return for issue_id in extractor.page: - yield (int(issue_id), cls(project_name, 'issue', issue_id=issue_id)) + try: + yield (int(issue_id), cls(project_name, 'issue', issue_id=issue_id)) + except HTTPError as e: + log.warn('Unable to load GC issue: %s #%s: %s', project_name, issue_id, e) + continue start += limit extractor.get_page('issues_csv', parser=csv_parser, start=start)