community-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From s...@apache.org
Subject svn commit: r1714305 - /comdev/projects.apache.org/scripts/cronjobs/parsecommitters.py
Date Sat, 14 Nov 2015 11:59:13 GMT
Author: sebb
Date: Sat Nov 14 11:59:13 2015
New Revision: 1714305

URL: http://svn.apache.org/viewvc?rev=1714305&view=rev
Log:
Add caching for the HTML file
Mainly for testing purposes, as the time between runs is much longer than the interval

Modified:
    comdev/projects.apache.org/scripts/cronjobs/parsecommitters.py

Modified: comdev/projects.apache.org/scripts/cronjobs/parsecommitters.py
URL: http://svn.apache.org/viewvc/comdev/projects.apache.org/scripts/cronjobs/parsecommitters.py?rev=1714305&r1=1714304&r2=1714305&view=diff
==============================================================================
--- comdev/projects.apache.org/scripts/cronjobs/parsecommitters.py (original)
+++ comdev/projects.apache.org/scripts/cronjobs/parsecommitters.py Sat Nov 14 11:59:13 2015
@@ -11,11 +11,14 @@ Reads http://people.apache.org/committer
 import re
 import urllib.request
 import json
+import urlutils
+fc = urlutils.UrlCache(interval=600)
 
 people = {}
 people_name = {}
 groups = {}
-data = urllib.request.urlopen("http://people.apache.org/committer-index.html").read().decode('utf-8')
+data = fc.get("http://people.apache.org/committer-index.html","committer-index.html", encoding='utf-8').read()
+#data = urllib.request.urlopen("http://people.apache.org/committer-index.html").read().decode('utf-8')
 x = 0
 for committer in re.findall(r"<tr>([\S\s]+?)</tr>", data, re.MULTILINE | re.UNICODE):
     x += 1



Mime
View raw message