ponymail-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From humbed...@apache.org
Subject incubator-ponymail git commit: crop out trailing whitespace for redundant archiver
Date Sat, 02 Sep 2017 08:02:44 GMT
Repository: incubator-ponymail
Updated Branches:
  refs/heads/master c8f4d3b7d -> df0b7ee1c


crop out trailing whitespace for redundant archiver

This deals with spurious whitespace that can exist on
clustered setups due to corrections inside the MTAs.
This only deals with trailing whitespace, everything else
is preserved.

Also add some spacing in the code.


Project: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/commit/df0b7ee1
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/tree/df0b7ee1
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/diff/df0b7ee1

Branch: refs/heads/master
Commit: df0b7ee1c5676b3286e4b30c135f410b1e0a92da
Parents: c8f4d3b
Author: Daniel Gruno <humbedooh@apache.org>
Authored: Sat Sep 2 10:02:10 2017 +0200
Committer: Daniel Gruno <humbedooh@apache.org>
Committed: Sat Sep 2 10:02:10 2017 +0200

----------------------------------------------------------------------
 tools/generators.py | 9 +++++++++
 1 file changed, 9 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-ponymail/blob/df0b7ee1/tools/generators.py
----------------------------------------------------------------------
diff --git a/tools/generators.py b/tools/generators.py
index e320a06..b37fa8b 100644
--- a/tools/generators.py
+++ b/tools/generators.py
@@ -22,6 +22,7 @@ This file contains the various ID generators for Pony Mail's archivers.
 import hashlib
 import email.utils
 import time
+import re
 
 # Full generator: uses the entire email (including server-dependent data)
 # This is the recommended generator for single-node setups.
@@ -95,8 +96,13 @@ def redundant(msg, body, lid, attachments):
     if not body: # Make sure body is not None, which will fail.
         body = ""
     xbody = body if type(body) is bytes else body.encode('ascii', 'ignore')
+    
+    # Crop out any trailing whitespace in body
+    xbody = re.sub(b"\s+$", b"", xbody)
+    
     # Use List ID
     xbody += bytes(lid, encoding='ascii')
+    
     # Use Date header. Don't use archived-at, as the archiver sets this if not present.
     mdate = None
     mdatestring = "(null)" # Default to null, ONLY changed if replicable across imports
@@ -106,14 +112,17 @@ def redundant(msg, body, lid, attachments):
     except:
         pass
     xbody += bytes(mdatestring, encoding='ascii')
+    
     # Use sender
     sender = msg.get('from', None)
     if sender:
         xbody += bytes(sender, encoding = 'ascii')
+    
     # Use subject
     subject = msg.get('subject', None)
     if subject:
         xbody += bytes(subject, encoding = 'ascii')
+    
     # Use attachment hashes if present
     if attachments:
         for a in attachments:


Mime
View raw message