ponymail-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From s...@apache.org
Subject [incubator-ponymail] branch master updated: Bug: push-failures.py expects to find 'id' key in json file
Date Fri, 18 May 2018 14:44:42 GMT
This is an automated email from the ASF dual-hosted git repository.

sebb pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-ponymail.git


The following commit(s) were added to refs/heads/master by this push:
     new 3418b63  Bug: push-failures.py expects to find 'id' key in json file
3418b63 is described below

commit 3418b6319d5224487d66e0f0ef27a3afb830d76c
Author: Sebb <sebb@apache.org>
AuthorDate: Fri May 18 15:44:40 2018 +0100

    Bug: push-failures.py expects to find 'id' key in json file
    
    This fixes #454
---
 CHANGELOG.md           | 1 +
 tools/archiver.py      | 1 +
 tools/push-failures.py | 8 ++++++--
 3 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index a5bf02b..656090b 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,4 +1,5 @@
 ## Changes in 0.11-SNAPSHOT
+- Bug: push-failures.py expects to find non-existent 'id' key in json file (#454)
 - Bug: ES 5.x does not support word-cloud (stats.lua) (#345)
 - Add version info to elastic module
 - Bug: setup.py fails with ES2 - fielddata (#453)
diff --git a/tools/archiver.py b/tools/archiver.py
index 15868a1..dc89e90 100755
--- a/tools/archiver.py
+++ b/tools/archiver.py
@@ -454,6 +454,7 @@ class Archiver(object): # N.B. Also used by import-mbox.py
                 mboxPath = os.path.join(dumpDir, "%s.json" % uid)
                 with open(mboxPath, "w") as f:
                     json.dump({
+                        'id': ojson['mid'],
                         'mbox': ojson,
                         'mbox_source': {
                             "message-id": msg_metadata['message-id'],
diff --git a/tools/push-failures.py b/tools/push-failures.py
old mode 100644
new mode 100755
index d3c22c8..e2b4b15
--- a/tools/push-failures.py
+++ b/tools/push-failures.py
@@ -70,17 +70,21 @@ for f in files:
     with open(fpath, "r") as f:
         ojson = json.load(f)
         if 'mbox' in ojson and 'mbox_source' in ojson:
+            try:
+                id = ojson['id']
+            except KeyError:
+                id = ojson['mbox']['mid']
             es.index(
                 index=dbname,
                 doc_type="mbox",
-                id=ojson['id'],
+                id=id,
                 body = ojson['mbox']
             )
             
             es.index(
                 index=dbname,
                 doc_type="mbox_source",
-                id=ojson['id'],
+                id=id,
                 body = ojson['mbox_source']
             )
             

-- 
To stop receiving notification emails like this one, please contact
sebb@apache.org.

Mime
View raw message