From commits-return-1176-archive-asf-public=cust-asf.ponee.io@ponymail.incubator.apache.org Thu Apr 25 17:43:17 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 49050180638 for ; Thu, 25 Apr 2019 19:43:17 +0200 (CEST) Received: (qmail 24849 invoked by uid 500); 25 Apr 2019 17:43:16 -0000 Mailing-List: contact commits-help@ponymail.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ponymail.incubator.apache.org Delivered-To: mailing list commits@ponymail.incubator.apache.org Received: (qmail 24838 invoked by uid 99); 25 Apr 2019 17:43:16 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Apr 2019 17:43:16 +0000 Received: by gitbox.apache.org (ASF Mail Server at gitbox.apache.org, from userid 33) id 76A52853D5; Thu, 25 Apr 2019 17:43:16 +0000 (UTC) Date: Thu, 25 Apr 2019 17:43:16 +0000 To: "commits@ponymail.apache.org" Subject: [incubator-ponymail] branch master updated: Update nullfav.py to work with ES 5.x MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Message-ID: <155621419643.8821.17696536999903335897@gitbox.apache.org> From: sebb@apache.org X-Git-Host: gitbox.apache.org X-Git-Repo: incubator-ponymail X-Git-Refname: refs/heads/master X-Git-Reftype: branch X-Git-Oldrev: efde1ceadb5e91b54fd34ca4efa6ef36cc78b6c1 X-Git-Newrev: 32555fbb5dae7dce70bcf10b1a9ae9aa78a8f077 X-Git-Rev: 32555fbb5dae7dce70bcf10b1a9ae9aa78a8f077 X-Git-NotificationType: ref_changed_plus_diff X-Git-Multimail-Version: 1.5.dev Auto-Submitted: auto-generated This is an automated email from the ASF dual-hosted git repository. sebb pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-ponymail.git The following commit(s) were added to refs/heads/master by this push: new 32555fb Update nullfav.py to work with ES 5.x 32555fb is described below commit 32555fbb5dae7dce70bcf10b1a9ae9aa78a8f077 Author: Sebb AuthorDate: Thu Apr 25 18:42:50 2019 +0100 Update nullfav.py to work with ES 5.x This addresses #489 by using the new scan_and_scroll iterator. --- CHANGELOG.md | 1 + tools/nullfav.py | 22 +++++----------------- 2 files changed, 6 insertions(+), 17 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bfd606b..5c196e0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,6 @@ ## Changes in 0.12: - Bug: Errors with Elasticsearch 5.x in edit-lists.py (#489) +- Bug: Errors with Elasticsearch 5.x in nullfav.py (#489) - Enh: Support scan/scroll for all current versions of ES (#489-contd) - Bug/Enh: setup.py tries to install Python modules (#465) diff --git a/tools/nullfav.py b/tools/nullfav.py index 4d0061e..f630cb2 100755 --- a/tools/nullfav.py +++ b/tools/nullfav.py @@ -35,23 +35,11 @@ FAVES='favorites' TARGET=None elastic = Elastic() -scroll = '5m' -page = elastic.scan(doc_type='account', - scroll = scroll, - body = { - "_source" : [FAVES], - } -) -sid = page['_scroll_id'] -scroll_size = page['hits']['total'] -print("Found %d accounts" % scroll_size) - -updated=0 -failed=0 -while (scroll_size > 0): - page = elastic.scroll(scroll_id = sid, scroll = scroll) - sid = page['_scroll_id'] - scroll_size = len(page['hits']['hits']) +scroll_size = None # Only show it first time round +for page in elastic.scan_and_scroll(doc_type='account', body = { "_source" : [FAVES] }): + if not scroll_size: + scroll_size = page['hits']['total'] + print("Found %d accounts" % scroll_size) for hit in page['hits']['hits']: mid = hit['_id'] source = hit['_source']