From java-user-return-64240-archive-asf-public=cust-asf.ponee.io@lucene.apache.org Tue Mar 19 09:25:08 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 199E4180626 for ; Tue, 19 Mar 2019 10:25:07 +0100 (CET) Received: (qmail 47270 invoked by uid 500); 19 Mar 2019 09:25:06 -0000 Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-user@lucene.apache.org Delivered-To: mailing list java-user@lucene.apache.org Received: (qmail 47258 invoked by uid 99); 19 Mar 2019 09:25:05 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 Mar 2019 09:25:05 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 0DA5FC58FC for ; Tue, 19 Mar 2019 09:25:05 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2 X-Spam-Level: ** X-Spam-Status: No, score=2 tagged_above=-999 required=6.31 tests=[DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=mikemccandless-com.20150623.gappssmtp.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id l1gXr6AtyGxf for ; Tue, 19 Mar 2019 09:25:03 +0000 (UTC) Received: from mail-vs1-f50.google.com (mail-vs1-f50.google.com [209.85.217.50]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 0929F60F6F for ; Tue, 19 Mar 2019 09:25:03 +0000 (UTC) Received: by mail-vs1-f50.google.com with SMTP id z25so9942856vsk.8 for ; Tue, 19 Mar 2019 02:25:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mikemccandless-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=6JO0xILz2dZH3qMEsxkJ71yXN7nmClyaXRlBPFU8IyY=; b=JGN9bJFIXDxVzcxd8a3wYgHJDykS2F0FSbDFArWHBey+6ZS7N17S8kRHF07qVSYWQE J/ehe8odDaOvFahD4SO48wmZL/g4j8XAO/x57M8RemZBCWT+VnV4waQGGpX8GHgOZ1It 5SYvA8cvqFZ5wwuCjDy9uqNib+y+pMwQcp3RL5tgvLReWKfWLDmkgPLzst8qfCBHWb/k 8+SA+s7fWvBGDJVClX0Okksz6Ng+BdKYTMHRoiIVXCQQfi9QkiM5Yda/CqI/IhRxpKfX TeNOnKJFpnl8Ls4NeDDUrChXe7/uhJwG5CwQUPo47QTxF7vG3EXDb+MOtwgEMVlO5s/Y yMjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=6JO0xILz2dZH3qMEsxkJ71yXN7nmClyaXRlBPFU8IyY=; b=UIqyhQ6Fx/ZIjVrpNZAGpHzfYfKR0m+1hvqY1CXOuHl+nL+BmZ3IrFkkO46aRaFjv/ E8flBdyu+PoP6or9BZMTG3pL4U8CukAW3wDTeH/AIgImRWnsaB4OboQyI0wQguBHoM4A 1vWkNu2LNekCLzDsbusUeftXzDVRLFStRFvm+5gIiFeV4sbmdg+FpPvkDRLRwTjRmgbZ efY9ybgTY+lKNWlG9qSIKtZS7sqFu5LscaPMh0YSy/rHfkiNKeqPn81SiOF8In+0uFvb zXePFdrNnZF+FG0MW9GrH+ZkE7cTjSuGBRhWZIIwovZNJkQIZpVAgve1wEqSMacTbQWd YBIg== X-Gm-Message-State: APjAAAWp35ENMQvmFrLrKUkvILSSsB/6txqijJvk/124DeaJo0oke8FN /ajZI+KyTMChBtqT1zozO4Sk+KR191AhIIpOOc/2ceve X-Google-Smtp-Source: APXvYqxrd3cWz8whXCnaGt0bw5mx1JnTArk/HdE7yfUbTd9ZBoL756twYn5WMfXg4J+0iIl0PHk8U7+vVgNMS+e4pNI= X-Received: by 2002:a67:f105:: with SMTP id n5mr710575vsk.181.1552987495735; Tue, 19 Mar 2019 02:24:55 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Michael McCandless Date: Tue, 19 Mar 2019 05:24:19 -0400 Message-ID: Subject: Re: Ask about Lucene/Core/Index DocumentsWriter To: Lucene Users , luyunchenglgn@gmail.com Content-Type: multipart/alternative; boundary="0000000000000636c705846f141c" --0000000000000636c705846f141c Content-Type: text/plain; charset="UTF-8" Can you try increasing your IndexWriter.setRAMBufferSizeMB? That flush control logic will block incoming threads if the number of bytes trying to flush to disk is too large relative to your RAM buffer. Mike McCandless http://blog.mikemccandless.com On Mon, Mar 18, 2019 at 2:30 PM yuncheng lu wrote: > When i check the code lucene/core DocumentsWriter.preUpdate code. I see the > flushControl is used when thread is Stalled. > When we have a lot of documents write into disk which is SSD. We monitored > that the thread is all in flush, and request continuously use addDocument > which can go into preUpdate check, it make more flush request into IO > queue. As this situation, it make all request blocked, and disk util is no > full of usage. So may be this optimization can take more condition into > consideration? > > > Thanks & Best Regards > Yuncheng Lu > --0000000000000636c705846f141c--