From solr-user-return-144114-archive-asf-public=cust-asf.ponee.io@lucene.apache.org Thu Sep 27 19:11:20 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 4B3E0180652 for ; Thu, 27 Sep 2018 19:11:19 +0200 (CEST) Received: (qmail 71072 invoked by uid 500); 27 Sep 2018 17:11:17 -0000 Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: solr-user@lucene.apache.org Delivered-To: mailing list solr-user@lucene.apache.org Received: (qmail 71060 invoked by uid 99); 27 Sep 2018 17:11:16 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 Sep 2018 17:11:16 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 519B01A0489 for ; Thu, 27 Sep 2018 17:11:16 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.888 X-Spam-Level: * X-Spam-Status: No, score=1.888 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id jmn6Rs0A12R5 for ; Thu, 27 Sep 2018 17:11:14 +0000 (UTC) Received: from mail-ot1-f46.google.com (mail-ot1-f46.google.com [209.85.210.46]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id F00595F254 for ; Thu, 27 Sep 2018 17:11:13 +0000 (UTC) Received: by mail-ot1-f46.google.com with SMTP id h26-v6so3271700otl.9 for ; Thu, 27 Sep 2018 10:11:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=6wvmMHkNzzithz58a388armFsMi8wlG3nQzemIuMSwI=; b=FYWS6hdItx8TsZsVOrCzAWw6ncJJkOG002obss9uydhFEaaYDSVw4eQIxkZg0qmU4c fmBzLbeiRJr1BGpJezteBRFVFMmEynQwhLE8dOdijmEHOi8OVwUK1OPdM24DbWZN4wLX YGyovocdyxACKeXLuOPfXAi4eyhQhf4WIe4qHzKQpeuL8U6rN+YmuaoaPp7NUl38JdsB i3M2rXEadrcStyUofa05zyfzqlWR9t4H3gVBynqjXzDbYO9k8t7zCFV4aeMrDfsSwH5z OgkhJAuBzk8GyXWXfO9HKLcOnZGIsKB7ZeFsENeqJJ+6sX89jGbU8FwhZWiSuv1iFCph V2rA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=6wvmMHkNzzithz58a388armFsMi8wlG3nQzemIuMSwI=; b=XbZi4RNzGHqBZ6RncMRCok2exZJyuuPJFGSGWccJgWJ2DRMO6mBGyeFkatUaDxQ0KJ xy6Bo5/c4PH0HF23oHbj7ADUm0gQ4Uk3JJ01EqklpoiQWsJZa5+MINlwIqM9zxIkfIBE NQRUJUaPqFB5hvMK7NBaip3R0QCSYET+fpCxm8pBY8So0PW0m+SDSvIGcLdIWeZdIho/ YjSVY0pAENt081l1Cz3tIrTsXA2NA10vIFg1nS1XnMGcJBfuC9/Ha9L8/FGF44Nt0XQS 2gGNc94b2iSpD5sCXYZpiEXzw+Z96Sv2pDnAdEoBROK0mML+uHD1APM/fr/VHk1V4I9t CkHQ== X-Gm-Message-State: ABuFfogZ4YFpnJGhIRtGohekSrWIkcpWdvW2QQgTjsMgzCrvlzTMfoDX PKEYYozg2vyflM0wxbcfI1qSD4HYfLU9/SdmUpmwQg== X-Google-Smtp-Source: ACcGV610QLzzOKPxInSjZk1zRKU6pp+UnfZ5F/gMiAv4gozTYpISO1jh2ZmoL8Vj8LqHHMX+otfJHiQadQS3R9acmGM= X-Received: by 2002:a9d:508e:: with SMTP id b14-v6mr6993649oth.218.1538068272407; Thu, 27 Sep 2018 10:11:12 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: sgaron cse Date: Thu, 27 Sep 2018 13:10:46 -0400 Message-ID: Subject: Re: Realtime get not always returning existing data To: solr-user@lucene.apache.org Content-Type: multipart/alternative; boundary="000000000000046a3d0576dd6d64" --000000000000046a3d0576dd6d64 Content-Type: text/plain; charset="UTF-8" Hey Erick, We're using SOLR 7.3.1, which is not the latest but still not too far back. No the document has not been recently indexed, in fact, I can use the /search API endpoint to find the document. But I need a fast way to find document that have not necessarily been indexed yet so /search is out of the question. Also to put you in context, last time the doc was modified was 3 days ago but we are still seing the occasional doc:null return from the Realtime Get API. Steve On Thu, Sep 27, 2018 at 12:52 PM Erick Erickson wrote: > What version of Solr are you running? Mostly that's for curiosity. > > Is the doc that's not returned something you've recently indexed? > Here's a possible scenario: > You send the doc out to be indexed. The primary forwards the doc to > the followers. Before the follower has a chance to process (but not > commit), you issue a RTG against that doc and it happens to be routed > to a node that hasn't received it from the leader yet. Does this sound > plausible in your scenario? > > Hmmm, I suppose it's not even a requirement that the request gets sent > to a follower, it could easily be "in process" on the leader/primary. > > Best, > Erick > On Wed, Sep 26, 2018 at 11:55 AM sgaron cse wrote: > > > > Hey all, > > > > We're trying to use SOLR for our document store and are facing some > issues > > with the Realtime Get api. Basically, we're doing an api call from > multiple > > endpoint to retrieve configuration data. The document that we are > > retrieving does not change at all but sometimes the API returns a null > > document ({doc:null}). I'd say 99.99% of the time we can retrieve the > > document fine but once in a blue moon we get the null document. The > problem > > is that for us, if SOLR returns null, that means that the document does > not > > exist but because this is a document that should be there it causes all > > sort of problems in our system. > > > > The API I call is the following: > > http://{server_ip}/solr/config/get?id={id}&wt=json&fl=_source_ > > > > As far as I understand reading the documentation, the Realtime Get API > > should get me the document no matter what. Even if the document is not > yet > > committed to the index. > > > > I see no errors whatsoever in the SOLR logs that could help me with this > > problem. in fact there are no error at all. > > > > As for our setup, because we're still in testing phase, we only have two > > SOLR instances running on the same box in cloud mode with replication=1 > > which means that the core that we run the Realtime Get on is only present > > in one of the two instances. Our script randomly chooses which instances > it > > does the query on but as far as I understand, in cloud mode the API call > > should be dispatched automatically to the right instance. > > > > Am I missing anything here? Is it possible that there is a race condition > > in the Realtime Get API that could return null data even if the document > > exist? > > > > Thanks, > > Steve > --000000000000046a3d0576dd6d64--