Return-Path: Delivered-To: apmail-lucene-solr-user-archive@minotaur.apache.org Received: (qmail 94352 invoked from network); 30 Jan 2009 09:46:18 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 30 Jan 2009 09:46:18 -0000 Received: (qmail 84486 invoked by uid 500); 30 Jan 2009 09:46:15 -0000 Delivered-To: apmail-lucene-solr-user-archive@lucene.apache.org Received: (qmail 84447 invoked by uid 500); 30 Jan 2009 09:46:15 -0000 Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: solr-user@lucene.apache.org Delivered-To: mailing list solr-user@lucene.apache.org Received: (qmail 84436 invoked by uid 99); 30 Jan 2009 09:46:15 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 30 Jan 2009 01:46:15 -0800 X-ASF-Spam-Status: No, hits=3.4 required=10.0 tests=FS_REPLICA,HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of shalinmangar@gmail.com designates 209.85.200.173 as permitted sender) Received: from [209.85.200.173] (HELO wf-out-1314.google.com) (209.85.200.173) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 30 Jan 2009 09:46:08 +0000 Received: by wf-out-1314.google.com with SMTP id 28so357402wfc.20 for ; Fri, 30 Jan 2009 01:45:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=150jpcuuwGfqIm4h+mnfy091ITDTAtGPZ3UG/Vp20DE=; b=FOTOGiur4AABKsWAegJ3NJ16dJyu0rLB2epYxZiYoW4D68sDiQ1PiIAS4od91NomA2 mfD6d8B+AqFco96VfKwGXaKfdWjEY6fCtO/67bBXiQ++FYl9e+Eph3aqa07nCKwGgO2h vqRxxY2vyzGSRYCKhz3kQP4mPnwtAnqDcYps0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=huIH8WPY6C6hsPhXTHvFWL02GAJ1MLNFF9K00aa8hhLXW1OPvyg+coYhOvh5nhL62z EbnQjr/wORlxIPLyO0hjnIyhX0VLhWiRdI2nVY5Ghh5ryXZuAFZHH31NvwXvf8R6dUSm SVGCgkk29BMSTu0eYIiVE/FqxK8oizbr4yMpA= MIME-Version: 1.0 Received: by 10.142.153.8 with SMTP id a8mr447972wfe.152.1233308748605; Fri, 30 Jan 2009 01:45:48 -0800 (PST) In-Reply-To: <8a717ebf0901291057w2912a252md798571f82758512@mail.gmail.com> References: <8a717ebf0901281656w2df7699ch51ed5ca78be546ae@mail.gmail.com> <5e76b0ad0901282020tdb0833ci76db78afecbaf842@mail.gmail.com> <8a717ebf0901291057w2912a252md798571f82758512@mail.gmail.com> Date: Fri, 30 Jan 2009 15:15:48 +0530 Message-ID: <69de18140901300145r4788ee04g125a10262c08492b@mail.gmail.com> Subject: Re: How to handle database replication delay when using DataImportHandler? From: Shalin Shekhar Mangar To: solr-user@lucene.apache.org Content-Type: multipart/alternative; boundary=000e0cd20b20dbe7a80461b01149 X-Virus-Checked: Checked by ClamAV on apache.org --000e0cd20b20dbe7a80461b01149 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On Fri, Jan 30, 2009 at 12:27 AM, Gregg Donovan wrote: > Noble, > > Thanks for the suggestion. The unfortunate thing is that we really don't > know ahead of time what sort of replication delay we're going to encounter > -- it could be one millisecond or it could be one hour. So, we end up > needing to do something like: > > For delta-import run N: > 1. query DB slave for "seconds_behind_master", use this to calculate > Date(N). > 2. query DB slave for records updated since Date(N - 1) > > I see there are plugin points for EventListener classes (onImportStart, > onImportEnd). Would those be the right spot to calculate these dates so > that > I could expose them to my custom function at query time? > Unfortunately, the Context object (which carries the context information and way to pass messages to other components) is not exposed to Evaluator. We should expose this information to be consistent with other DIH components. I've opened an issue to track this at https://issues.apache.org/jira/browse/SOLR-996 -- Regards, Shalin Shekhar Mangar. --000e0cd20b20dbe7a80461b01149--