Return-Path: X-Original-To: apmail-lucene-solr-user-archive@minotaur.apache.org Delivered-To: apmail-lucene-solr-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0A95B18FBA for ; Thu, 11 Feb 2016 18:14:11 +0000 (UTC) Received: (qmail 73948 invoked by uid 500); 11 Feb 2016 18:14:01 -0000 Delivered-To: apmail-lucene-solr-user-archive@lucene.apache.org Received: (qmail 73882 invoked by uid 500); 11 Feb 2016 18:14:01 -0000 Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: solr-user@lucene.apache.org Delivered-To: mailing list solr-user@lucene.apache.org Received: (qmail 73870 invoked by uid 99); 11 Feb 2016 18:14:01 -0000 Received: from Unknown (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 11 Feb 2016 18:14:01 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id D20761804A9 for ; Thu, 11 Feb 2016 18:14:00 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.802 X-Spam-Level: X-Spam-Status: No, score=-0.802 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id 87fiYR0Q054E for ; Thu, 11 Feb 2016 18:13:58 +0000 (UTC) Received: from mail-io0-f169.google.com (mail-io0-f169.google.com [209.85.223.169]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTPS id ABC8620CB8 for ; Thu, 11 Feb 2016 18:13:57 +0000 (UTC) Received: by mail-io0-f169.google.com with SMTP id l127so63122386iof.3 for ; Thu, 11 Feb 2016 10:13:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=+MFNaimWha/9n1+A9wU5NevgPAAhcpXujhTNEC4rI6o=; b=njMYFR6OSuc4TJxZFMNaqAa0j3ApNrrmU941QERk8oO0boGm5cVuXuYDGoFQ7/cbi9 5+fdrJj5YVQa7Ud4jKOwnU0FFeJZtdThnju2pBRfczNlPlOcdOqC1/6ck0pWDgZ+/jiW dB2PyMxYg3f/u769FeXp3Vs9XEXrqf2/42uiBxWoavUbuUv6IXfW8JWzwl18na/aO0+F Smzp8XZIIQTvJlOMNu0vtcqB7aWmhMR02kgt9DxEk6nxzuV54HSGwC5vKVaSiiTjhXOR lDMGSPXRABPAnuSkWOK/zULze/B2WaDNtsJz9VgfspvxXh4hR3CLibWoPYXSi5i+gCsS oeXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=+MFNaimWha/9n1+A9wU5NevgPAAhcpXujhTNEC4rI6o=; b=Wr7EEdVBNWzYZuG7MDBjO4nninQ+o0+be2xM48CYHVUo0JC6GxehKS2tTOggKzlp2X Qsn7iov0zu8RB5/kmQ/v1n7oUdqW+x3OF5j4lj50WXJ3wTQHw5ClnPI4rQi3Wxq+MIyC Kk7qcpeDM51i3AukparfeH7j6ZGGGRKDSmI4qIQM/bq/d6M5nMrBs85cgvArlnW0xtgR /zvLPqEQF3cF5hGXtzpkli7yy6z6r4ZfN8vH3/IWVnJJzrBzJRU1eoGehzDBmwPZd05x nQIJh5S20U8lDipwvSF9hTVbQboju7jIi7jg0ibBoMlgOcRngmM1uK0w2HOJvi7fEE9E Th8w== X-Gm-Message-State: AG10YOT+gzOV9NIdXwrrALi9LBTzuOUWyatVKP3PrjriQSij+3QmOXca7Vi7/SzOad9qPVyN0KqfMg+uJp8j0Q== MIME-Version: 1.0 X-Received: by 10.107.30.144 with SMTP id e138mr42966666ioe.158.1455214436755; Thu, 11 Feb 2016 10:13:56 -0800 (PST) Received: by 10.107.53.130 with HTTP; Thu, 11 Feb 2016 10:13:56 -0800 (PST) In-Reply-To: References: Date: Thu, 11 Feb 2016 10:13:56 -0800 Message-ID: Subject: Re: Tune Data Import Handler to retrieve maximum records From: Erick Erickson To: solr-user Content-Type: text/plain; charset=UTF-8 It's possible with JDBC settings (see the specific ones for your drive), but dangerous. What if the number of rows is 1B or something? You'll blow Solr's memory out of the water.... Best, Erick On Wed, Feb 10, 2016 at 12:45 PM, Troy Edwards wrote: > Is it possible for the Data Import Handler to bring in maximum number of > records depending on available resources? If so, how should it be > configured? > > Thanks,