lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tobias Rübner <...@tobr.eu>
Subject Re: maximum rows for dataimport using solrj
Date Wed, 07 Sep 2011 13:18:17 GMT
Thanks Erik,
but I want to change the whole configuration to e.g. connect to an other
database.
So there is also the need to add/remove (sub-)entities to resolve database
relations.

tob


On Wed, Sep 7, 2011 at 3:04 PM, Erick Erickson <erickerickson@gmail.com>wrote:

> Hossman just posted this, does it help?
>
> You should be able to paramaterize hte table name in the SQL using a
> request parameter...
>
> https://wiki.apache.org/solr/DataImportHandler#Accessing_request_parameters
>
>
> Best
> Erick
>
> On Wed, Sep 7, 2011 at 9:00 AM, Tobias Rübner <dev@tobr.eu> wrote:
> > Thanks for your quick response.
> > The reason why I am using the debug mode is to dynamically add a data
> config
> > with my request.
> > I want to create a new core and add a new configuration to index a new
> > database with the DIH, without changing the solrconfig.xml.
> >
> > Do you see any other options?
> > tob
> >
> >
> >
> >
> >
> > On Wed, Sep 7, 2011 at 12:15 PM, Shalin Shekhar Mangar <
> > shalinmangar@gmail.com> wrote:
> >
> >> On Wed, Sep 7, 2011 at 1:45 PM, Tobias Rübner <dev@tobr.eu> wrote:
> >>
> >> > Hi,
> >> >
> >> > I want to trigger the data import handler remotly using the solrj api.
> >> > So I added a dih request handler to my solr config and tried to call
> the
> >> > handler as described here
> >> > http://wiki.apache.org/solr/SolJava
> >> >
> >> > ModifiableSolrParams params = new ModifiableSolrParams();
> >> > params.set("qt", "/dataimport");
> >> > params.set("command", "full-import");
> >> > params.set("commit", "true");
> >> > params.set("debug", "true");
> >> >
> >> > solrServer.query(params);
> >> >
> >> >
> >> > That worked fine, but it only added the first 10 rows.
> >> > So I added a rows param
> >> > params.set("rows", 50000000);
> >> >
> >> > which only allows 50M as maximum value.
> >> >
> >> > Having databases with more records, this wouldn't index all entries.
> >> >
> >> > Do you hav any suggestions how to change this?
> >> >
> >> >
> >> You are sending debug=true as a request parameter. The debug mode limits
> >> the
> >> number of rows to 10 by default and it also forces indexing to be
> >> synchronous with the request.
> >>
> >> Just remove debug=true from the params and DIH will index all entries.
> >>
> >> --
> >> Regards,
> >> Shalin Shekhar Mangar.
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message