lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nathan Friend <nathan.fri...@Maximuscanada.ca>
Subject RE: [Marketing Mail] Re: Request return behavior when trigger DataImportHandler updates
Date Fri, 01 Dec 2017 19:28:26 GMT
Thanks for the reply, Shawn.  One additional detail I should have mentioned - in both scenarios,
I'm making a POST request, not a GET.  Even when using a POST, Solr seems to be able to accept
URL query parameters.

I'm making test requests using Postman (https://www.getpostman.com/), so it's also possible
this issue is related to how Postman is sending the request (although I doubt it).

Either way, it sounds like relying on the synchronous behavior of the HTTP request is not
a good option.  I've been looking into the "onImportEnd" event listener functionality and
think this feature will provide a more robust way of generating notifications when imports
have finished.

Thanks!

Nathan

-----Original Message-----
From: Shawn Heisey [mailto:apache@elyograg.org] 
Sent: December 1, 2017 3:04 PM
To: solr-user@lucene.apache.org
Subject: [Marketing Mail] Re: Request return behavior when trigger DataImportHandler updates

On 12/1/2017 9:37 AM, Nathan Friend wrote:
> When triggering the DataImportHandler to update via an HTTP request, I've noticed the
handler behaves differently depending on how I specify the request parameters.
>
> If I use query parameters, e.g. http://localhost:8983/solr/mycore/dataimport?command=full-import,
the HTTP request is not fulfilled until the import is completed.
>
> However, if I instead make a request to http://localhost:8983/solr/mycore/dataimport
with a form-data key of "command"="full-import", the HTTP request returns immediately. In
this case, the documentation seems to suggest the best way to detect when the import has completed
is to poll Solr for status updates until the status switches from "busy" back to "idle".

At one time, way back in version 1.4 and 3.2, I was making such requests from a Perl program,
and they were definitely GET requests.  These requests did NOT wait until the import completed.

I just checked my most recent program, which uses SolrJ, and I discovered that I am using
Method.POST there, so I can't confirm or deny what you're seeing yet.  At one point, I went
through all my SolrJ code and made sure all requests were using POST, so that I would never
run into the limitations on the URL size.  It wasn't really necessary for my code that starts
DIH, but I was making the change everywhere.

> Is this difference in behavior intentional?  I would like to be able to rely on the first
behavior - to expect that the import will be finished when the HTTP call returns.  However,
I can't seem to find any documentation on this difference; I'd prefer not to have my component
rely on what might be a convenient bug :).

As far as I know, Solr should be unaware of any difference between these ways of supplying
the parameters, because the servlet container (usually
Jetty) should just make parameters available to Solr, no matter how it received them.  I
think that the dataimport handler has always been async in nature.  I've NEVER seen an import
command wait to respond until the import is complete.  My imports take many hours to run. 
If that's what you're seeing happen with a GET request, I'd call that a bug.  We need to
confirm the problem in the most recent versions before filing an issue.

Thanks,
Shawn


Confidentiality Notice:
This electronic transmission, and any documents attached to it, may contain confidential information
belonging to the sender. This information is intended solely for the use of the individual
or entity named above. If you are not the intended recipient, you are hereby notified that
any disclosure, copying, distribution or the taking of any action in reliance upon the contents
of this information is prohibited. If you have received this transmission in error, please
notify the sender immediately and delete the message and all documents.
Mime
View raw message