manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karl Wright (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CONNECTORS-1408) Request-URI Too Long
Date Sat, 15 Apr 2017 17:27:41 GMT

    [ https://issues.apache.org/jira/browse/CONNECTORS-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970031#comment-15970031
] 

Karl Wright commented on CONNECTORS-1408:
-----------------------------------------

I tried creating a test environment here that would reproduce the issue, but so far I'm having
no luck turning on wire debugging when running tests; not sure why.

I can download and run Solr for real but I've got other things I need to attend to so it may
be a day or two before that can be done.

It's possible that the problem is not in SolrJ but instead in HttpClient.  The SolrJ code
for a non-multipart form would be something different than "text/plain":

{code}
        if (!isMultipart) {
          postOrPut.addHeader("Content-Type",
              "application/x-www-form-urlencoded; charset=UTF-8");
        }
{code}

Parameters from request.getParams() are all entered in the multipart form here:

{code}
        List<FormBodyPart> parts = new LinkedList<>();
        Iterator<String> iter = wparams.getParameterNamesIterator();
        while (iter.hasNext()) {
          String p = iter.next();
          String[] vals = wparams.getParams(p);
          if (vals != null) {
            for (String v : vals) {
              if (isMultipart) {
                parts.add(new FormBodyPart(p, new StringBody(v, StandardCharsets.UTF_8)));
              } else {
                postOrPutParams.add(new BasicNameValuePair(p, v));
              }
            }
          }
        }
{code}

We should see a List of FormBodyPart's being created and filled.  The ModifiedMultipartEntity
is then created with all the parts:

{code}
        if (parts.size() > 0) {
          ModifiedMultipartEntity entity = new ModifiedMultipartEntity(HttpMultipartMode.STRICT,
null, StandardCharsets.UTF_8);
          //MultipartEntity entity = new MultipartEntity(HttpMultipartMode.STRICT);
          for (FormBodyPart p : parts) {
            entity.addPart(p);
          }
          postOrPut.setEntity(entity);
        } else {
          //not using multipart
          postOrPut.setEntity(new UrlEncodedFormEntity(postOrPutParams, StandardCharsets.UTF_8));
        }
{code}

... which should then generate a multipart form when transmitted, including setting the ContentType
to the appropriate value:

{code}
    protected String generateContentType(
            final String boundary,
            final Charset charset) {
        StringBuilder buffer = new StringBuilder();
        buffer.append("multipart/form-data; boundary=");
        buffer.append(boundary);
        if (charset != null) {
            buffer.append("; charset=");
            buffer.append(charset.name());
        }
        return buffer.toString();
    }
{code}

Somewhere along this chain of logic multipart form POST is not happening anymore.  Let's figure
out where.  Anything you can do to narrow this down is greatly appreciated.



> Request-URI Too Long
> --------------------
>
>                 Key: CONNECTORS-1408
>                 URL: https://issues.apache.org/jira/browse/CONNECTORS-1408
>             Project: ManifoldCF
>          Issue Type: Bug
>          Components: Email connector, Solr 6.x component
>    Affects Versions: ManifoldCF 2.6
>            Reporter: Cihad Guzel
>            Assignee: Karl Wright
>             Fix For: ManifoldCF 2.8
>
>         Attachments: http-wire2.log, http-wire.log
>
>
> I run email connector job and follow "Simple History" from UI. I see an error as follow:
> {code}
> Error from server at http://localhost:8983/solr/mycore: non ok status: 414, message:Request-URI
Too Long
> {code}
> It is sent by Solr. 
> Solr logs say: 
> {code}
> HttpParser - URI is too large >8192
> {code}
> and 
> {code}
> HttpParser - bad HTTP parsed: 414 for HttpChannelOverHttp@2b6931dd{r=0,&#8203;c=false,&#8203;a=IDLE,&#8203;uri=null}

> {code}
> ManifoldCF ModifiedHttpSolrClient.java has following code:
> {code}
>  // It is has one stream, it is the post body, put the params in the URL
>       else {
>         String pstr = toQueryString(wparams, false);
>         HttpEntityEnclosingRequestBase postOrPut = SolrRequest.METHOD.POST == request.getMethod()
?
>             new HttpPost(url + pstr) : new HttpPut(url + pstr);
> {code}
> There is "pstr" field appended to the URL. "pstr" field have all Solr params. It contains
email content. We have "URI is too large" error when email has large content.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message