lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (Issue Comment Edited) (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Edited] (SOLR-2878) Regression in SolrDispatchFilter.java concerning the getOutputStream vs getWriter
Date Sat, 05 Nov 2011 13:42:51 GMT

    [ https://issues.apache.org/jira/browse/SOLR-2878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144687#comment-13144687
] 

Uwe Schindler edited comment on SOLR-2878 at 11/5/11 1:41 PM:
--------------------------------------------------------------

Your class still ignores the charset, you simply have to pass it everywhere (when creating
writers, when creating strings,...). Also you are also mixing bytes with characters at lots
of places in an inconsequent way (out.write should take the bytes, not the string).

If you are simply calculating a hash of the response, why not remove the string handling at
all? Hashing is always done on bytes not characters so all transformations between strings
and bytes are useless and slowdowns everything (+ it introduces bugs). A filter that simply
hashes the output is constructed like that: it buffers the response *bytes* (using ByteArrayOutputStream),
then calculates the hash using MessageDigest (it can also do this during writing), writes
the hash as header, and finally copies the buffered bytes to the output stream. Why transform
from/to strings everywhere?

This code is so chaotic and violates character encodings at lots of places, so I refuse to
correct it. Ask somebody who understands your use-case. Sorry :-(
                
      was (Author: thetaphi):
    Your class still ignores the charset, you simply have to pass it everywhere (when creating
writers, when creating strings,...). Also you are still mixing bytes with characters (out.write
should take the bytes, not the string). If you are simply calculating a hash of the response,
why not remove the string handling at all? Hashing is always done on bytes so all transformations
between strings and bytes are useless and slowdowns all and itroduces bugs. A filter that
simply hashes the output takes buffers the response bytes (using ByteArrayOutputStream), then
calculates the hash using MessageDigest (it can also do this during writing), writes the hash
as header and finally copies the buffered bytes to the output stream. Why transform from/to
strings everywhere?

This code is so chaotic and violates character encodings at lots of places, so I refuse to
correct it. Ask somebody who understands your use-case. Sorry :-(
                  
> Regression in SolrDispatchFilter.java concerning the getOutputStream vs getWriter
> ---------------------------------------------------------------------------------
>
>                 Key: SOLR-2878
>                 URL: https://issues.apache.org/jira/browse/SOLR-2878
>             Project: Solr
>          Issue Type: Bug
>          Components: Build, Response Writers
>    Affects Versions: 3.4
>         Environment: Any unix system, it's a  global problem
>            Reporter: Nick Veenhof
>
> In solr 1.4 we used getWriter in the writeResponse for solrDispatchFilter::doFilter which
invoked writeResponse.
> This code looked in summary like this :
> {code:title=solrDispatchFilter.java|borderStyle=solid}
> private void writeResponse(SolrQueryResponse solrRsp, ServletResponse response, 
>    QueryResponseWriter responseWriter, SolrQueryRequest solrReq, Method reqMethod) 
>    throws IOException {
> ...
>  PrintWriter out = response.getWriter();
>  responseWriter.write(out, solrReq, solrRsp);
> ...
> {code}
> In solr 3.x this has changed to something like this 
> {code:title=solrDispatchFilter.java|borderStyle=solid}
> private void writeResponse(SolrQueryResponse solrRsp, ServletResponse response,
>   QueryResponseWriter responseWriter, SolrQueryRequest solrReq, Method reqMethod) 
>   throws IOException {
> ...
> String charset = ContentStreamBase.getCharsetFromContentType(ct);
> Writer out = (charset == null || charset.equalsIgnoreCase("UTF-8"))
>   ? new OutputStreamWriter(response.getOutputStream(), UTF8)
>   : new OutputStreamWriter(response.getOutputStream(), charset);
> out = new FastWriter(out);
> responseWriter.write(out, solrReq, solrRsp);
> out.flush();
> ...
> {code}
> Now, when we add another filter that tries to modify the output it is being blocked by
the out.flush(). 
> flush() is telling our outputstream that it can write directly to the destination (similar
to the out.close()), since this normally happens automatically there shouldn't be a need to
execute this flush.
> In our case this secondary filter is trying to add headers to the response object. When
we were using getwriter() it was not closing the writer so we could still modify this output.
Since the flush happens now we are no longer able to modify the headers accordingly. 
> It would be an easy fix if the flush could be commented out and everything would work
but that is not the case. The headers are working when this happens but there is no more output.
> When I modify both classes to use getWriter() everything is working as expected.
> This is a severe regression for our use of solr.
> Our code that is used in the filter
> {code:title=solrCustomFilter.java|borderStyle=solid}
> public void doFilter(ServletRequest req, ServletResponse res,FilterChain chain) 
>   throws IOException, ServletException {
> ...
> Writer out = new OutputStreamWriter(response.getOutputStream(), "UTF8"); //auto flush
> out = new FastWriter(out);
> // convert to a chartext
> CharResponseWrapper wrapper = new CharResponseWrapper((HttpServletResponse) response);
> chain.doFilter(request, wrapper);
> String responseBody = wrapper.toString();
> //write the outgoing header. Only succeeds when flush of solrDispatchFilter is commented
out
> response.addHeader("pragma", "somevalue;");
> out.write(responseBody);
> ...
> {code}
> Sources :
> {quote}
> SRV.5.5 Closure of Response Object
> When a response is closed, the container must immediately flush all remaining content
in the response buffer to the client. The following events indicate that the servlet has satisfied
the request and that the response object is to be closed:
> • The termination of the service method of the servlet. 
> • The amount of content specified in the setContentLength method of the response has
been written to the response. 
> • The sendError method is called. 
> • The sendRedirect method is called.
> {quote}
> Solr 1.4 https://svn.apache.org/repos/asf/lucene/solr/branches/branch-1.4/src/webapp/src/org/apache/solr/servlet/SolrDispatchFilter.java
> Solr 3.4 https://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_3_4/solr/core/src/java/org/apache/solr/servlet/SolrDispatchFilter.java

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message