Responses of two different requests concatinated

Question

We recently got an error report from a user of one of the systems we have developed, showing that the response from a different request had been appended to the original response.

The original response was the front page of a site, generated with jsp, and the appended response was a excel file generated a few seconds earlier.

Our current hypothesis is that some buffer in either httpd(2.2.22) or Tomcat (7.0.35) has been recycled. Httpd is connected to Tomcat using http proxypass.

I have started trying to reproduce the problem, but thought I should try the mailing lists of both httpd and tomcat before continuing.

What I have done thus far is: concatenating html and excel to verify that it is possible to open and looks the way the user experienced; building a custom version of Tomcat that uses the same Processor for each request and configured to use only one thread.

This seems a bit similar to the issue described in http://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.12 (CVE-2011-1475)

Some results when googleling suggests that this could happen when jsp tags are not coded properly, but I have not found any such code in our applications.

Answer 1

Usual culprit is a bug in web application that uses request/response objects outside of their life cycle.

The first step that I'd recommend is to set
org.apache.catalina.connector.RECYCLE_FACADES=true for better security and to ease detection of such misuse.

See
http://tomcat.apache.org/tomcat-7.0-doc/security-howto.html#System_Properties
http://tomcat.apache.org/tomcat-7.0-doc/config/systemprops.html

There is also exists a known issue in Java ImageIO API,
https://wiki.apache.org/tomcat/FAQ/KnownIssues#ImageIOIssues

There also exists CVE-2013-2071 (fixed in 7.0.40).

Answer 2

In order of likelihood:
- app bug
- Tomcat bug
- httpd bug

I'd look for code that retains a reference to the request and/or response object or maybe an OutputStream. The usual cause of this type of issue is retaining a reference across requests and re-using the object from the old request rather than the current one.

Did the appended response include HTTP headers? If yes, this could just be the result of pipe-lining.