http request (no only session) replication in cluster

Question

What can be done to guarantee failover in below scenario:

2 tomcats behind cisco loadbalancer
1 http request can last very long about 50 seconds - response from webservice can take so long
load is 200 requests per second
I must response in max 4 seconds more than backing webservice

is there something like http request replication ?

50 s * 200 req/s = 10.000 pending requests

if one tomcat is eg killed, can in any way other tomcat serve his requests ?
Is there any out of the box solution, eg similar to session replication ?

Answer 1

The best way to do this is to configure your load balancer to buffer responses and re-try another cluster node in the case of an unexpected disconnect.

If you can't buffer the response, then it is entirely inappropriate to re-process a request: instead, you should let the failure propagate all the way back to the client and let them decide what to do.

Comments

Is that just me, or does this look like a *massive* imbalance between the load, and the  resources dedicated to serve that load ?
I somehow have trouble to envision any system working in any stable way, when right from  the start it is assumed to have 10,000 requests simultaneously being in various stages of processing. Unless one would have some Google-like server farms behind the thing anyway.