Friday, June 19, 2015

Burp and TCP Connection Reuse / TCP Streaming

Recently we were working on an engagement to test a fat client using a web service and ran into a problem with Burp. Surprisingly enough, there was not a single resource on the Internet to help us out. Hoping that others dealing with the same issue won't lose their sanity like we almost did, I am writing this blog post now ;)

We were trying to test a web application, or rather a client application (a binary!) communicating with a web service built on top of an HTTP REST API, with Burp as a transparent/invisible proxy in between. For some completely unknown reason, when Burp was between the client software and the server, the client application just refused to log in.

A comparison of HTTP requests and responses showed no difference at all (except for session cookies set by the server during Step 2 of a 3-step login process). In other words: On application layer (i.e. HTTP), packets and communication were completely identical between client and server, both when connecting through Burp or directly through the Internet. Still, logging in did not work when Burp was active even though HTTP requests and responses were identical.

By accident, when following TCP streams in Wireshark while debugging this problem, we discovered the reason: TCP connection reuse. HTTP 1.1 allows for one TCP connection to be reused for multiple HTTP requests and responses. Burp, however, follows a Store, Modify and Forward model which initiates a new TCP connection for each proxied HTTP request. In other words: A different connection model than what the client application assumed for communication with the server, which assumed one TCP connection for three requests and responses.

The workaround for the problem was activating Streaming Responses in Burp for the target host. Streaming Responses were introduced for a different use case, but still solved our problem. Cf. the following screenshot:


So, in case you ever bump into problems with Burp as a transparent/invisible proxy and can't figure out why, have a look if the application is reusing one TCP connection for many HTTP requests and try activating Streaming Responses in Burp. This allows Burp to deal with TCP connection reuse for multiple HTTP requests/responses.

That's it folks!