A Modern and Elastic Design for the Burp Collaborator Server | Blog
When we launched Burp Collaborator in 2015, PortSwigger deployed a public Collaborator server that anyone could use. This meant that OAST testing with Burp Collaborator could work immediately, without any configuration. The automated OAST, in particular, was a game-changer – bringing many hard-to-find vulnerabilities to the masses for the very first time.
As you can see from the diagram below, all Burp Collaborator servers currently follow a monolith-like model – storing data in memory (RAM):
The problem with this is that it’s not exactly scalable. Since 2015, Burp Suite Professional’s user base has grown by an order of magnitude (from around 8,000 in mid-2015 to over 60,000 in early 2022), putting increased pressure on the Burp Collaborator public server.
To compound this situation, in 2018 we introduced Burp Suite Enterprise Edition, which enables scalable and automated analysis of entire web portfolios. Since Burp Suite Enterprise Edition uses the same Burp Scanner (and Collaborator) found in Burp Suite Professional – and at the time of writing is used by 800 organizations – this adds significantly to the work done by Burp Collaborator’s public server.
Then consider that these pressures have only become more pronounced with the advent of the recent Log4Shell vulnerability found in the Log4j library. OAST is great at detecting this type of vulnerability – so naturally many of you use tools like Log4Shell Scanner to check for this. Again, this seems to have had a marked effect on the amount of traffic to the public Collaborator server.
Essentially, the Burp Collaborator server had become a victim of its own success. The time had come for an update – and luckily we had been working on it for a little while.
We have now revamped the Burp Collaborator public instance with a new design. This is cloud-native, elastic, and will greatly increase resiliency. While users shouldn’t notice a functional difference, it will allow us to continue to provide you with a reliable public collaboration server – at no additional cost on top of your Burp Suite subscription fee.
As you can see from the diagram below, we have introduced a number of changes in the new architecture. These changes enable multiple instances of Collaborator services, running behind load balancers. This change required the addition of a database – which we implemented using Redis. The new architecture dramatically increases horizontal scalability, while ensuring that we can upgrade the Burp Collaborator public server without worrying about downtime or loss of user data:
During the reorganization, we took the opportunity to design a system that provides employee data with an enhanced level of protection while it is at rest. Consider the scheme below. Here you can see how all interaction data generated by Collaborator is encrypted before being stored. A user secret and a master secret are then required to decrypt this data:
We have always taken the security of Collaborator data very seriously, and as such, Burp Collaborator has never stored any data that could be used to identify an individual Burp Suite user (such as an account name or license key ). This is still very much the case.
In order to decrypt all interaction data, the Collaborator query service requires both the user secret (provided to it by a user’s Burp Suite instance) and the master secret. Once your data is returned, it is immediately deleted from the database – and if an instance of Burp Suite does not request stored interaction data within 14 days of its creation, that data will be deleted from the database as obsolete.
Burp Collaborator Private Servers
Of course, the option of deploying a private Burp Collaborator server still exists. This is especially useful for those with large scale or particularly sensitive use cases, where full control of the Collaborator server is required.
Currently, private Collaborator servers all use the original monolithic type model, but we also plan to open the new elastic model for use on private Collaborator servers. If you have a relatively large use case for Burp Collaborator at the service provider level and would like to help test this functionality, we would love to hear from you. Send us an email, and we’ll let you know.