Exchanges are high liability systems, with millions of dollars’ worth of orders and user funds at stake at any given moment. As a result, exchanges are required to achieve consistent performance, even during busy trading periods. But what does exchange performance actually look like, and what does it mean? We explore the concept of exchange performance and measurement here.

We can measure exchange performance via two key metrics. The first one of these is how many requests per second an exchange’s system can handle effectively, this is known as ‘throughput’. Secondly, we can examine how much time is required for a single request to complete on the exchange, a metric known as ‘latency’.

The distinction between these two metrics is very important. For example, imagine an exchange collects 60 million requests per minute, and responds to everyone at the same time. This would yield what may seem to be an impressive throughput, at one million requests per second, but latency would be a poor one minute per request. So, let’s dive in and examine how we might properly measure exchange performance.

How is Performance Measured?

Exchanges have multiple components, and each component has its own unique performance characteristics. To assess exchange performance, we need to know how performance was measured, and also what component was measured.

Therefore, in order to be trustworthy, performance data should specify the following parameters:

Hardware – What hardware is the exchange running on?
Databases – How is the exchange using queues and databases in their setup?
Load Type – A mixture of limit and market orders from many users may produce different results than just limit orders from a single user, this should be accounted for.
Requests per second (rps) – If an exchange can process 10,000 rps, then sending 20,000 rps will overload the system and affect performance.
Percentiles – The 99th, 95th and 90th percentiles indicate how long it takes to complete 99, 95 and 90% of exchange requests respectively.
Throughput – The average rps during the test.

Unfortunately, exchange software vendors seldom disclose such detailed data, despite claiming that their exchange infrastructure can handle millions of rps without giving relevant context. This incomplete picture of exchange performance often means reported data isn’t trustworthy, and can’t be compared alongside other metrics of exchange performance.

Realistic Exchange Performance

So, with that said, what are some realistic figures for exchange performance? The best order matching engines can process around one million rps, taking just a few hundred nanoseconds to handle a single order. This level of performance requires careful tuning of specific hardware and the use of highly specialized programming languages, which is a difficult and costly process.

However, most exchanges don’t require such high performance. The Naive order matcher can handle hundred of thousands of orders per second, which is perfectly suited to most applications.

In respect to order processing times, within the 98th percentile, a 10 – 20 ms latency can be considered a very good time. Other trading systems may feature microsecond latency, but their systems are often far simpler than a full exchange. Much of the processing time within an exchange system is occupied by the network, because the API first has to pass the users order to an order matcher, and then wait to receive a response.

Cloud exchange hosting offers cheap hardware and services but is costly for performance. Virtualisation, docker containers, and software-defined networks will all contribute to increased latency times. Likewise, cloud hosting does not allow modification of hardware to vastly increase processing speed. As a result, a 98th percentile latency of 20 – 40 ms can be considered good for cloud hosting.

Some top crypto exchanges have latencies of more than 100 ms to process a single order. This is not because they have very slow engines, but rather that they often feature a web application firewall in front of them. In this case, it is the HTTPS and other factors unrelated to the exchange which slows performance.

Performance Requirements

What level of performance does an exchange need? Let’s examine a smaller exchange. Our exchange runs 10 major crypto pairings, and perhaps 20 pairs with lesser known cryptocurrencies. Only the 10 major pairs have market-maker bots, which generate 10 rps each. This is a realistic scenario for many existing exchanges and brokerages, who receive around 100 rps from market-makers and few other requests from normal traders. Actual traders do not tend to generate much load in the form of other order requests.

Although 100 rps does not sound particularly high, it is already at the point where it’s impractical to use a database for every request. Therefore, some form of caching must be implemented.

With exchanges this is particularly hard, because data inside these caches is volatile and changes with each order. This is why exchanges do not use existing technologies oriented towards throughput, instead opting to use their own tailored performance solutions.

Summary

The key exchange performance takeaways are that when assessing exchange performance, latency is as important as rps, which can make claims of ‘millions of orders per second’ from many vendors a meaningless and misleading statement without the proper context.

Instead, when discussing performance, or seeking information from vendors, it’s necessary to be specific about exactly which metrics were measured, and their relationship to each other.

For exchange operators whose exchange engines can handle single requests in <40 ms, this is already a good result, and even after factoring in HTTPS, load balancers and other effects, if the latency is still around 100 ms, this is perfectly acceptable performance.