The story behind the hotel booking engine test

Editorial review: 8 October 2026. This historical case account retains the originally reported project figures. Statistical language and references to the benchmark have been clarified. The original publication date is retained.

Beginning: Why run the test?

Like many recent digital stories, this one starts with “We were once in a meeting.” But, as with all stories, the interesting part is not the beginning, but the challenge and the outcome. About the beginning, we can say that it has been one of the most ordinary of the 21st century, with four people in an office trying to find solutions. One morning, years ago, we went to meet a customer (a chain of hotels), who had been our client for two years at that point.
We were in charge of strategy in digital media and analysis. Additionally we collaborated with them in the process of selecting a booking engine, a process that was being lengthened by doubts about which was the best booking engine for them among all those we had evaluated.

The challenge: Which is the best booking engine for us?

One of our services is to help hotels and hotel chains select the booking engine that best suits their needs. At the time described in this article, our benchmark covered 80 booking engines and before recommending and implementing a booking engine, we perform an analysis of the hotel’s specific needs at the level of rate plans, prices, offers, sales channels, markets, connectivity with channel manager and PMS, typology of users with access to the booking engine, etc. For that assessment, we compared these needs with more than 200 functionalities of each booking engine, and based on this information and our experience we recommend the booking engine that we consider appropriate in each case.

In this case we had already spent more than half a year on an in-depth analysis of the pros and cons of the three booking engines that we had finally chosen as the most suitable for them, because they had the functionalities they needed, adequate costs and in all cases they were booking engines that offered high conversion rates based on our experience.

That morning, at that meeting, with all the analyses completed, we realized that small variations in the conversion rate tipped the balance between the booking engines at the ROI level, but it is a fact that we didn’t know what conversion rate we would achieve in this specific implementation with each booking engine. For a customer like this, a large chain, a small variation in the conversion rate could mean a difference of more than one million euros a year.

That’s when a person on the team commented: “Why don’t we do a test like we do in campaigns to optimize the conversion rate?” Of course it was the best solution, so we could determine empirically in this case which of the three engines offered better results and a higher return on investment (ROI).

Outcome: A positive result.

In that case we designed a complete test. The 3 booking engines were implemented on the client’s site and the one they had up to that point was maintained and in total we tested 4 booking engines for 3 months. At the end of the test, at least 1,000 bookings had been received in each booking engine, providing data for comparing conversion and ROI in that particular project. A booking count alone does not establish statistical reliability: allocation, duration, uncertainty and the definition of the outcome also matter. The reported case does not guarantee the same result for another hotel.
The customer was fully satisfied and confident of having made the best choice. At the time of the original publication, we continued to work with them and were proposing another booking-engine test.