Cloud performance testing is a major trend in many fields nowadays, and it’s a good example of successfully leveraging the potential of cloud-based solutions in unique ways. It’s an approach used by companies of different sizes, and so far it’s been showing great results across the board. However, it’s not the “magic wand” solution that some seem to see it as. Like any other testing approach, cloud performance testing requires some points to be addressed with a priority, and it’s also recommended to avoid using certain procedures that you normally would in other contexts.
Do: Try to Be as Close to Reality as Possible
There are likely some observable patterns in the behavior of your users, and your cloud performance testing should try to replicate those as closely as possible. For example, don’t just go for an even geographical distribution – it may be the most straightforward option, but there’s also a good chance that it’s quite detached from the reality of your situation. If you don’t have the data to make a viable decision on that front, it may be better to wait until you’ve collected some additional information about your operations. Otherwise, you could waste a lot of money on testing that ultimately doesn’t reveal anything valuable.
Do: Experiment with Different Cloud Providers
Different cloud providers can provide you with completely different services in this market, and it’s important to work with at least a couple of companies to ensure that you have good coverage over as many parameters as possible. The MQL5 Cloud Network is a good example, having recently reached a capacity of 34,000 cores and growing. The network has a prominent place in the current market, with a long track record of usage for historical data backtests of automated trading solutions.
This network is used for backtesting trading strategies by developers presenting their ready-made trading solutions at MQL5 Market. The company behind it is also constantly putting effort into improving its service and keeping it relevant to the current market.
StormForge is another notable player on this market, with a network optimized for enterprise-grade performance testing and various optimization modes. They are specifically focused on risk testing for real-world scenarios and are best used in combination with another provider that covers your other fronts.
Serious service providers like this should be your main target here, although it could also be worthwhile to get in touch with at least one smaller provider on the side. This can offer you some additional flexibility in how you handle your tests and can give you some extra resources when you need to stretch your testing just a little bit.
Do: Track Everything in a Centralized Location
Always ensure that you have a centralized place for tracking your progress, comparing results, and evaluating how well things are going. You must be able to measure the outcome of each test run and align that data with other samples as frequently as possible. It will take a while to gather enough information to draw some viable conclusions from, but it will play a huge role in ensuring that you’re in control of everything. Later on, you’ll definitely appreciate having taken the time to track your results in a more organized manner, especially as you start to ramp up your testing and generate even more data to work with.
Don’t: Pollute Test Results by Running Load Generators on the Same Infrastructure
A common mistake is to use the same cloud provider where your services are running for your load tests. The implications of this might be obvious to you, but they aren’t to everybody. Especially when using a large-scale service that can normally handle heavy loads without issues. You wouldn’t expect testing performance to be affected at all in this case, but you’d be wrong. Because of this, it’s important to ensure that your load generators are as isolated as possible from the rest of your infrastructure and that any potential contamination of the results is accounted for in your performance tracker accordingly.
Don’t: Forget Local Load Testing
Running performance testing through cloud solutions is a great approach, but it shouldn’t be the only one you use. You should still not forget the benefits of local testing, especially as your service continues to grow and gets exposed to more fronts that could generate potential issues. It’s important to maintain active testing on all levels, and in fact, your local load testing should probably get ramped up at some point when your services start to grow.
Conclusion
In the end, cloud performance testing brings a lot to the table. But like any other tool, it won’t do you much good if you don’t use it properly. It’s crucial to measure your results in an objective manner, have a diverse network of cloud service providers to work with, and always ensure that you’re configuring your tests to be as close to reality as possible. In many cases, you’ll need to work with more than one provider to ensure proper coverage for all types of faults you want to simulate, making it even more important to start your research early on.
