A Match Made in Heaven – Transactional Systems and Erlang/Elixir
- Lee Sigauke
- 8th Feb 2024
- 5 min of reading time
Transactional systems implemented with Domain Driven Design are not largely complex; they do, however, face a very critical challenge: managing an influx of real-time data while maintaining system reliability and responsiveness. The core issue lies in organising and updating vast amounts of data in real-time coupled with handling intermittent but significant spikes in user traffic. The stakes are high; any delay in updating any information could prove disastrous in that it could cost both credibility and revenue.
In response to these demands, Elixir stands out as a strategic solution due to its exceptional capabilities in managing high-volume, real-time data processing courtesy of the Erlang virtual machine – BEAM. Stale data is not just costly but also exploitable by end-users which underscores the importance of a system’s capability to scale effortlessly and handle sudden intense loads without compromising performance or reliability – qualities entrenched in Elixir since day one.
Elixir, rooted in the robust framework of Erlang, was originally designed for developing telephony applications which demand swift responses within milliseconds, consistently and reliably. This characteristic perfectly aligns with the demands of transactional systems where instantaneous updates during pivotal moments are crucial. Elixir’s natural ability to manage these constant updates while maintaining minimal latency becomes a critical asset in such an environment. The Grand National, an event notorious for overwhelming bookmakers every year, is a perfect example where Elixir’s real-time responsiveness can shine due to the avalanche of transactions occurring simultaneously in a very small window.
Handling such monumental traffic volumes presents a significant challenge in sports betting systems. High-profile events produce an overwhelming amount of transactions which necessitates a system capable of managing these surges without falling to its knees. Enter Elixir/Erlang, both of which are well known for their proficiency in handling surges without flinching. Discord, the instant messaging giant, exemplifies Elixir’s ability to handle escalating demands as they were successful in scaling to accommodate 5 million concurrent users with millions of events per second, vividly showcasing Elixir’s prowess.
Elixir/Erlang was built from a foundation designed for concurrency, simplifying the creation of concurrent systems with a fundamental emphasis on isolation, and fault tolerance. Its architecture revolves around processes that operate independently and communicate solely via message passing without sharing state. This feature ensures that processes will not interfere with one another, enabling individual processes to be monitored and revived in the event of failure.
In the context of a transactional system, having a single process to manage each user interaction means any issues with one process remains contained and does not affect the rest, therefore, the system keeps running smoothly. This approach prevents the unfortunate situation where a solitary user’s problem could otherwise impact the entire platform, thereby preserving user trust and system integrity amid surges in usage.
The phrase “let it crash” is very common among Elixir/Erlang developers, it’s not to disregard the significance of system crashes or errors; rather, it signifies the resilience of the system. Crashes or errors are often confined to within their respective processes, thereby averting a domino effect that could bring down the entire system. Recovery from these isolated incidents is often as straightforward as restarting the affected processes and that might be all that is required to get the processes up and running again, there are several strategies for managing how process failures are handled so there is some flexibility afforded to developers in that realm.
The concurrency model implemented by Erlang/Elixir allows for seamless vertical scaling to accommodate escalating demands without compromising service quality. As the resource requirements increase within a node, the system spawns more processes which ensures a consistent and reliable service even during rapidly increasing user loads. The scalability has been validated by the industry giant Bet365 as they were able to seamlessly increase users supported on a single node from tens to hundreds of thousands.
Bleacher Report, the second largest sports website in the world, is another success story as they were able to handle 8 times their normal traffic without autoscaling, all the while using 8 serves in comparison to the 150 they were using before they used Elixir.
While concurrency isn’t unique to Elixir and Erlang, their strength lies in leveraging the power of the BEAM virtual machine – a time-tested, battle-hardened system designed explicitly for concurrent, fault-tolerant, and real-time applications. As surely as the sun rises, the synergy between Elixir/Erlang and the BEAM virtual machine embodies a level of reliability and resilience that will not falter.
In conclusion, companies that require highly transactional systems call for a platform that is capable of managing extreme data loads, ensuring constant responsiveness, and remaining resilient in the face of unpredictability. Erlang and Elixir, through their unique strengths and support from the BEAM, stand out as the ideal solution, not just for sports betting but for any industry facing similar demands for reliability, scalability, and real-time processing.
Pawel Chrząszcz introduces MongooseIM 6.3.0 with Prometheus monitoring and CockroachDB support for greater scalability and flexibility.
Here's how machine learning drives business efficiency, from customer insights to fraud detection, powering smarter, faster decisions.
Phuong Van explores Phoenix LiveView implementation, covering data migration, UI development, and team collaboration from concept to production.