Twenty Years of Open Source Erlang

Erlang was released as Open Source Tuesday December 8th, 1998. Do you remember where you were that week? I was in Dallas (Texas); one of my many visits helping the Ericsson US branch set up an Erlang team working with the AXD301 switch. Waking up on Tuesday morning, I got the news.

Please accept marketing-cookies to watch this image.

The release was uneventful. There was no PR around the release, no build up or media coverage. Just a sparce erlang.org website (handcrafted using vi). An email was sent to the Erlang mailing list, a post made the front page on slasdot, alongside a mention on comp.lang.functional (which Joe dutifully followed up on). There were no other marketing activities promoting the fact that Ericsson had released a huge open source project. My highlight that week was not the release of Erlang, but going to a Marky Ramone and the Intruders gig in a dive in downtown Dallas. Little did I know how Open Source Erlang would affect the tech industry, my career, and that of many others around me.

Getting it out of Ericsson

What made it all happen? Many of us wanted Erlang to be released as open source, for a variety of reasons. Some of my Ericsson colleagues wanted to leave their current positions, but continue building products with what they believed was a silver bullet. Others wanted to make the world a better place by making superior tools for fault tolerant and scalable systems available to the masses. For Ericsson management, a wider adoption of Erlang would mean a larger pool of talent to recruit from.

Jane Walerud was amongst us trying to sell Erlang outside of Ericsson and one of the few who at the time knew how to speak to management; she understood that the time of selling programming languages was over. Håkan Millroth, head of the Ericsson Software Architecture Lab suggested trying this new thing called “Open Source”. Jane, armed with an early version of The Cathedral and the Bazaar paper, convinced Ericsson management to release the source code for the Erlang VM, the standard libraries and parts of OTP.

Until Erlang was out, many did not believe it would happen. There was a fear that, at the last minute, Ericsson was going to pull the plug on the whole idea. Open Source, a term which had been coined a few months earlier, was a strange, scary new beast large corporations did not know how to handle. The concerns Ericsson had of sailing in uncharted territory, rightfully so, were many. To mitigate the risk of Erlang not being released, urban legend has it that our friend Richard O’Keefe, at the time working for the University of Otago in New Zealand, came to the rescue. Midnight comes earlier in the East, so as soon as the clocks struck midnight in New Zealand, the erlang.org website went online for a few minutes. Just long enough for an anonymous user to download the very first Erlang release, ensuring its escape. When the download was confirmed, the website went offline again, only to reappear twelve hours later, at midnight Swedish time. I was in Dallas, fast asleep, so I can neither confirm nor deny if any of this actually happened. But as with every legend, I am sure there is a little bit of truth behind it.

The Dot Com Bubble Era

Adoption in first few years was sluggish. Despite that, the OTP team, lead by Kenneth Lundin was hard at work. In May 1999, Björn Gustavsson’s refactoring of the BEAM VM (Bogdan’s Erlang Abstract Machine) becomes the official replacement of the JAM (Joe’s Abstract Machine). Joe had left Ericsson a year earlier and the BEAM, whilst faster, needed that time to make it production ready.

I recall the excitement every time we found a new company using Erlang/OTP. Telia, the Swedish phone company, was working on a call center solution. And One2One – the UK mobile operator – had been initially using it for value added services, expanding its use to the core network. IdealX in Paris, did the first foray into messaging and XMPP. Vail System in Chicago and Motivity in Toronto were using it for auto dialler software. And Bluetail, of course, had many products helping internet service providers with scalability and resilience.

The use of Erlang within Ericsson’s core products continued to expand. This coincided with my move to London in 1999, where I increasingly came across the need for Erlang expertise within Ericsson. Erlang Solutions was born. Within a year of founding the company, I had customers in Sweden, Norway, Australia, Ireland, France, the US, and of course, the UK. In 2000, we got our first non Ericsson customer; training, mentorship and a code review for IdealX in Paris.

It was the Bluetail acquisition by Alteon Web Systems for $152 million (a few days later Alteon were acquired by Nortel), which sent the first ripples through the Erlang community. An Ericsson competitor developing Erlang products! And a generation of successful entrepreneurs who had the funds to get involved in many other startups; Synapse, Klarna and Tail-f being some of them.

Soon after the Bluetail success comes the dot com crash, the industry went into survival mode, and then later recovery, mode. The crash, however, did not affect academics who were moving full steam ahead. In 2002, Prof. John Hughes of Chalmers University managed to get the Erlang Workshop accredited by SIGPLAN and the ACM. We didn’t really know what this all meant, but were nonetheless, very proud of it. The ACM SIGPLAN Erlang Workshop in Pittsburgh (Pennsylvania) was the first accredited workshop. Here, a PhD student from Uppsala University named Richard Carlsson presents the Erlang version of try-catch to the world.

In September 2004, Kostis Sagonas, from Uppsala University hijacks the lightning talks at the ACM SIGPLAN Erlang workshop in Snowbird (Utah) and gives the first public demo of Dialyzer. He runs it on a code base from South African Teba Bank. It was the first of many amazing tools he and his students contributed to the ecosystem.

Erlang had for a long time been used to teach aspects of computer science in many Universities all over the world. This in turn lead to research, Master Thesis and PhD projects. The workshop provided a forum for academics to publish their results and validate them with industrial partners. Downloads from the erlang.org site kept on increasing, as did adoption.

In 2003, Thomas Arts, program manager at the IT University of Gothenburg, invites me to teach an Erlang course to his undergraduate class. Prof. John Hughes, despite already knowing Erlang, wanted to learn about it from someone who had used it in production, so he joins the class. One morning, he shows up to class tired, having pulled an all nighter. He had developed the first version of Erlang QuickCheck, which he was dutifully using to test the course exercises. That was the start of Quviq and a commercial version of QuickCheck, a property based testing tool second to none. I ended up teaching at the IT University for a decade, with over 700 students attending the course.

Getting Into Messaging

During the dot com crash, Alexey Shchepin starts work on an XMPP based instant messaging server called ejabberd. After working on it for three years, he releases version 1.0 December 1st, 2005. Facebook Chat forks it, rolling out a chat service to 70M users. At around the same time, Brian Acton and Jan Koum founded WhatsApp, also based on a fork of Ejabberd. As forking Ejabberd was all the hype, MongooseIM did the same, becoming a generic platform for large scale messaging solutions.

In May 2006, RabbitMQ is born, as we find out that work was underway to define and implement a new pub/sub messaging standard called AMQP. RabbitMQ is today the backbone of tens of thousands of systems. By the end of the decade, Erlang had become the language of choice for many messaging solutions.

The Multi-Core Years

It was not only Universities innovating during the dot com recovery. In May of 2005, a multi-core version of the BEAM VM is released by the OTP team, proving that the Erlang concurrency and programming models are ideal for future multi-core architectures. Most of the excitement was on the Erlang mailing list, as not many had realised that the free lunch was over. We took Ejabberd, and just by compiling it to the latest version of Erlang, got a 280% increase in throughput when running it on a quad-core machine.

In May 2007, the original reel of the 1991 Erlang the Movie was anonymously leaked from a VHS cassette in an Ericsson safe and put on the erlang.org site, eventually making its way to YouTube. Still no one has publically taken responsibility for this action. The world, however, finally understood the relief felt by those still under Ericsson NDA that none of the computer scientists featured in the film had given up their day jobs for a career in acting. The film got a sequel in 2013, when a hipster tries to give Erlang cool look. This time, the curpruit releasing it is identified as Chicago resident Garrett Smith.

In 2007, Programming Erlang by Joe Armstrong is published by the The Pragmatic Programmers. The following year, in June 2008, I held the first paper copy of Erlang Programming; a book Simon Thompson and I had spent 18 months writing. At the time, an O’Reilly book was the seal of approval that emerging programming languages needed, giving way to many other fantastic and diverse range of books in many languages.

The book launch party happened in conjunction with the first commercial Erlang conference, the Erlang eXchange in London June 2008. It was not the first event, as Bjarne Däcker, the former head of the Ericsson Computer Science Lab, had for almost a decade been running the yearly Erlang User Conference in Stockholm. But November in Sweden is cold, and the time had come to conquer the world. The Erlang eXchange gives way to the first Erlang Factory, taking place in March 2009 in Palo Alto (California). Much more exotic, though equally beautiful locations.

For the first time, the European Erlang community meet their American peers. We all got on like a house on fire, as you can imagine. At the conference, Tony Arcieri presents Reia, a Ruby flavoured version of Erlang running on the BEAM. Who said that a Ruby like syntax is a bad idea? Other presenters and attendees that year had stellar careers as entrepreneurs and leaders in the tech space.

An Erlang user in the US at the time was Tom Preston Werner. He was using it to scale the Ruby front-end of a social coding company called Github. In November of 2009, when in Stockholm for the Erlang User Conference, I introduced him and Scott Chacon to the OTP team. They spent an afternoon together, prompting the OTP team to move the development of Erlang to github, making it its main repository.

Conferences spread all over the world. Events have been held in Amsterdam, Bangalore, Berlin, Buenos Aires, Brussels, Chicago, (many places I can not spell in) China, Krakow, Los Angeles, Paris, Moscow, Mexico City, Milan, Munich, New York, Rome, San Francisco, St Andrews, Tel Aviv, Vancouver, Washington DC and many many other places.

The Cappuccino Years

In 2010, I teach my first graduate class at Oxford University. Erlang was picked for the Concurrency Oriented Programming course. It was also the year Bruce Tate’s Seven Languages in Seven Weeks was released. It was through this book where one of Rails’ core committers, José Valim, realized that Erlang was ahead of everyone in the concurrency race because it also tacked distribution.

In January 2011, the first commit in the Elixir repo happens. The results are presented the following year at the Krakow Erlang Factory, reaching version 1.0 in September 2014. Like all successful languages, he was trying to solve a problem, namely bringing the power of Erlang to wider communities, starting with Web.

The time was right. In January 2012, WhatsApp announce that by modifying FreeBSD and the BEAM, they achieved 2 million TCP/IP connections in a single VM and host. Their goal was to reduce operational costs, running a scalable service on a hardware footprint that was as small as possible. These results were applicable to many verticals, the Web being one of them.

The same month as the WhatsApp announcement, a group of companies pool together knowledge, time and resources to create the Industrial Erlang User Group. They worked with Ericsson to move Erlang from a derivative of the Open Source Mozilla Public License to the Apache Licence, contribute to the dirty scheduler, get bug tracking tool launched, fund a new erlang.org site, launch Erlang Central, and worked together with an aim of setting up a foundation.

Elixir Comes of Age

In July 2014, Jim Freeze organises the first Elixir Conference in Austin (Texas). There were 106 attendees, including keynote speaker Dave Thomas’ dog. Chris Mccord presented Phoenix, rising from the ashes. Robert Virding and I are part of the lineup and I recall my message loud and clear: just because you know Ruby, don’t believe them when they tell you that learning Elixir is easy. Your challenge will be thinking concurrently.

The main idea behind Elixir is concurrency, and knowing how to deal with it is critical to the success of the project. A year later, in August 2015, Phoenix 1.0 was released. It had the same effect Rails had on Ruby, bringing people to Elixir. Now, you didn’t need to master concurrency to get it! Nerves came along soon after, moving Elixir away from being a language solely for the web.

At Elixir Conf, I spoke about the book I was co-authoring with Steve Vinoski, Designing For Scalability with Erlang/OTP. At the time, it was available as a beta release. Little did I know that I had to wait for June 2016 to hold to a paper copy. The last four chapters, which should have been a book on their own, ended up taking 1.5 years to write. The main lesson to others writing a book is that if your partner tells you “you are going to become a father”, you have 8 months to finish the book. The alternative is you ending up like me, attending the release party a few days before your second child is due. The book was dedicated to Alison, Peter and our baby bump. Baby bump was born in early July, bringing truth to the Erlang saying that “you do not truly understand concurrency until you have your second child”.

The Erlang Ecosystem

Throughout 2016, Elixir adoption kept on growing. Conference talks on Lisp Flavoured Erlang and Effene – two other languages on the BEAM – revealed they had code running in production. New experimental ports kept appearing on our radar; the era of a language was over. As with .net, containing C#, F#, Visual Basic and others or the JVM ecosystem encompassing Java, Scala, Clojure, Groovy, to mention but a few. The same happened with Erlang and the BEAM, prompting Bruce Tate to coin the term Erlang Ecosystem.

Alpaca, Clojerl, Efene, Elixir, Elm-beam,, Erlog, Erlua, Fez, Joxa, Lisp Flavoured Erlang and Reia, which alongside Erlang and Elixir, are opening an era of interaction and cooperation across languages. Together, we are stronger and can continue evolving!

In December of 2018, the paperwork for the Erlang Ecosystem Foundation was submitted, setting up a non profit whose goal is to foster the ecosystem. I am looking forward to more languages on the BEAM gaining in popularity, as we improve interoperability, common tools and libraries. And as the demand for scalable and fault tolerant systems increases, so does the influence of Erlang’s constructs and semantics in the new languages within and outside the ecosystem. I hope this will set the direction for the next 20 years as a new generation of technology leaders and entrepreneurs spreading their wings.

The Future

In 2018, at Code BEAM Stockholm conference discovering the Erlang Ecosystem (formerly known as Erlang User Conference), Johan Bevemyr from Cisco announces they ship two million devices per year with Erlang applications running on them. That blew the audience away, as it meant that 90% of all internet traffic went through routers and switches controlled by Erlang. Erlang powers Ericsson’s GPRS, 3, 4G/LTE and if recent job ads are anything to go by, their 5G networks. Or IoT infrastructure through VerneMQ and EMQ, the most popular MQTT and CoAP brokers. Erlang powers not only the internet and mobile data networks, it is the backbone of tens of thousands of distributed, fault tolerant systems. Switches billions of dollars each day through its financial switches and even more messages through its messaging solutions. You can’t make this stuff up!

These are just some of my personal highlights from the last 20 years. In it all, there is a realisation that we are far from done. Joe Armstrong, in 1995, told me Erlang will not be around forever. Some day, he said, something better will come along. Fast forward to December 2018, I am still waiting, with an open mind, for that prophecy to come true. Whatever it is, there is no doubt Erlang will be a heavy influence on it.

A big thank you to Joe, Mike and Robert for making that first phone call, and to Bjarne for enabling it. A shout out to Jane who, by getting it outside of Ericsson, ensured its survival. You all started something which has allowed me to meet, work and learn with amazing and talented people using a technology that we are all passionate about. It has given us a platform enabling many of us to drive the innovation forward for the next 20 years (at least)!