Applying Scrum to a system of fluid teams working on an open-source product

The modern IT workplace is weird.

You get a desk that sometimes has a chair, but often also wheels or a button that makes it go up and down. There are often bean bags involved, bosses that say they are not really bosses, complimentary fruit and candy, engagement programs, and people constantly pushing you to “self-actualise” and grow.

Your projects often come with a lot of trust and freedom, but that also means that project structures are fluid, and there is a lot of responsibility for individuals to set up the direction. On the spectrum of freedom and security, everything serves the most important goal – adapting to change.

The approach of Agile and Scrum

A lot has been said about how Agile and Scrum differ from other approaches to organising work. They are born out of experiencing first-hand how detrimental it can be to follow ill-fitting methods that work well in other industries. At their core, they are about creating effective communication channels – between team members, teams and the market, different stakeholders and the various people responsible for orchestrating the process. A culture and environment that promotes sharing, listening and taking ownership of your work is required for this approach to shine and foster dynamic progress and satisfaction of all involved.

It would be tempting to say that following the Scrum guide verbatim is a silver bullet for running all IT projects, and many would argue it is.

In this short article, I would like to tell you the story of how we took the spirit of Agile, measured it against our needs and then iteratively made it our own. Did we manage to reach the shore of a procedure that works for us? For now – yes. Will we stick with it? I have no idea – come back and check in with us in a few months. One thing is for sure, if we are not, it will be born out of a test-and-learn approach which has led us to something that fits our way of working better.

Mongoose wrangling – you have to be agile!

MongooseIM is an open-source product developed by a commercial company. 100% of the code is available to the public; everyone can use it, customise it, contribute, point out bugs, and raise feature requests. At the same time, the core development team works not only on developing the product but also on assisting companies that want to create custom extensions, optimise their installations, scale or adjust their setup. This means we have two groups contributing to MongooseIM – the open-source community and Erlang Solutions developers. The second group can’t be considered a monolith, since some of them only work with the open source product while others assist the company’s customers with different projects.

In short, what we have on our hands is a system of fluid teams. By definition, a fluid team consists of “experts from disparate functions and geographies who must get a temporary project or task up and running, sometimes with completely different priorities, beliefs and values”. As challenging as it may sound, this form of organisation is typical for modern IT, it has been widely accepted that nowadays “this is how we roll”. What we have at MongooseIM goes a step further – it is not just a fluid team, it’s a system of fluid teams. We all work on MongooseIM in one way or another, but our goals, priorities and procedures are often very different, since what is important for a single client is not always a must for the entire product (and vice versa). Different projects follow different procedures; they follow different guidelines and assumptions and sometimes they even look for solutions to contradicting problems. We slip in and out of different roles and teams, and the direction of the entire system is forged somewhere in between these parallel interests.

This also means implementing Scrum is tricky.
You have to remember that Scrum is not only valuable, it’s also costly. You spend a lot of time and effort on inspection – meetings, analysis, discussions and syncing. It’s all an investment you make to empower your team, allow them to learn, minimize certain risks and make sure you generate value, but to have a return on that investment, you have to be consistent in your actions.

For a team to be considered totally “scrummy” it has to meet a few criteria. It has to be cross-functional, and self-organising, it can’t have any hierarchies, and it has to be small. Ideally, it should have between 3 and 9 team members. Many of these assumptions make little sense when talking about an open-source project like MongooseIM. For example – who do we consider to be a part of the team? Everyone who contributes to the project? If so, there is no way of determining how many people that is or to coordinate their efforts. So do we define it as all Erlang Solutions employees working on MongooseIM? Some of them are embedded in customer teams, making it near impossible to have them conform to Scrum ceremonies and requirements. The cost of abiding by the client’s procedure and our Scrum would make them unable to do any actual work. Perhaps we should narrow down our focus to just the core development team? Very few members will be with the team for more than 2 consecutive sprints, there will be sprints with just 2 or 3 members, followed by sprints with 9 or more, we will constantly pay the cost of onboarding new people and relearning and restructuring day-to-day work. In short – it will be very hard to let the team members do their work, learn from each other, form long term plans, and benefit from what Scrum offers. Taking all of that and more into consideration, for us, there was little sense in going “full Scrum”. To be able to progress and not lose our minds, we had to remain Agile, but also, we had to make it our own.

Building a frame

Let’s skip the part where we start with an eager team, an inexperienced Scrum master and a willingness to do everything just as the Scrum guide says. That was our starting point, but it is not exactly the basis of what we do now. For me that elusive basis was best expressed by Tomasz Włodarek at one of his amazing workshops. To paraphrase the thought, he encourages management to steer away from creating a workflow as a set of rules, or set pipelines with different steps and stages. You do need those to create predictability and thus stability, but at the heart of how you operate, there should be something else. Your work culture should be built on two sets of objects later filled by your self-organising teams. Those two are the frames and attractors. You build these by answering two simple questions – “what are the things we want to discourage?” and “what are the things we want to encourage?”. The answers will never be complete and might change over time, but they need to be communicated to the team.

For our team, there were a few things we did not want. We did not like it when we were swamped with ad hoc requests. We did not like it when people started a lot of tasks and never finished any. We did not like external factors interfering with our goals. We did however, and still do, love our product.

Working with Erlang, XMPP and MongooseIM is something the team is genuinely excited about. We love to learn, explore, come up with new ideas, we love the open source culture and commercial projects we get to work with. It is exciting to get to work with customers who have massive, running installations and then assist a hobbyist writing his own MongooseIM module for fun. It is great seeing in action how people use and build upon our work.

Luckily this state of mind aligned perfectly with what our management wanted – a highly professional, efficient team always growing their capabilities, ready to tackle any task.

The initial attractors and frames we’ve set reflected that:
We wanted to encourage knowledge sharing – so we created channels to share the lessons we’ve learnt and award people who actively engage in assisting others.
We wanted the team to have agency – so we made sure that both our backlog and pipeline of commercial projects is transparent (everyone can volunteer, contribute or comment) and that everyone is encouraged to submit their ideas on what to do next. This also eliminated many of the bottlenecks we have run into in the past.
We wanted the team to have the comfort of working on their goals without interruptions – so we’ve introduced work iterations and separated the development team from other teams.

The process that crystallized around those assumptions gets more and more refined over time, but these add up to an agreed-upon basis of all the changes we introduce.

What does that mean in practice?

Let’s perhaps start with what we have in common with Scrum. We have sprints with all the prescribed Scrum ceremonies that produce potentially shippable increments. We do Daily Scrum, Sprint Planning, and Sprint Retrospectives. Our backlog is always growing, and the Release Owner orders it based on their best judgement and value calls. Have you noticed that? The first difference?

We do not have a Product Owner. We tried and we are very grateful to our past Product Owners for all their contributions, passion and the great ideas they brought to inspire the team. What became apparent after working with these talented individuals is that if your product is an open source project with an elaborate business model built around it, traditional POs do not get enough space to shine. What has worked for us is actually an expression of the self-organising principle of Scrum – we put things up for a vote.

Every release ends with a small “political” campaign where members of the development team present their ideas about what the focus of the next release should be. We have a few meetings discussing the ideas, some people contribute to the existing proposals, others form their own. We’ve had elections with multiple candidates and elections where one candidate asked us to vote on release themes and priorities to create guiding principles for the upcoming months. The result is a vision everyone understands and an uncanny level of transparency and insight into the plan.

And, since everyone has contributed to it, we all feel responsible, engaged and encouraged to take ownership of the work that needs to be done. The Release Owner has extra powers (they decide on priorities and can overrule a tied vote) but also extra responsibilities (managing external stakeholders including the marketing team, gathering extra information, attending many meetings that the other team members can skip). As they say – with great power comes little time to do some actual coding yourself. The Backlog Grooming is of course an ongoing process. Everyone can add stuff to the backlog at any point, but it is the Release Owner who decides the order. We do hold backlog grooming meetings to which the team should bring their ideas, preferably in the form of PBIs we can all see, discuss and ask questions about. Top priority, groomed items get placed on the top of the list and are likely to be included in the next sprint’s backlog.

So far we are not that far from a traditional Scrum with a PO who is part of the team. Where is the key difference?

Let’s switch the perspective and ask – if you happen to be an Erlang solutions employee and part of the MongooseIM core team: does this mean you are on the sprint? Not always. Does it mean that you are a part of the sprint work? Most yes, no, maybe, kinda, can you clarify what you mean?

We have people who join the sprint. At Sprint Planning they state their availability and we populate the sprint backlog based on their estimates and capacity. There are people who know they will have some availability but have no idea how much. For them, we have a neat Kanban board with community support items, research, occasional marketing requests, side projects that can bear fruit in the form of estimable PBIs for the Scrum team. There are people on commercial projects who are invaluable to us and the way we plan. They have hands-on experience with live implementations of MongooseIM, they have battle stories, insights, ideas, they know the pain points and have a unique perspective on the work. It is critical for them to know what is happening on the open source product and it is critical for the Scrum team to know what’s up with them. The two latter groups are not included in the sprint capacity and we make sure we do not depend on them to complete the sprint work, but their feedback is welcome and assistance is invaluable.

Another important thing is that the division between the groups is strict only from the single sprint perspective and in no way permanent. There are commercial projects that last months and others that are completed within days. People who technically are not on the sprint are still invited to join our meetings, listen in and join slack conversations – they are much more than external stakeholders.

The result of all this?
We are still “scrummy” (all the roles, rules, ceremonies and artifacts are there and we get to benefit from them).
We are inclusive and responsive to specific needs of team members, external stakeholders and projects.
We are highly communicative and flexible.
We have a structure that gives us predictability but also the capacity to change.

Cool tricks

There are many supplementary techniques we use in our work to make sure the attractor/frame setup is something we actually live and breath and not just sometimes brag about. Here are some examples that can inspire you to come up with ideas to introduce to your own project/team/organization.

Feedback

Creating a culture of feedback is not just a way to inspect and adapt but also to provide people with a sense of agency. We’ve introduced many tools team members can use to share their thoughts with different levels of formality. Let’s, for example, explore Pigeon Post – a form where you can send a short note and a cookie (or not) to people who did something notable. The person not only gets to enjoy the cookie but can also include these notes in their appraisal process as evidence of what a great teammate they are. It’s a fun way to express appreciation to your colleagues and point out specific behaviours we value (or not – it’s not just for praise and positive feedback). Other systems we have, allow people to explicitly raise points and introduce ideas that are then included in the office-wide processes and day-to-day work. Most of the mechanisms we use in the project pipeline or for knowledge sharing purposes, stem from suggestions from the team. One important rule to keep in mind, is that when you ask for feedback or ideas, it’s on you to act on the response. Whether it’s a suggestion box, a retrospective action point, informal conversation or an official request, you have to have a timely reply. If nothing else, make a short presentation listing what you’ve done to address the ideas, preferably no later than 3 months after they have landed on your desk. If you don’t, you are not really encouraging your team to engage, rather just proving that what they say does not matter.

Knowledge sharing

That is the number one reason for most of the Pigeon post cookies. “I loved your report on Helm – you get a cookie”, “I really enjoyed your Lessons Learned – here’s a cookie”, “Your workshop was awesome – One cookie coming right up!”. Developers love learning, love exploring new ideas, sharing the knowledge, discussing stuff. If your team members are afraid to speak up, unwilling to share or reluctant to talk, something has gone terribly wrong and needs to be addressed right now. Perhaps they are afraid of ridicule? Work on how safe they feel. Perhaps they feel they don’t have enough time to “waste” on knowledge sharing? There’s something wrong with the workflow. Perhaps they do not see how, where or when to let people know they have something to share? Create a Lessons Learned series! The format that worked for us is a company-wide library of presentations from different people and teams sharing stuff they find interesting. It ranges from project post-mortems, lessons learned from different projects, panel discussions to presenting a pet project or favourite technology. Whatever it is, it is welcomed, encouraged and rewarded. Not by a “knowledge-sharing bonus” or an official letter of praise but by attention and appreciation from the rest of the team. This is not only how we build expertise, but also ourselves as teachers and experts.

Candy

How do you make people actually follow up on all these values and ideas? A lot has been said about what drives us, how to build or destroy motivation. My favourite tool is humour, empathy and showing people you actually like them (the prerequisite here is that you actually do).
Let’s explore an example. MongooseIM Scrum Team has daily standups, but other teams and standalone consultants have separate, internal meetings to worry about. We would like them to write short standup notes every morning to let us in on what they’re doing, but we do not want to enforce it or annoy them with the request (they are very busy people!).
The solution? Candy. Everyone who shares what they are doing on a given day, gets a personal visit from the Scrum Master carrying a bag of candy. They get to pick one. Ridiculous, right? No one would change their behaviour for such a small bribe. “If you want me to write standups, just make it part of my job description, we are all adults here, do not insult my intellect!” But it’s not a bribe. It’s a nice gesture, an occasion to talk for a minute without big disruptions, to quickly see how the other person is doing. It’s small and it’s fun. After a while everyone sees the benefit of the standups and they write it regardless of the reward, but the candy stays. You get to learn everyone’s favourite flavour, you get people requesting different types of chocolate, finally you get a petition asking for a “Healthy jar” with nuts as an alternative to candy. Some people choose either the candy or nuts depending on how their day went – what a great conversation starter you just earned! Your team gets closer, gets friendlier, gets engaged. Of course this technique will not work for every team, and I am not advocating for getting everyone off their healthy and balanced diet. The tool you choose for this type of exercises can be whatever you see fit, but it should:

1. Not be too valuable or attached to salary/bonuses/promotions (or it will provoke people to try to cheat the system, see if they can break it – they are developers afterall)
2. Be easily accessible and of similar value (we use candy for all kinds of exercises and mini-procedures)
3. Not cause too much formality or fuss (it should be a nice gesture, a conversation starter not a potential black market currency).

Want some other great ideas on how to engage people and make them enjoy their work more? Ask your team!

The takeaway

The modern IT workplace is weird. It makes you think of things, observe the nature and craziness that is life. All of a sudden you see how complex systems emerge from simple rules and how trying to design or foresee something can sometimes only make things worse.

Before working in IT, you might think that being proactive is better than being reactive, that defending against failure, planning for all possible cases is the smart way to go. The more you observe nature, Erlang or self-organizing teams, the more certain you will be that that’s wrong. Evolution is smarter than revolution or any sort of planning, it’s more mature, natural, it allows order to emerge painlessly. Embracing your team’s potential, allowing mistakes and experimentation helps expose their strength, building an environment that fosters creativity where traditionally it was limited and curbed. No one person can design a perfect system or process for a diverse team facing challenges that are hard to foresee. What you can do is listen, encourage and adapt.
It’s more fun that way.