MongooseIM 5.1 Configuration Rework

MongooseIM is a modern messaging server that is designed for scalability and high performance. The use of XMPP (Extensible Messaging and Presence Protocol) extensions (XEPs) means it is also highly customisable. Since version 4.0 it has been using the TOML configuration file format, which is much more user-friendly than the previously used Erlang terms. The latest release, MongooseIM 5.1, makes it more developer-friendly as well by reworking how the configuration is processed and stored, hence making it easier to understand and extend. Other significant changes in this release include improvements of the Inbox feature, support for the latest Erlang/OTP 25 and numerous internal improvements, like getting rid of the dynamically compiled modules. For all changes, see the Release Notes.

The configuration process

Until version 3.7, MongooseIM was configured with a file called mongooseim.cfg, that contained Erlang terms. It was a legacy format that was the primary obstacle encountered by new users. The configuration terms were interpreted and converted to the internal format. This process was difficult because there was no configuration schema – all conversion logic was scattered around the system. Some of the conversion and validation was done when parsing the configuration terms, and some was done afterwards, sometimes deeply inside the logic handling specific use cases. This meant that many errors were only reported when the corresponding features were used instead of when the system started, leading to unwanted runtime errors. As a result, the whole configuration subsystem needed to be rewritten. The new configuration file is now called mongooseim.toml and it uses the TOML format, which is simple, intuitive and easy to learn.

Because of the significant code base size and the tight coupling of system logic with configuration processing, it was very difficult to write a new configuration subsystem from scratch in one release cycle. What is more, simultaneous support for both formats was needed during the transition period. This led us to dividing the work into three main steps:

1. Addition of new config parsing, validating and processing logic, which used the old internal format as the target (version 4.0).
2. Dropping the old Erlang configuration format (version 4.1).
3. Reworking the internal format to be consistent with the TOML configuration file (version 5.1).

Each of the steps was further divided into multiple small changesets, keeping all CI tests passing the entire time. We also periodically load-tested the code to monitor the performance.

What did we do?

Here is one of the most complicated sections from the default mongooseim.cfg file taken from the old MongooseIM 3.7.1:

{8089, ejabberd_cowboy, [
  {num_acceptors, 10},
  {transport_options, [{max_connections, 1024}]},
  {protocol_options, [{compress, true}]},
  {ssl, [{certfile, "priv/ssl/fake_cert.pem"}, {keyfile, "priv/ssl/fake_key.pem"}, {password, ""}]},
  {modules, [
    {"_", "/api/sse", lasse_handler, [mongoose_client_api_sse]},
    {"_", "/api/messages/[:with]", mongoose_client_api_messages, []},
    {"_", "/api/contacts/[:jid]", mongoose_client_api_contacts, []},
    {"_", "/api/rooms/[:id]", mongoose_client_api_rooms, []},
    {"_", "/api/rooms/[:id]/config", mongoose_client_api_rooms_config, []},
    {"_", "/api/rooms/:id/users/[:user]", mongoose_client_api_rooms_users, []},
    {"_", "/api/rooms/[:id]/messages", mongoose_client_api_rooms_messages, []},
    %% Swagger
    {"_", "/api-docs", cowboy_swagger_redirect_handler, {priv_file, cowboy_swagger, "swagger/index.html"}},
    {"_", "/api-docs/swagger.json", cowboy_swagger_json_handler, #{}},
    {"_", "/api-docs/[...]", cowboy_static, {priv_dir, cowboy_swagger, "swagger", [{mimetypes, cow_mimetypes, all}]}}
  ]}
]},

This section used to define one of the listeners that accept incoming connections –  this one accepted HTTP connections, because it used the ejabberd_cowboy module. It had multiple handlers defined in the {modules, [...]} tuple. Together they formed the client-facing REST API, which could be used to send messages without the need for XMPP connections. The whole part was difficult to understand, and most of the terms should not be modified, because the provided values were the only ones that worked correctly. To customise it, one would need to know the internal details of the implementation. However, the logic was scattered around multiple Erlang modules, making it difficult to figure out what the resulting internal structures were. For version 5.0, we used TOML, but the configuration was still quite complex because it reflected the complicated internals of the resulting Erlang terms – actually this was one of the very few parts of the file needing further rework. By cleaning up the internal format in 5.1, we have managed to make the default configuration simple and intuitive:

[[listen.http]]
  port = 8089
  transport.num_acceptors = 10
  transport.max_connections = 1024
  protocol.compress = true
  tls.verify_mode = "none"
  tls.certfile = "priv/ssl/fake_cert.pem"
  tls.keyfile = "priv/ssl/fake_key.pem"
  tls.password = ""

  [[listen.http.handlers.mongoose_client_api]]
    host = "_"
    path = "/api"

By just looking at the top line, one can see that the section is about listening for HTTP connections. The [[...]] TOML syntax denotes an element in an array of tables, which means that there could be other HTTP listeners. Below are the listener options. The first one, port, is just an integer. The remaining options are grouped into subsections (TOML tables). One could use the typical section syntax there, e.g.

[listen.http.transport]
  num_acceptors = 10
  max_connections = 1024

Alternatively, an inline table could be used:

transport = {num_acceptors = 10, max_connections = 1024}

However, for such simple subsections the dotted keys used in the example seem to be the best choice. The TLS options make it obvious that fake certificates are used, and they should be replaced with real ones. The verify_node option is set to none, which means that client certificates are not checked.

The biggest improvement in this configuration section was made in the API definition itself – now it is clear that [[listen.http.handlers.mongoose_client_api]] defines a handler for the client-facing API. The double brackets remind that there can be multiple handlers, if, for example, you would like to have another API hosted at the same port. The question is: what has happened to all these specific handlers that were present in the old configuration format? The answer is simple: they are all enabled by default, and if you want, you can control which ones are enabled. The options configurable for the user are limited to the ones that actually do work, limiting unnecessary frustration and easing up the learning curve. See the documentation for more details about this section. By the way, the docs have improved a lot as well.

How did we do it?

One of the challenges of supporting the old internal format was that the new TOML configuration format should be defined once and from then changed as little as possible, to avoid forcing users to constantly change their configuration with each software version. This was achieved with a customisable config processor that takes configuration specification (called config spec for short) and the parsed TOML tree (the tomerl parser was used here) as arguments. Each node is processed recursively with the corresponding config spec node and the current path (a list of keys from the root to the leaf). The config spec consists of three types of config nodes, specified with Erlang records corresponding to the following TOML types:

The root node is a section. TOML processing for each config node is done in up to 5 steps (depending on the node type). Each step is configured by the fields of the corresponding node specification record, allowing you to customise how the value is processed.

The flexibility of these options enabled processing of any TOML structure into arbitrary Erlang terms that was needed to support the legacy internal configuration format. The complete config spec contains 1,245 options in total, grouped into 353 sections and 117 lists, many of which are deeply nested. The specification is mostly declarative, but the fact that it is constructed with Erlang code makes it possible to reuse common sections to avoid code duplication, and to delegate specification of the customisable parts of the system, e.g. the extension modules to the modules themselves, using Erlang behaviours. This way, if you fork MongooseIM and add your own extensions, you can extend the configuration by implementing a config_spec function in the extension module.

In the recent MongooseIM versions (4.0-5.0), the resulting top-level config options were stored in three different Mnesia tables for legacy reasons. This was unnecessarily complicated, and the first step towards version 5.1 was to put each top-level configuration node into a persistent term. Next, the internal format of each option was reworked to resemble the TOML tree as much as possible. Some options, like the extension modules, needed a more significant rework as they used custom ETS tables for storing the processed configuration. The effort resulted in a significant reduction of technical debt, and the code base was also reduced by a few thousand lines increasing readability and maintainability without any loss in overall performance. The table below summarises the whole configuration rework that has been undertaken over the last few releases.

Config processing in action

As an example, let’s see how the first option in the general section, loglevel, is defined and processed:

general() ->
  #section{items = #{<<"loglevel">> => #option{type = atom,
                                               validate = loglevel,
                                               wrap = global_config},
                     (...)
                    }
          }.

The TOML configuration file contains this option at the top:

[general]
  loglevel = "warning"
  (...)

The type atom means that it will be converted to an Erlang atom warning. It is then validated as loglevel – the validators are defined in the mongoose_config_validator module. The format_items step does not apply to options, and the process step is skipped as well, because there is no custom processing function defined. The option is then wrapped as global_config and will be put in a separate persistent term (it is not a nested option). It can be accessed in the code like this:

mongoose_config:get_opt(loglevel)

There are many more features, e.g. nested options can be accessed by their paths, and there are host-type-specific options as well, but this simple example shows the general idea.

Plans for the future

Going back to the topic of HTTP API, you can still find multiple REST API definitions in the default configuration file, which can be a bit confusing. This is because of the long evolution of the APIs and the backwards compatibility. However, for the upcoming release we are redesigning the API and command line interface (CLI), adding brand new GraphQL support, and making the APIs more complete and organised. Stay tuned, because we are already working on a new release!

If you would like to talk to us about how your project can benefit from using MongooseIM, you can contact us at general@erlang-solutions.com and one of our expert team will get right back to you.