In my previous post, I’ve talked about the idea to split KRunner into a few separate processes that will execute the different searching plugins (also known as runners) isolated from one another.

This time, I’m going to talk about the Voy library (framework?) that was born from this idea and that is still evolving to better suit the use-cases of KRunner, while also being general enough for a few more test apps that I build against it.

The Model

The idea is simple – the system should be split into a bunch of components. Each component should follow the UNIX philosophy – do one thing, and do it well. The components need to be isolated in the sense that the only way for two components to interact with each other is by sending messages – no shared memory, no shared data, nothing – just messages.

You can think of it like these components were persons. We don’t know what the next person is thinking about unless he/she tells us.

This approach allows implementing complex systems without any synchronization primitives like mutexes, and if done well, they lift the requirement that all components need to be in the same process or even on the same host. We just need to be able to serialize messages and transfer them over the network.

This model does introduce complexities in other areas – we need to think of the program logic in a slightly different way, but it is worth it.

There are a few libraries that implement this and similar models like the C++ Actor Framework and SObjectizer, but they have a significantly different project aims to this one. Still, they are worth checking out.

The Messaging System

In this model, the messages are crucial, so the most important part of the Voy library is its message handling.

Like in the real life, in order to communicate with others, we either need to know to whom we want to pass information, or we can yell to make sure everybody hears us.

In Voy, this is a bit more granular. Each agent (we will call these isolated components agents from now on) has an address comprised of three components:

  • the unique id of the host it is running on
  • a unique id of the process instance it belongs to (called a ‘group’ in Voy)
  • and a unique id of the agent itself

This way, if we know the address, we can send a targeted message to another agent. We could also choose to send messages to all agents belonging to a specific group, or to all agents located on a specific host. This gives us some control, but it is mostly based on the physical location of the Agent.

Sometimes, we might want to target a specific group of agents that are not in the same location, but still not to spam other agents with messages they do not need.

As an example, imagine a community of software developers working on a huge Free Software project. They are usually located in completely different parts of the world, but they still belong to the same team. They should be able to communicate without agents that are not in that team receiving those messages.

Just for the reminder what distributed KRunner will look like
Just for the reminder what distributed KRunner will look like

This brought named teams to Voy. While everything else is based on UUIDs, teams have proper names. This means that we are able to split agents into named groups, and they will be able to communicate even if they don’t know the addresses of other team members.

This works similar to mailing lists – I don’t need to know mail addresses of all Plasma developers, but I still can send them an e-mail through the mailing list.

For improved granularity, sending messages to a team can also be limited to a specific host or process instance.

To be continued…

Next parts:

  • The message sending API
  • The new KRunner design
  • The future for Voy evolution