Just a screen-shot this time.

The setup goes as follows:

• GUI (in the screenshot) runs on my main computer
• The runner that searches for applications with few less important ones is in a separate process on the same system
• Baloo runner is on another computer (since I have Baloo running only on that system)
• And the Recoll runner runs on yet another separate system

The result is in the screenshot:

I have a lot of issues to tackle, and to make it all usable by normal people, but I had to share this milestone since it is quite cool. :)

p.s. Mind that the Recoll runner is even able to return the section of the file where the search item appears

p.p.s. Don’t mind the ugly UI, it is just for the testing purposes.

It is strange to talk about improvements when the product does not yet exist, but there are a few important ideas that deviate from original KRunner that I’ve been working on.

I’ve already ported a few of the more important runners in order to be able to test the infrastucture. While doing this, I’ve been able to detect a few deficiencies of the current design. So I decided to go over the design of KRunner and the way it used to generate the results in order to which parts could be improved.

Reusing the results

In the old KRunner, the user would type a query, and the runners would generate the results for that query. Now, if you added another character to the query, the runners would generate all the results for that new query from scratch.

This is suboptimal. When the user types ‘kons’ and gets Konsole as the result, typing ‘o’ should not remove konsole from the list only to add it again because it still matches the input string ‘konso’.

Now each runner can add the whole string it used for matching to the result. So, in the case of Konsole, it could pass something like ‘Konsole - Terminal’. This way, the GUI part of KRunner will be able to tell that this result is still relevant and that it should keep it in the list. Runner that searches for files might even pass a section of the text where the query appears (this might even be useful to show in the UI - like Recoll does).

Merging the results

The second thing that I always found strange is that if two different runners returned the same result, KRunner will show the result twice. Just enter ‘konsole’, and you will get ‘Konsole’ and ‘Run konsole’.

Now, if two runners return the same thing, the results will be combined, and the result will get a higher score. Since the score (or relevance) is a number between 0 and 1, these are combined in a fun math way used in fuzzy logic.

Result owner

The other problem in KRunner is that each runner is tasked to handle the results it has created. This means that you might get different available actions for the same file if it was returned by two different runners.

So, the second change was to rethink what is a result in KRunner.

Instead of each runner owning its result, it will now, by default, only need to explain what the result is. That way, if it is a file, the GUI will know what actions to provide, if it is an application, the GUI will know what actions to provide, etc. If it is something unorthodox, then, and only then, the runner should be asked how to execute it.

When?

It is difficult to predict, but I hope I’ll be able to make a preview version (called ‘Blade’) by the end of July. The current plan is to release it as a separate application – for testing purposes only, and after that I’ll have to go one KRunner feature at a time to make sure it has no obvious regressions.

The worst part is that, thanks to Kai Uwe, KRunner is now a moving target and I’ll have to play catch-up with all the features he adds to main KRunner. :)

In my previous post, I’ve talked about the idea to split KRunner into a few separate processes that will execute the different searching plugins (also known as runners) isolated from one another.

This time, I’m going to talk about the Voy library (framework?) that was born from this idea and that is still evolving to better suit the use-cases of KRunner, while also being general enough for a few more test apps that I build against it.

The Model

The idea is simple – the system should be split into a bunch of components. Each component should follow the UNIX philosophy – do one thing, and do it well. The components need to be isolated in the sense that the only way for two components to interact with each other is by sending messages – no shared memory, no shared data, nothing – just messages.

You can think of it like these components were persons. We don’t know what the next person is thinking about unless he/she tells us.

This approach allows implementing complex systems without any synchronization primitives like mutexes, and if done well, they lift the requirement that all components need to be in the same process or even on the same host. We just need to be able to serialize messages and transfer them over the network.

This model does introduce complexities in other areas – we need to think of the program logic in a slightly different way, but it is worth it.

There are a few libraries that implement this and similar models like the C++ Actor Framework and SObjectizer, but they have a significantly different project aims to this one. Still, they are worth checking out.

The Messaging System

In this model, the messages are crucial, so the most important part of the Voy library is its message handling.

Like in the real life, in order to communicate with others, we either need to know to whom we want to pass information, or we can yell to make sure everybody hears us.

In Voy, this is a bit more granular. Each agent (we will call these isolated components agents from now on) has an address comprised of three components:

• the unique id of the host it is running on
• a unique id of the process instance it belongs to (called a ‘group’ in Voy)
• and a unique id of the agent itself

This way, if we know the address, we can send a targeted message to another agent. We could also choose to send messages to all agents belonging to a specific group, or to all agents located on a specific host. This gives us some control, but it is mostly based on the physical location of the Agent.

Sometimes, we might want to target a specific group of agents that are not in the same location, but still not to spam other agents with messages they do not need.

As an example, imagine a community of software developers working on a huge Free Software project. They are usually located in completely different parts of the world, but they still belong to the same team. They should be able to communicate without agents that are not in that team receiving those messages.

This brought named teams to Voy. While everything else is based on UUIDs, teams have proper names. This means that we are able to split agents into named groups, and they will be able to communicate even if they don’t know the addresses of other team members.

This works similar to mailing lists – I don’t need to know mail addresses of all Plasma developers, but I still can send them an e-mail through the mailing list.

For improved granularity, sending messages to a team can also be limited to a specific host or process instance.

To be continued…

Next parts:

• The message sending API
• The new KRunner design
• The future for Voy evolution

KRunner is one of the rare areas of Plasma that have been mostly stagnating since 4.x, and is one of the rare parts of Plasma that are still known to crash. At least, the UI has improved in the last few releases thanks to Kai who rightfully became the new KRunner maintainer.

Now, while the UI is still getting some love, the backend is mostly not. During the course of Plasma 5.x development, Aaron had a really great idea (inspired by his newfound love of Erlang) of creating a more mature infrastructure for KRunner that would (among other things) allow it never to block the UI while calculating the results. Unfortunately, this never got integrated into KRunner UI for various reasons.

KRunner infrastructure now

KRunner is plugin-based. When the user performs a search, it loads the plugins and sends them the query data. This allows it to be extended easily, and it makes it one of the most advanced launchers in the free world (and proprietary as well).

The downside is that the UI is not isolated from the engine, and the plugin loading and more advanced searching plugins (runners) can slow the interface down, and ruin the user experience. Whats even more problematic, a single misbehaving runner (one that tries to run outside of the running track) can bring the whole application down.

For this reason, the application menus that use KRunner to power its search usually white-list only the runners we consider safe.

KRunner - a new hope

One of the things I’ve been spending my time lately on is a possible new infrastructure design for KRunner. I took a couple of ideas that Aaron had (I’m not fond of Erlang as a language, but its communication model is unbeatable), forgot some of the others, and added a few of my own.

The idea is to make runners to be simple isolated nodes in the system that are just able to send and receive messages. Nothing more. Most of them should not even need to keep any internal state – they just need to be able to receive a request, and send back the response.

The fact that they share nothing with each other, and with the main application means that they can all be running in a separate process, or that different runners can be grouped into a few different processes based on their speed and stability. They could even all live in the same process, as long as they do not live in the same process as the UI.

This architecture will allow the crashy runners to crash as much as they want, the UI and the rest of the runners will continue their journey uninterrupted.

Querying other computers on the network

The fact that components are isolated and that they can live in separate processes, while only being able to send messages to each other has one really neat implication: They can even live on separate computers, and the messages can be exchanged through the local network.

This will allow the KRunner UI that lives on one computer, to return results from other computers as well. While this is not really useful for most runners (the calculator will calculate the same value regardless of the system it is running on), it would be really useful for runners like Baloo or Recoll which search the local file-system for files containing the search phrase. You’d be able to search for files on as many systems as you desire.

Naturally, the systems would need to be paired first for privacy reasons – you will not be able to search through other people’s data if they did not allow it.

Current status

So far, I’ve been working on the lowest level of the stack – on the library that will allow easy creation of isolated agents (or peers), and allow a network-transparent message passing between them. So far, it works reasonably well, the different instances of the system are able to discover each other either via KDNSSD framework, or, if it fails as it often does, via UDP broadcasts and by sharing the ‘contact information’ about other agents to the newly arrived one.

The cool thing is that none of the agents actually know whether they are talking to an agent from the same process, from the same computer, or from some other machine on the network.

Plans of mice and men

Now, is this one going to be the new KRunner engine? I hope so, but it first needs to become mature enough to even be considered as a replacement.

I’ll post progress updates from time to time, and screenshots when I actually manage to hack it into the KRunner UI. :)

One thing that always thrilled me is the idea that physics today is quite similar to software bugs triaging. You have a currently (or previously) accepted theory, and you make experiments that will show that the reality sometimes does not work according to that theory.

That is essentially what happens at CERN and the CMS experiment. There were suspicions (theories) that the universe contains more types of particles than we supposed at some time. In order to show that, physicists decided to build a large particle collider and measure monitor for events that don’t fit the previously accepted theories.

Now, when these ‘bugs’ in the fabric of reality are found, the scientists do not try to fix them (you can’t change reality), but rather to make a mathematical model that describes the real world better than the last one.

Fixing bugs in Plasma

This is where software developers differ. We also try to make collisions which will expose problems in our software, but we are allowed to change the reality and fix the bugs instead of ‘just’ writing a new specification of our programs that includes the bugs as its essential parts.

My main task at the Plasma sprint we had in CERN was to actually perform as many collisions I can with the activity switcher, and fix all the bugs that I could find. Sometimes the collisions exposed issues outside the switcher – in other parts of Plasma.

I am glad that I can say that I’ve fixed quite a few issues where most were not previously known. Plasma 5.6 will have a more streamlined activity switcher than it used to because its back-end was completely revamped, and it will be much more stable thanks to all the bugs that went away.

Plasma is for scientists

We usually tend to cater to ordinary users, and computer geeks, and tend to get excited every time we see a computer screen with KDE software displayed on some popular TV show or a movie.

But I have to say that all these screenshots that many of us collected during the years fade a bit in comparison to seeing Plasma running on most computers in the control centre of the CMS experiment.

Plasma, KWin, Dolphin, and all the shebang. :)