Ivan Čukić

dr Ivan Čukić

Author of "Functional Programming in C++", KDE developer and Free/Libre software enthusiast

For inquiries about KDE projects (requesting features, submitting bugs, asking general questions etc.), you can use:

KDE Community Forum
Bug reporting site
#plasma IRC channel

You need to contact me directly?

KDE related:
ivan.cukic [at] kde.org

Non KDE related things:
ivan [at] this domain

IRC Network: libera.chat
Nick: ivan|home
Channel: #plasma

Projects

Blog categories

Functional Programming in C++

If you like C++, you might be interested in my book. It contains quite a few gems of modern C++ programming.

It is available for purchase at:

Patronage

Time to rely less on MySQL?

Published by Ivan Čukić in the KDE development section, on 18 August 2012

KDE software tends to require mysql as the database engine (either a hard requirement like Amarok, or recommended backend like in Akonadi) so things like these genuinely worry me:

Disappearing test cases or did another part of MySQL just become closed source?

Thoughts? MariaDB? PostgreSQL? …

p.s. I don’t have influence on amarok nor akonadi, just want to see what the people think

Comments:

Kollya
why not Firebird?
http://www.firebirdsql.org/
momek
Mysql dependency is one of the reasons, why some distribution, can no longer make CD installer.
Is there any other music player using mysql?
GSoC 2012 - http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/claudex/1
Trueg post about Nepomuk in Akonadi - http://trueg.wordpress.com/2009/02/19/akonepomuk-neponadi-friendly-takeover-or-real-love-marriage/
One more vote for nepomuk/virtuoso for all.
Ivan Čukić
@<a href="#comment-27807" rel="nofollow">Enrique Dans</a>: See http://amarok.kde.org/blog/archives/812-MySQL-in-Amarok-2-The-Reality.html regarding sqlite.
@<a href="#comment-27808" rel="nofollow">MariaDB</a>: Postgre slow? Any links to back that up?
@<a href="#comment-27810" rel="nofollow">Stan aka ‘Xeno’</a>: See the link above.
@<a href="#comment-27811" rel="nofollow">genstorm</a>: gentoo people sometimes astonish me :)
@<a href="#comment-27812" rel="nofollow">hananoko</a>: virtuoso is a sql database as well as sparql. So, it shouldn't be that much of a problem. Guessing it has more sql compatibility than mysql anyways.
@<a href="#comment-27813" rel="nofollow">hron84</a>: Not a valid comparison definitely. The second thing is that android developers are not /satisfied/ with sqlite, they don't really have a choice. The last time I checked, android's sqlite (java api) didn't have updateable cursors. Noone with any database programming experience would be satisfied with that.
@<a href="#comment-27815" rel="nofollow">Lysander</a>: Not a bad idea :)
destiel
@momek: +1 for nepomuk
*sql dependency are too big; archlinux:
libmysqlclient-5.5.27-3 mysql-clients-5.5.27-3 mysql-5.5.27-3 = 115MB
libmariadbclient-5.5.25-4 mariadb-clients-5.5.25-4 mariadb-5.5.25-4 = 155MB
genstorm
@<a href="#comment-27813" rel="nofollow">hron84</a>: You think that's a valid comparison?
Lysander
Why rely on one backend? Why not use a *good* ORM *and* SQL-Abstraction Framework? Furthermore you should evaluate, whether (most) of your data is really easy to put into a schema - for arbitrary data with emphasize on 1:n relations a NOSQL-DB is imho the better choice.
Much things to think about ;-)
Trapanator
Yes, please allow us to choose which SQL backend to use.
Alternatives:

- PostgreSQL
- SQLite
Enrique Dans
Using MySQL as a storage backend for a desktop application has always been a complete overkill, but you KDE developers don't seem to understand that.
Should you REALLY need a storage database (AND MAKE SURE YOU REALLY NEED IT!) you MUST choose SQLite.
MariaDB
MariaDB. PostreSQL is too slow and this will be noticeable with large collections.
hananoko
It would be nice if KDE could stick to one extenal database, Virtuoso. It would be interesting to know how it would work as Akonadi DB as if I'm not mistaken it would need a lot of changes.
hron84
I highly recommend to use SQLite. Using database _server_ for a desktop application is a nonsense. And if the Android developers are satisfied with SQLite, then it is good enough for KDE, too.
Stan aka 'Xeno'
For Amarok, why not store things via nepomuk/virtuoso just like Bangarang does.
For Akonadi - i'm using Postgresql backen for over a year and it works great. BTW thre was a idea to move at some point db part to virtuoso too.
genstorm
postgresql works really fine here for the akonadi backend.
I remember there was a loud outcry when the postgresql option was removed from amarok-2, several issues with embedded mysql and the sqlite backend being just too slow for huge collections.
point given back then simply was lack of time to maintain 3 different db backends.
PS: since mariadb is able to serve as binary drop in replacement for mysql, you can already use it as the backend for amarok, akonadi etc. - in fact the Gentoo dependency for KDE SC is virtual/mysql which lets you choose either dev-db/mysql or dev-db/mariadb
genstorm
Oh puh-lease, CD installers are so 90s... :)
No one in their right mind should rely on package versions on an install disc these days, they are outdated on the day of release already. An ideal install disc should fire up a minimal environment for the setup and pull in all recent packages via the package manager.
Offline folks, they will need a DVD anyway as _tons_ of other stuff can't be included on a CD.
Now for amarok, read the link provided by Ivan Čukić.
Ivan Čukić
@<a href="#comment-27819" rel="nofollow">destiel</a>: Can you check the same for libmysqld (if arch has that one - library for embedded mysql)
destiel
no official libmysqld package for archlinux http://www.archlinux.org/packages/
genstorm
@<a href="#comment-27822" rel="nofollow">Ivan Čukić</a>: for me that's 9.5M
And a pretty stripped down mariadb-5.2.12 tarball comes down to 28M
maninalift
I am no kind of ex[pert, but I nevertheless take up your invitation to chip-in.
BerkeleyDB unless the application requires SQL in which case HSQLDB, seems like a reasonable choice to me.
Funo
Do we really need a RDBMS? I always wondered how KDE software use a RDBMS, I'd say a noSQL database would be lighter and more appropriate to applications' uses.
Even in web development, mysql is being used only because it's practically a standard, not because the application need its feature.
CajunArson
How about: 1. Akonadi: My address book should *not* require a full-blown SQL database!! Really! I know, you're going to say "But what if I have 10,000,000 contacts in my address book! You need a SQL database to scale that big! Wahh!"
Well: 1. If you do have that many contacts you probably are a spammer. and 2. If you are in a large corporate setting where there is a legitimate reason to have that many contacts then you'll have large-scale email servers that could host a DB like that externally, but my *FREAKIN TABLET* should not have to run MySQL!
My favorite startup command for KDE is "akonadictl stop" now. Gmail won my email even though I did use (and like) Kmail way back in the day.

2. Amarok: Considering Amarok's reason for existence seems to be to make the most bloated music player possible, MySQL is more justified in this context. Having said that the best music player for KDE I have is yakuake + C*mus + keyboard bindings. That's why I love KDE: you can set it up to do what *you* want it to do.</div>
Ivan Čukić
@<a href="#comment-27828" rel="nofollow">CajunArson</a>: Dude. if you don't have anything useful to say, please don't just troll.
1. The database is not there for the contacts, but for mails mostly. Only on my gmail account I have 14000 of those, and I wipe out my mail on a regular basis.
2. Not a bad alternative.
momek
@genstorm
And what about this link? For "some" reason no one else uses mysql for music player. Why can't you understand, that mysql is too much for desktop applications?
genstorm
@<a href="#comment-27830" rel="nofollow">momek</a>: I don't understand because it works very, very well on my very aged laptop. :)
Diederik van der Boor
<div class="comment-good">@everyone suggesting SQLite.. It was used in the past by KDE, and it didn't perform well, nor did it provide all required features. Maybe this has changed recently, but that's how projects ended up with MySQL-embedded.
Note that's not a full-blown SQL server, but embedded configurationless system like SQLite is. In some blog posts you may find the whole story behind it.</div>
stanley miller
Nobody using Open Suse seems to care... Six votes.
https://features.opensuse.org/312858
"Given the Oracle ownership of MySQL and their move to an OpenCore development model it is time OpenSUSE transitions away from MySQL as the default database to the MariaDB."
Dylon
PostgreSQL is absolutely not slow! If you look at the dates of the benchmarks between PostgreSQL and MySQL online, you will notice that they are very much out of date, in fact, the performance difference between PostgreSQL and MySQL, now, is negligible and you get much more bang for your buck with Postgres. I develop against Postgres daily, and find it quite enjoyable.
My vote is with PostgreSQL, but one's application should really not be dependent on any specific DBMS, unless one has a good reason to do so. If the one's application will not be dependent on proprietary functionality, why should one tightly couple one's application with a particular DBMS? Instead, one should use a database-independent data access layer, which will vary according to which language(s) you are developing in.
orangotango
Since MySQL was a dependancie to run Amarok, then I removed Amarok. Too bloated for a desktop that aims to be usable. A "KISS" suggestion to developers ;-)
¿Va a cerrar Oracle MySQL?
[...] curiosidad, Ivan Cukic, uno de los desarrolladores de KDE, se hace eco de esta noticia en su blog, preguntándose si ha llegado el momento de dejar de depender tanto de MySQL (por ejemplo, en [...]
Alejandro Nova
Please, use Virtuoso for everything (SQL mode for Akonadi, SPARQL mode for Nepomuk). Don't rely on MySQL. We simply don't need two, or three, or four, concurrent full database systems.
Better yet, do an API to enable external programs to store info in a centralized database and make that centralized store a Nepomuk-enabled one. With that, you could say good bye to Amarok and Digikam embedded MySQL instances.
Luiz
Don't use MySQL, MariaDB, PostgreSQL... or a 20 node Hadoop cluster. Those things are not suitable for desktop applications.
Linux Caracas - ¿Va a cerrar Oracle MySQL?
[...] curiosidad, Ivan Cukic, uno de los desarrolladores de KDE, se hace eco de esta noticia en su blog, preguntándose si ha llegado el momento de dejar de depender tanto de MySQL (por ejemplo, en [...]
Ivan Čukić
Please read Diedrik's answer above regarding amarok and 'full-blown sql server'.
MySQL -> TheirSQL? | Lindeas
[...] се съмнения в обещаната от Oracle политика към MySQL, когато ги [...]
Konstantin Tokarev
PostgreSQL is definitely high quality software, but it's more suitable for enterprise application then for desktop.
I'd recommended you to go with SQLite, or even NoSQL solutions like Redis or LevelDB
taz
Use PostgreSQL.
Francesco R.
not read all the every application which require a database backend should be able to use these three: - SQLite - Mysql, a clone (MariaDB, Percona) or fork (drizzle) - Postgres The problems are: - program for muliple databases is costly, every developer know just one (most know mysql or sqlite) - Abstraction layer sucks and badly, every db has optimization which are totally missed with abstractions and make an XML database appear like heaven.
Others database exist but for a reason or another they are more a problem than a solution: BerkleyDB for example change ABI at every version, not to mention is now Oracle (right?).
P.S. I'm the introducer for the virtual/mysql in gentoo, something like 6 years ago ;-) happy to see it's useful now
Jason
Solution:
QUIT MAKING KDE RELY ON A FULL FUCKING BLOWN DATABASE SERVER FOR A SIMPLE DESKTOP ENVIRONMENT.
Now get off my lawn.</div>
Links 19/8/2012: SolusOS Eveline 1.2 Released, Unity Favours 3-D | Techrights
[...] Time to rely less on MySQL? KDE software tends to require mysql as the database engine (either a hard requirement like Amarok, or recommended backend like in Akonadi) so things like these genuinely worry me: [...]
ElReyHelado
I think the DB must be a option for the user along their current PC: 1. A noSql DB for legacy PC's 2. Postgresql for entusiast users.
That must be good. For me.
mariuz
I can suggest Firebird embedded , you don't need to start a service for it http://www.firebirdsql.org/manual/fbmetasecur-embedded.html
I could help with optimizing the queries you have also i would drop virtuoso and migrate it to firebird
Ivan Čukić
@<a href="#comment-27874" rel="nofollow">mariuz</a>: Virtuoso supports RDF/SPARQL - that is the only reason Nepomuk didn't use some more common DB.
So, Nepomuk really *is* tied to it.
It would be cool to find a nice comparison of embedded firebird and embedded mysql.
Ivan Čukić
@<a href="#comment-27880" rel="nofollow">moltonel</a>: Definitely an answer worth framing.
Virtuoso is not the only RDF database, but /is/ one of the better ones.
Vin
SQLite is a good solution as "default" backend, but is more important IMHO to leave the user to choose the backend database server (using QtSql and don't rely on SQL extension).
moltonel
<div class="comment-good">Obviously still a lot of heartfelt gut reactions on that topic, after a few years. This calls for technical, researched decisions, not an opinion poll.
Some mythbusting: * mysql is used by KDE in "embedded" mode, that means no extra process, just library calls, exactly like SQLite works. * The original amarok devs really wanted and tried to use SQLite, but gave up mainly for performance reason. It's probably worth reasessing that today. * PostgreSQL and MySQL have very comparable performance. Neither can laugh at the other. Any benchmark win by one over the other is for a specific usage pattern in a specific configuration, too specific to be used as an argument to choose "the default KDE db" as a whole. * Postgres lacks an embeded mode (and implementing one is not going to happen anytime soon, unless it is restricted to one-query-at-a-time), and that was a show-stoper for amarok devs at the time. Lack of an embeded mode obviously wasn't a show-stoper for the use of virtuoso, though.
Some opinions: * If nothing else changes, MariaDB should be used instead of MySQL. It's a drop-in replacement with proper backing, wereas Oracle has demonstrated so much contempt and bad decisions that I would now stay away from any product they touch. * PostgreSQL is overall a better db than MySQL. No nonsense gotchas, cleaner code, better legal/comercial status, a good few more features, smaller package, better documentation, clear versioning, backport, and release policies... I had a teenage love with MySQL, but moved on and never looked back. * Virtuoso is good for RDF data (being the only db that handle this type of data is an obvious plus), but I'm afraid it would perform horribly for other types of uses. On my system, 90% of program multi-second freezes are due to waiting on virtuoso. I'd love somebody to prove me wrong, though. * NoSQL is not the answer, at least not a universal one. Look at your data and usage patterns, and *then* choose a db, not the other way around. Oh, and "NoSQL" is as well-defined as "cloud" these days, I wish people would stop using that term altogether and start using product names, or at least architechture types.</div>
¿Va a cerrar Oracle MySQL?
[...] curiosidad, Ivan Cukic, uno de los desarrolladores de KDE, se hace eco de esta noticia en su blog, preguntándose si ha llegado el momento de dejar de depender tanto de MySQL (por ejemplo, [...]
moltonel
The problem with enabling support for multiple SQL engines is that it is *hard*. If you "avoid sql extensions", or more precisely "dont use features not available in all supported backends", you'll find that the lowest common denominator is frustratingly small, and will only allow you the simplest queries. Sometimes a query is accepted by an engine, but behaves in a nonstandard way (MySQL is particularly guilty of that). Even if a particular query works in all engines, chances are that for one of the engines it underperforms horribly and should be writen a different way. ORMs only make that kind of problem worse.
If your queries are straightforward enough to be supported by all engines with a similar performance profile... Lucky you :) Otherwise, you'll either have to a) use braindead queries or an ORM, and pay the performance price, b) write different code for different engines, and pay the maintenance price, or c) focus on a single engine, enjoy the speed/features, and pay the portability price. YMMV.
Ivan Čukić
@<a href="#comment-27917" rel="nofollow">Chaser</a>: Amarok does use a lightweight embedded database that is faster than sqlite. And it can be used with a remote server as well.
I agree that it would be awesome to have a choice of databases, but it is a hell to maintain, and there aren't enough developers for that.
Chaser
I think, that support multiple databases is the better choice, because: 1. It's the UNIX-way. Having alternative always is good idea. 2. There are computers with ~512 RAM. Not only old desktops, but also SoC such as Rasperry Pi. So, everyone should be able make a choise - multiuser large database with network support or light embedded db with minimal memory overhead. 3. Not everyone want to share DB over network ;) 4. Old Amarok worked on old hardware. Are there really reasons to buy new hardware for new music player? How about make it more fast and lightweight?
FACORAT Fabrice
<div class="comment-good">People are talking about desktop usage, but what about workstation. I do work for a small enterprise where all workstations/servers/laptops are running under Linux. For workstations we are using /home on NFS ( v3 ). Mails are fetch using POP protocol. Since migrating to latest KDE release using Akonadi : kmail is broken. - MySQL won't start because of locking issues ( whereas every others apps works fine, notably firefox ). On top of that it is not advised to use MYSQL on NFS - migrating to SQLite did not seem to have much success either.
So what can I do : migrate to NFSv4 hoping that MySQL or SQLite will work ? switch to another network FS ? use a central MySQL server ? Migrate to a web based groupware ( and finding how to import back in the mail server several Gb of users mails in their $HOME) ? Switch to another mail client ( and how to import kmail mails + folders hierarchy ) ?
All of this is not cheap for me :(</div>
Ivan Čukić
@<a href="#comment-32697" rel="nofollow">FACORAT Fabrice</a>: Another comment that deserves to be in focus.
I'd guess you need to ask all that people that work on Kontact. While the rest of us are mostly thinking about desktop, I assume they are not.
This is the most I can help you since I have no code in KDEPIM.

Write a new comment...