Question

What is the output of the following code:

QString separator = "/";
auto date = "14" + separator + "7" + separator + "2013";
separator = " ";
auto datetime = date + separator + "17:32";
qDebug() << datetime;

Scroll down…

Expected output: "14/7/2013 17:32"
Actual output:   "14 7 2013 17:32"

What happened?

In his great talk, Volker mentioned the optimizations of QString concatenation via the auto, expression templates and QStringBuilder that allow to do a single memory allocation for all the concatenations in the example above.

I just wanted to repeat a warning I gave to the audience - for the people who were not present.

In this case, auto allows the lazy evaluation of the result (which provides the actual optimization). “Lazy” means that the separator value is only taken at the point of the actual evaluation which happens only when the QStringBuilder is being converted to a string in order to be able to output it via qDebug. At that point, the value of separator is ” “, and the old value”/” has been forgotten.

The lazy evaluation is usually a trait of pure functional languages (Haskell) where there is no mutable state. When you have the mutable state, you need to really pay attention whether you are using an expression template-based API.

When?

This mechanism is only used when either QT_USE_FAST_OPERATOR_PLUS or QT_USE_QSTRINGBUILDER are defined. And those are, by default, in KDE Frameworks 5.

Ivan Čukić
@Syam
QString uses CoW. If the operator+ took a value instead of a const-ref, the method would get a cheap shallow copy, but it would still be more expensive than the current version.
Disabling copying would not help - this example doesn't really need the second auto at all - just changing a string that is involved in the concatenation will have the same effect.
Links 16/7/2013: Linux 3.11 Name, Another NSA Scandal | Techrights
[...] String concatenation in Qt5/KF5 [...]
Syam
@Ivan
After reading a bit about QStringBuilder, I agree with you. This has nothing to do with the compiler. The 'optimized' implementation of the string builder has the problem similar to reference counting. But here there's no mechanism to implement something like CoW. If we had something like that, we could avoid lazy evaluation when the referenced strings are modified.
To handle the auto issue, would it be possible to disable the copy constructor for the builder class? I haven't given a lot of thought about it though.
Ivan Čukić
"This is the same programming model as that of C++" ~ Johan Ouwerkerk
Let me reiterate - it is a destructive update for the value. Not for the reference. So, if you pass a function a reference, everything is ok - the reference is not changed - no reason for the compiler to intervene. If the function accepted a value instead (as it arguably should), this behaviour would be different (with no surprises for anyone).
Essentially, the compiler can (and does) ensure that the thing you passed to the function doesn't change. Much like in the Haskell case.
It is not a bug of C++. It does exactly what it should. It /could/ be marked as a bug of Qt, but the reason I don't see it as such is that the point of the ...FAST_PLUS... is the performance. And if you want to go for it, you should go all the way and be aware of the potential problems. That is the reason I posted this in the first place.
Jacky Alcine
This could break a lot of code; looks like I might be avoiding the "auto" keyword when using Qt5.
Giuseppe D'Angelo
I believe it's actually even worse than that:
QString foo() {
return QString::number(42);
} void bar() {
auto x = "123" + foo();
// enjoy the fireworks
}
I think the cause of this is the QStringBuilder's ``feature'' of storing references to the strings it's building up, instead of (shallow) copies. I have a wonderful proof of that, but this margin is too easy, so I'll go out and enjoy Bilbao together with other KDE hackers. :-)
Johan Ouwerkerk
Please note: it is not lazy evaluation what triggers the bug. It is **broken** implementation of lazy evaluation what triggers it.
Correct lazy evaluation does not implicitly throw away side effects, and destructive update is very much fraught with side effects. Therefore a correct implementation of lazy evaluation would force evaluation of the expression at each assignment, simply because in C++ that is a destructive update.
That C++/the compiler does not enforce this means it is broken by design; it does not mean "lazy evaluation" causes or triggers a bug.
For example in equivalent Haskell code (abusing the I/O monad for simplicity) the bug does not occur. Reason is that the compiler/language actually does understand and implement lazy evaluation properly (including forcing evaluation to meet constraints imposed by the need for correctness):
sep1 :: IO String sep1 = return "/"
sep2 :: IO String sep2 = return " "
date :: String -> IO String date sep = return ( "14" ++ sep ++ "7" ++ sep ++ "2013") datetime :: String -> String -> IO String datetime date sep = return ( date ++ sep ++ "17:32")
test = do
separator <- sep1
d <- date separator
separator <- sep2 -- destructive update
dt <- datetime d separator
putStrLn dt
Girish
This seems broken. To store a reference is a mistake in QStringBuilder because '+' is a value based operation. Has this been reported to the Qt devs?
Syam Krishnan
I always thought optimizations are allowed as long as they don't change the 'observable behaviour' of the code. This one here then is clearly broken.
Ivan Čukić » String concatenation in Qt5/KF5 (contd.)
[...] were quite a few comments on my last post. I’d like to clear up a few (imo) misconceptions [...]
Ivan Čukić
@Johan Ouwerkerk
Haskell is a special beast. Though, there is a significant difference between that code and the C++ one. QStringBuilder takes the reference to the QString instead of copying the value. Again, in the name of efficiency. So, since the reference did not change, there was no destructive behaviour in the code.
@Syam Krishnan
Optimizations that need to be explicitly enabled can drag in side-effects that you need to know of.
Johan Ouwerkerk
@Ivan Čukić
Yes, Haskell is quite a beast; however the crucial distinction is not really reference vs. value based semantics of the assignment. Indeed you don't really know what it actually is in Haskell since this is hidden by the language & compiler for you. However what matters is the contract of the code.
And here the important part of the contract is in the semantics of a `do' block. Semantically a `do' block is a sequence of events/actions. Regardless of how the underlying implementation might choose to optimise the actual execution order, at the Haskell layer of abstraction a `do' block represents sequential in-order execution of statements. This is the same programming model as that of C++.
The bug in the lazy evaluation is not in the value-v-reference red herring. After all there is most definitely a destructive update inside the referred-to separator object. The bug is that the introduction of lazy evaluation causes the observable order of program execution to violate the C++ sequence-of-statements model. Effectively one statement is "hoisted" and dropped as a term in another statement, and a second statement is deleted entirely -- but note that there is no code or mechanism in C++ which actually asks the compiler to do that. For correctness, this must never be allowed to happen: compiled code is plainly not doing (observable behaviour) what the source code specifies which is plainly incorrect behaviour. This is the sort of thing that a compiler is supposed to avoid by enforcing the semantics of things like "a statement" and "statement delimiter". It is arguably a bug in C++/compilers and not in Qt but it is nevertheless a bug.