Sunday, September 2, 2007

Iconcache benchmarking results

In the last three days I've been fixing some last bugs in KPixmapCache, benchmarking it and analyzing the results. Here they are.

It's going to be a long post, so the short story is: the cache reduces KDE startup time by about 1 second (~ 9%), depending on how many and which applications are autostarted. Startup time of individual applications is somewhat reduced as well, but the exact numbers depend on the application (more specifically, on how many icons they use).

I benchmarked KDE startup on two computers: my desktop and laptop.
Desktop is custom built with AMD Athlon 2800+, 1Gb memory and SATA hard drive.
Laptop is Dell Inspiron 6400, with Intel Core Duo T2350 with 2Gb of memory.
Operating system on both computers is Kubuntu Feisty, with all updates.
KDE was pretty much 4.0 beta2, I updated SVN at 29/08 about 1400 UTC.
I used a bit customized Bootchart and kpc-stats patch for benchmarking.

For numerical data I used two things: first, CPU time which is amount of time that system was not idle, gotten from /proc/uptime.
Second, startup time reported by kpc-stats which is time elapsed from when the first KPixmapCache object is created (by kded4 at the very beginning of the KDE session) until when the last icon is found or inserted to the cache (at which point all applications should be started). This can be thought of as a perceived startup time, it shows how long it takes until KDE is ready for use.
I tested with both default KDE startup (referred to as bare KDE) which starts only Plasma, Kicker, KWin, KTip (and background programs) as well as "full KDE" which also starts Konqueror, KBounce, KMPlot and Dolphin and is meant to simulate full KDE session where many programs are started and/or restored.
Every case was ran 3 or 4 times and average time was used. Disk caches were dropped before each run.

The numbers:
CPU time: Bare KDE on desktop computer: 9985 vs 8933 ms, win is 1052 ms or 11,8%. On laptop 6717 vs 6088 ms, win is 630 ms / 10,3%.
Full KDE: desktop - 20636 vs 19525 ms, win is 1111 ms / 5,7%. On laptop - 9722 vs 8968 ms, win: 755 ms / 8,4%.
Startup time: Bare KDE on desktop - 13810 vs 12572 ms, win is 1237 ms / 9,8%. On laptop - 9889 vs 9075 ms, win: 814 ms / 9,0%.
Full KDE: desktop - 24905 vs 23690 ms, win: 1215 ms / 5,1%. Laptop - 13138 vs 11965 ms, win: 1174 ms / 9,8%.

Some bootchart images:

Bare KDE startup on desktop, without (left) and with (right) iconcache.

Full KDE startup on laptop, without and with iconcache.

You can see from the numbers that the icon cache gives almost 10% improvement. Another interesting observation is that improvement in full KDE session isn't much bigger than in bare KDE. On desktop the improvement is pretty much the same, only in laptop is it a bit bigger with full session. Possible reason is that laptop's CPU is faster and thus I/O becomes bigger bottleneck, especially as it's HDD is also slower (5400 vs 7200 rpm).
Most of the 1-second improvement probably comes from the fact that icon theme initialization, which scans all icon dirs, can be skipped when icon cache is present because needed info is already stored in the iconcache header.
You can visually see the improvement when flipping between bootchart image pairs. On with-iconcache images, the later part of the graph has shifted about 1-1.5 seconds forward. You can also see there that the 1-second improvement comes from kded process which is where the expensive icontheme initialization takes place.

I didn't benchmark the applications again since I have already done so earlier (and the numbers shouldn't be really different) and because it was more painful to do than benchmarking full KDE.
Only exception was Dolphin which was the first app I tried to benchmark (after that one I didn't bother with the rest) and which showed a 400ms (11.4 vs 11.0 sec) improvement in cpu time and 1.5 sec (12.6 vs 11.1 sec) improvement in startup time. This fits with my earlier benchmarks quite well. Also note that the earlier benchmarks were all done on the laptop while this one was done on desktop, that's why the absolute times are so different.

Last but not least, keep in mind that absolute performance improvements aren't the only advantage the icon cache brings. It also makes some things possible which would have been way too slow without it. Also there's the generic pixmap cache which is already being used in quite a few games to speed up rendering of game elements and background from SVGs.

And finally, since this is quite likely my last GSoC-related post, I'd like to thank my mentors and the whole KDE community for helping and supporting me as well as Google for organizing the SoC and supporting open-source development.
Keep up the great work everyone! :-)


Anonymous said...

Hang on there, am I missing the obvious or is KDE4 going to be way slower than KDE3 generally? 11.1 seconds to start Dolphin when I just tested 0.5 seconds here (warm start but I don't think the difference would be so much). 12 - 20 seconds to start KDE4, that is no better than KDE3 when I thought there were going to be speed improvements.

Rivo Laks said...

Actually the cold disk cache makes quite a difference. All the libraries etc have to be reloaded as well.
Also the 11 seconds is until the time Dolphin's UI has stopped loading icons. The window pops up much faster, but then it loads the directory contents which takes up some additional time.

Anonymous said...

Well, here on Dapper, kde 3.5.4, Dolphin 0.8 shows up from a cold start in ~2 seconds. 1 second from a warm start.
Anyway, good work with the cache!

Anonymous said...

Don't compare absolute numbers!!
First of all, I bet that Rivo has quite a lot of debug symbols and debugging code still compiled in that are not in the final release.
Second; the icon cache replaces a LOT of disk-seeks with just a couple. Thats practically speaking all it does. So if you have a slow HD or not a lot of memory for dir caches, it helps you quite a lot.

But please don't take these numbers as being an example of how fast KDE4 in general starts.


Anonymous said...

Is there a testing project or a test suite for KDE start up time?

Anonymous said...

How do you drop disk cache?

I want to test my application but I haven't found any way to flush disk cache.

Rivo Laks said...

There isn't a special testing suite AFAIK. I just took Bootchart and had it start the startkde script and collect statistics for some seconds. I guess I might put a tarball with my scripts somewhere if people are interested...

Disk caches can be dropped by doing
echo 3 > /proc/sys/vm/drop_caches
See for more info

Anonymous said...

Thank you very much for donating you time to the KDE project and producing such excellent code. I hope that you will continue with the KDE project and that other sub projects can benefit from your time.

Ian Ventura-Whiting

Sebastian Sauer said...

Cool thing. Thank you Rivo!

From someone who has a pretty cool (but this days slow compared to others) Compaq Armada with 500MHz and 128MB RAM loaded with KDE 3.5.6 and who will profit a lot from this.

Hans said...

Excellent. Even though it doesn't sound very much, I'm sure the overall KDE experience will feel much better with this. And as you've written, it isn't only about performance.

>> Also there's the generic pixmap cache which is already being used in quite a few games to speed up rendering of game elements and background from SVGs.

Sorry for stupid question, but how does this work? I guess the games might start faster, but if you resize the window, there will still be a "lag"? (notice quotation marks. I hope you understand what I mean).