Iconcache benchmarking results

In the last three days I've been fixing some last bugs in KPixmapCache, benchmarking it and analyzing the results. Here they are.

It's going to be a long post, so the short story is: the cache reduces KDE startup time by about 1 second (~ 9%), depending on how many and which applications are autostarted. Startup time of individual applications is somewhat reduced as well, but the exact numbers depend on the application (more specifically, on how many icons they use).

I benchmarked KDE startup on two computers: my desktop and laptop.
Desktop is custom built with AMD Athlon 2800+, 1Gb memory and SATA hard drive.
Laptop is Dell Inspiron 6400, with Intel Core Duo T2350 with 2Gb of memory.
Operating system on both computers is Kubuntu Feisty, with all updates.
KDE was pretty much 4.0 beta2, I updated SVN at 29/08 about 1400 UTC.
I used a bit customized Bootchart and kpc-stats patch for benchmarking.

For numerical data I used two things: first, CPU time which is amount of time that system was not idle, gotten from /proc/uptime.
Second, startup time reported by kpc-stats which is time elapsed from when the first KPixmapCache object is created (by kded4 at the very beginning of the KDE session) until when the last icon is found or inserted to the cache (at which point all applications should be started). This can be thought of as a perceived startup time, it shows how long it takes until KDE is ready for use.
I tested with both default KDE startup (referred to as bare KDE) which starts only Plasma, Kicker, KWin, KTip (and background programs) as well as "full KDE" which also starts Konqueror, KBounce, KMPlot and Dolphin and is meant to simulate full KDE session where many programs are started and/or restored.
Every case was ran 3 or 4 times and average time was used. Disk caches were dropped before each run.

The numbers:
CPU time: Bare KDE on desktop computer: 9985 vs 8933 ms, win is 1052 ms or 11,8%. On laptop 6717 vs 6088 ms, win is 630 ms / 10,3%.
Full KDE: desktop - 20636 vs 19525 ms, win is 1111 ms / 5,7%. On laptop - 9722 vs 8968 ms, win: 755 ms / 8,4%.
Startup time: Bare KDE on desktop - 13810 vs 12572 ms, win is 1237 ms / 9,8%. On laptop - 9889 vs 9075 ms, win: 814 ms / 9,0%.
Full KDE: desktop - 24905 vs 23690 ms, win: 1215 ms / 5,1%. Laptop - 13138 vs 11965 ms, win: 1174 ms / 9,8%.

Some bootchart images:

Bare KDE startup on desktop, without (left) and with (right) iconcache.


Full KDE startup on laptop, without and with iconcache.

You can see from the numbers that the icon cache gives almost 10% improvement. Another interesting observation is that improvement in full KDE session isn't much bigger than in bare KDE. On desktop the improvement is pretty much the same, only in laptop is it a bit bigger with full session. Possible reason is that laptop's CPU is faster and thus I/O becomes bigger bottleneck, especially as it's HDD is also slower (5400 vs 7200 rpm).
Most of the 1-second improvement probably comes from the fact that icon theme initialization, which scans all icon dirs, can be skipped when icon cache is present because needed info is already stored in the iconcache header.
You can visually see the improvement when flipping between bootchart image pairs. On with-iconcache images, the later part of the graph has shifted about 1-1.5 seconds forward. You can also see there that the 1-second improvement comes from kded process which is where the expensive icontheme initialization takes place.

I didn't benchmark the applications again since I have already done so earlier (and the numbers shouldn't be really different) and because it was more painful to do than benchmarking full KDE.
Only exception was Dolphin which was the first app I tried to benchmark (after that one I didn't bother with the rest) and which showed a 400ms (11.4 vs 11.0 sec) improvement in cpu time and 1.5 sec (12.6 vs 11.1 sec) improvement in startup time. This fits with my earlier benchmarks quite well. Also note that the earlier benchmarks were all done on the laptop while this one was done on desktop, that's why the absolute times are so different.

Last but not least, keep in mind that absolute performance improvements aren't the only advantage the icon cache brings. It also makes some things possible which would have been way too slow without it. Also there's the generic pixmap cache which is already being used in quite a few games to speed up rendering of game elements and background from SVGs.

And finally, since this is quite likely my last GSoC-related post, I'd like to thank my mentors and the whole KDE community for helping and supporting me as well as Google for organizing the SoC and supporting open-source development.
Keep up the great work everyone! :-)

Iconcache is Done :-)

The coding phase of the Summer of Code is now over (for two days already).
Within the last days I managed to add mmap-implementation which instead of opening/reading/closing cache files all the time maps them into shared memory and then just uses that memory. It seems to increase performance (of reading icons from the cache) by about 30%. More than I anticipated.

In the end I managed to get in all the features and ideas that I had, so the work on the icon/pixmap cache can be considered complete now. There might of course be some smaller performance tweaking and bugfixing and apidocs additions and so on, but I don't expect to do any bigger changes.
It also means that if you see a bug concerning icon loading or pixmap cache (I hope you won't ;-) then report it. Either contact me directly or add a comment here or whatever.

I still haven't gotten to doing the final benchmarking... maybe at the end of this week or in the next one...

Iconcache status

Summer of Code is ending (the official deadline for writing code is 20th August) and so my work on the icon cache is also coming to an end.

Yesterday I committed the last crucial piece of code. The cache now periodically checks for icon theme updates, so as soon as you install some new icons, they should be used and the obsolete cache be discarded.
This means that the cache is now fd.o icontheme spec-compliant and all the necessary features should be implemented :-)

Some smaller things remain to be done, such as further performance improvements and final benchmarking. I hope to complete the code part in the coming days and most of benchmarking will probably be done after the 20th.

Speaking of benchmarks, I did some a few days ago. As you have probably heard already, perhaps the most extreme case is kfind which starts up in about 4 secs without the cache and in a bit less than 1 sec (of which just 300ms is icon loading) with the cache. That's with hot caches (i.e. operating system has cached some of the data already). With cold caches it's 12secs vs 2.5secs.
Dolphin uses quite a few icons as well. With cold caches it takes 3.9 secs with iconcache and 5.2 secs without it to start. With hot caches it's 1.2 vs 1.5 secs.

I also noticed about one second improvement (12 vs 13 secs) in full KDE session startup time (with Dolphin, Konqueror, Konsole and KBounce autostarted). I have to admit it's a bit less than I had hoped for, but KDE's startup time will probably be improved prior to 4.0 anyway. Anyway I will make further benchmarks to find out more accurate numbers.

The times are measured from when KPixmapCache contructor is called (which should be at most 100ms after the program is started) until the last find() call (after which the application should be ready to be used).

Icon/pixmap cache now in kdelibs

As the title says, I've finally merged my Summer of Code project - icon and pixmap cache - into kdelibs.

Buildsystem changes are not done yet, I'll do them in the next days. Until then, you might see old icons after new ones are installed. If you really need the new ones, do touch -m $KDEDIR/share/icons/hicolor/. Then the cache will notice the new icons.

The pixmap cache is meant to make disk caching of e.g. pixmaps rendered from SVGs very easy for app developers. It's API is similar to that of the QPixmapCache and I'm planning to create a TechBase tutorial soon to make it's adoption even easier.

aKademy impressions

I'm one of the many KDE people who spent the last week in Glasgow, at aKademy 2007. It was a really great event and the first chance for me to meet many KDE people in real life.

The arrival went quite smoothly. I arrived late Sunday, a day after the incident at the Glasgow International. Luckily there weren't any problems there, although one could see higher security measures such as concrete blocks before the main entrance and a group of police with SMGs.

The problems started once I reached my hostel. I was supposed to be in a room with three other SoC students. But as it turned out, there were four others instead of three so my bed was already taken. After some searching they put me in another room. In the morning I was told I'd stay in that room until my departure, but to my surprise all my things had been removed when I went back in the evening. After talking to the staff again, they gave me yet another room. And after staying in two different rooms for two nights, they finally told me I should go and talk to the organizing team as they really don't have a place for me. So I went and talked to Kenny Duffus who put me into the university accommodation which was much more convenient and more importantly absolutely problem free. Big thank you to Kenny Duffus and the rest of the aKademy team for sorting that out (not to mention organizing this great event) :-)

Other than the hostel issue, everything was great. I met many KDE people, including my mentors Seli and aseigo whom I brought some chocolate for their good work (and almost made aseigo cry in the process ;-)). We also had a nice barbeque out of the city on Thursday although the end of it was somewhat ruined by the rain (yeah, it rained quite a bit in Scotland).

Tons of work also got done. There was a heap of additions to KWin which probably deserve their own blog post. Also, there were some very important additions to my icon cache. It got split into two classes: KIconCache, which is the KDE-wide icon cache, and KPixmapCache which provides application-specific cache functionality. This way e.g. applications that use SVGs so that they can cache the generated pixmaps without having to regenerate them every startup. The code is available from KDE SVN, under /branches/soc-iconcache/. It contains kdeui replacement (which I'm trying to keep mostly up-to-date with the trunk), a demo app showing how to use the app-specific caches as well as an old visualization app which was meant to list all entires in the icon cache but doesn't work anymore due to file format changes and me being too lazy to update it...
I've still got some hope to get this into 4.0 as there certainly would be apps in kdeedu and kdegames that would make use of it. And probably Plasma as well. We should know for sure in a week or two.

Introduction

Hello everyone.

Short introduction: I'm an Estonian student, soon finishing my first year in university. I've been involved in open-source development for several years, working on the Boson game in the past though unfortunately it's development has halted for now.

Since the beginning of this year I've been involved in KWin development, creating various compositing effects as well as separating effects interface into a library to support 3rd-party effects. Stay tuned for more KWin-related stuff soon.

Last but not least, I was accepted into Google's Summer of Code program and will be working on an icon cache, mentored by aseigo. The purpose of it is to make application startup faster by putting icons into cache which eliminates the need to search for them on the disk (there are many directories where the icons might be and searching through them takes time). Also it will act as SVG cache so that SVG icons can be used without having to convert them into raster format every time.

This blog will contain news about KWin, my GSoC project and anything else KDE-related that I might be working on in the future.