changeset 58:0cd05745be92

add a blog entry about the portage metadata cache
author Dirk Olmes <dirk@xanthippe.ping.de>
date Sat, 28 Feb 2015 02:42:21 +0100
parents 7897bdc57faf
children 8f5f6c388664 f31796d4c36a
files content/Linux/portage-metadata-cache.md
diffstat 1 files changed, 38 insertions(+), 0 deletions(-) [+]
line wrap: on
line diff
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/content/Linux/portage-metadata-cache.md	Sat Feb 28 02:42:21 2015 +0100
@@ -0,0 +1,38 @@
+Title: speeding up portage's metadata cache
+Date: 2015-02-26
+Lang: en
+
+[Gentoo's](http://www.gentoo.org) portage keeps metadata about installed ebuilds in `/var/cache/edb`. Dependency info for all installed ebuilds is in a `dep` subdirectory which typically looks something like this:
+
+    .
+    ├── usr
+    │   └── portage
+    │       ├── app-admin
+    │       │   ├── eselect-1.4.1
+    │       │   ├── eselect-lib-bin-symlink-0.1.1
+    │       │   ├── eselect-opengl-1.2.7
+    │       │   ├── gamin-0.1.10-r1
+    │       │   ├── logrotate-3.8.7
+    │       │   └── perl-cleaner-2.16
+    ...
+
+When portage needs to process dependency info, it reads those files. On a normal system, you will have a couple of hundred packages installed. This means that dependency processing does quite a bit of file processing.
+
+Now if that dependency info was stored in some kind of database, wouldn't that speed up dependency processing?
+
+I stumbled over [a wiki page](http://gentoo-en.vfose.ru/wiki/Portage_SQLite_Cache) describing how to switch portage's metadata cache to sqlite. Even the portage man page talks about this - try running `man portage` and read the section about the `modules` file.
+
+So I gave the sqlite metadata cache a try to measure if it really speeds up portage. After configuring the database and rebuilding the metadata cache, `/var/cache/edb/dep` looks a bit different now:
+
+    .
+    └── usr
+        ├── portage
+        └── portage.sqlite
+
+Now let's get to the interesting part: does the database really speed up portage? To measure, I ran a couple of `emerge -vp` commands using the normal setup and again using the database. The results are quite disappointing, though:
+
+The best improvement was about 6% with the metadata database. 
+
+57% of the ebuilds did run slower with the metadata database, the worst increase was about 43%
+
+So it looks like fiddling with portage's metadata cache is not really worth the hassle.