Last active
August 29, 2015 14:21
-
-
Save johnnyutahh/f4e3d2d3fb07de5fa146 to your computer and use it in GitHub Desktop.
notmuch vs mu (maildir-utils) indexing performance
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The following tests were run in 2014 (the November timestamps) and 2015 | |
(the March and May timestamps). | |
After initial email ingest, the following command-line demonstrate the | |
performance of re-checking the same maildir folders | |
'notmuch new' = 11-13 seconds (mean 12s) | |
'mu index' = 170-190 seconds (mean 180s) | |
Based upon these figures, notmuch offers a 180s / 12s = 15X improvement. | |
The above presumably represents the most-frequently-executed command for | |
each application, given said command is typically executed every time | |
after receiving new email. | |
Question: where these tests configured and executed correctly? The | |
performance difference is remarkable, generating concerns about correct | |
application setup, environment. | |
---- command-line sessions ---- | |
Nov 10 14:56:57 ~$ /usr/bin/time mu index --maildir=~/data.email/mbsync | |
indexing messages under /Users/user1/data.email/mbsync [/Users/user1/.mu/xapian] | |
\ processing mail; processed: 1007100; updated/new: 1720, cleaned-up: 0 | |
cleaning up messages [/Users/user1/.mu/xapian] | |
\ processing mail; processed: 1007332; updated/new: 0, cleaned-up: 171 | |
elapsed: 78 second(s), ~ 12914 msg/s | |
| processing mail; processed: 1007332; updated/new: 0, cleaned-up: 171 | |
elapsed: 180 second(s), ~ 5596 msg/s | |
181.35 real 16.51 user 131.11 sys | |
Nov 10 15:00:15 ~$ | |
Nov 10 15:10:59 ~$ /usr/bin/time notmuch new 2>&1 | grep -vi ignoring | |
Processed 9466 total files in 40s (233 files/sec.). | |
Added 725 new messages to the database. Detected 171 file renames. | |
40.87 real 10.09 user 10.28 sys | |
Nov 10 15:11:41 ~$ /usr/bin/time notmuch new 2>&1 | grep -vi ignoring | |
Processed 74 total files in 11s (6 files/sec.). | |
No new mail. | |
11.93 real 0.74 user 2.81 sys | |
Nov 10 15:12:42 ~$ /usr/bin/time notmuch new 2>&1 | grep -vi ignoring | |
Processed 74 total files in 12s (5 files/sec.). | |
No new mail. | |
12.53 real 0.76 user 2.97 sys | |
Nov 10 15:12:58 ~$ /usr/bin/time mu index --maildir=~/data.email/mbsync | |
indexing messages under /Users/user1/data.email/mbsync [/Users/user1/.mu/xapian] | |
\ processing mail; processed: 1007100; updated/new: 0, cleaned-up: 0 | |
cleaning up messages [/Users/user1/.mu/xapian] | |
| processing mail; processed: 1007161; updated/new: 0, cleaned-up: 0 | |
elapsed: 74 second(s), ~ 13610 msg/s | |
/ processing mail; processed: 1007161; updated/new: 0, cleaned-up: 0 | |
elapsed: 170 second(s), ~ 5924 msg/s | |
170.64 real 11.58 user 134.26 sys | |
Nov 10 15:15:54 ~$ | |
Mar 16 16:54:51 ~/.../mbsync$ /usr/bin/time mu index -m . | |
indexing messages under /Users/user1/data.email/mbsync [/Users/user1/.mu/xapian] | |
/ processing mail; processed: 961350; updated/new: 961059, cleaned-up: 0 | |
cleaning up messages [/Users/user1/.mu/xapian] | |
/ processing mail; processed: 1031361; updated/new: 0, cleaned-up: 69999 | |
elapsed: 214 second(s), ~ 4819 msg/s | |
- processing mail; processed: 1031361; updated/new: 0, cleaned-up: 69999 | |
elapsed: 2519 second(s), ~ 409 msg/s | |
2519.66 real 1510.73 user 489.33 sys | |
Mar 16 17:37:04 ~/.../mbsync$ | |
Mar 16 17:40:57 ~/.../mbsync$ | |
Mar 16 17:41:42 ~/.../mbsync$ | |
Mar 16 17:41:42 ~/.../mbsync$ /usr/bin/time mu index -m . | |
indexing messages under /Users/user1/data.email/mbsync [/Users/user1/.mu/xapian] | |
/ processing mail; processed: 961350; updated/new: 0, cleaned-up: 0 | |
cleaning up messages [/Users/user1/.mu/xapian] | |
| processing mail; processed: 961362; updated/new: 0, cleaned-up: 0 | |
elapsed: 92 second(s), ~ 10449 msg/s | |
/ processing mail; processed: 961362; updated/new: 0, cleaned-up: 0 | |
elapsed: 190 second(s), ~ 5059 msg/s | |
191.29 real 12.36 user 156.05 sys | |
Mar 16 17:44:56 ~/.../mbsync$ | |
Mar 16 17:52:29 ~/.../mbsync$ /usr/bin/time notmuch new 2>&1 | grep -vi ignoring | |
Processed 13199 total files in 4m 55s (44 files/sec.). | |
Added 3443 new messages to the database. Removed 3719 messages. Detected 53619 file renames. | |
296.94 real 216.72 user 31.81 sys | |
Mar 16 17:57:31 ~/.../mbsync$ | |
Mar 16 17:57:41 ~/.../mbsync$ /usr/bin/time notmuch new | |
No new mail. | |
11.11 real 0.76 user 2.86 sys | |
Mar 16 17:57:54 ~/.../mbsync$ /usr/bin/time notmuch new | |
No new mail. | |
11.56 real 0.77 user 2.73 sys | |
Mar 16 17:58:13 ~/.../mbsync$ | |
May 16 17:07:43 ~$ mu --version | |
mu (mail indexer/searcher) version 0.9.11 | |
Copyright (C) 2008-2013 Dirk-Jan C. Binnema | |
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>. | |
This is free software: you are free to change and redistribute it. | |
There is NO WARRANTY, to the extent permitted by law. | |
May 16 17:07:47 ~$ notmuch --version | |
notmuch 0.19 | |
May 16 17:07:51 ~$ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment