List:Internals« Previous MessageNext Message »
From:Mark Robson Date:December 2 2007 11:58am
Subject:Patch for MYISAM to mmap keys
View as plain text  
Hi all,

I had this idea to eliminate the MyISAM key buffer and let the OS
manage it by mmap()ing the whole of every .MYI file.

This might improve performance but might also make MyISAM more
self-tuning (because the OS gets to choose what blocks to keep in ram
etc).

Initial testing suggests that the mmap method is marginally faster for
some (e.g. bulk) operations- but I've not done anything really formal
yet.

Anyway, it's currently in the concept stage, but I've created a quick
and dirty patch for 5.1.22-rc which implements it.

As far as I can tell this patch seems to work in a subset of cases at
least. There are many caveats:

* It effectively behaves as if delay_key_writes=ALL all the time (I think)
* Running out of address space is a real possibility on a 32-bit box
and will stop tables from being opened etc. There may be OS limits on
mmap()s which I haven't hit yet.
* Error checking isn't as good as it could be
* IO errors or running out of disc space will almost certainly cause
something bad to happen (crash?)

This option is not currently parameterised (it cannot be turned on/off
even at compile time).

---

There is a lot of copying of blocks in MyISAM key reads/writes, this
could probably mostly be eliminated if mmap() was used for indexes.
However, this would require considerable changes to the code base that
are currently well beyond my ability to understand the code :)

Comments please

Mark

Attachment: [text/x-patch] myisam-mmap.patch
Thread
Patch for MYISAM to mmap keysMark Robson2 Dec
  • Re: Patch for MYISAM to mmap keysMasood Mortazavi20 Mar
    • Re: Patch for MYISAM to mmap keysMARK CALLAGHAN20 Mar
    • scope of SCA? (was: Re: Patch for MYISAM to mmap keys)Kristian Nielsen20 Mar
      • Re: scope of SCA? (was: Re: Patch for MYISAM to mmap keys)Masood Mortazavi21 Mar
Re: Patch for MYISAM to mmap keysSergey Vojtovich23 Mar
Re: Patch for MYISAM to mmap keysSergey Vojtovich24 Mar