List:Internals« Previous MessageNext Message »
From:MARK CALLAGHAN Date:March 20 2009 5:04pm
Subject:Re: Patch for MYISAM to mmap keys
View as plain text  
On Fri, Mar 20, 2009 at 9:04 AM, Masood Mortazavi
<masoodmortazavi@stripped> wrote:
> Hi Mark,

Must be for the other Mark.

>
> The MySQL server engineering team has started going through the
> backlog of contributions, and would like to look at your idea, here,
> too.
>
> Please note that MySQL is now using a new contributor agreement, i.e.
> the "Sun Contributor Agreement."
>
> Under the Sun Contributor Agreement (SCA), the contributor retains
> copyrights while also granting those same rights to Sun as the project
> sponsor. It supersedes the previously used MySQL Contributor License
> Agreement (CLA).
>
> You can get a copy of the SCA document here:
>    http://www.sun.com/software/opensource/sca.pdf
>
> You can sign and send your copy of the SCA document as described here:
>    http://forge.mysql.com/wiki/Sun_Contributor_Agreement
>
> Your signatory status along with your interest in contributing to
> MySQL will be recorded here:
>    https://sca.dev.java.net/CA_signatories.htm
>
> Having a clear SCA status will enable and facilitate discussion
> regarding your ideas and code contributions to MySQL.
>
> I hope this will help move things forward.
>
> Best regards,
> - m.
> a
>
> On Sun, Dec 2, 2007 at 4:58 AM, Mark Robson <markxr@stripped> wrote:
>>
>> Hi all,
>>
>> I had this idea to eliminate the MyISAM key buffer and let the OS
>> manage it by mmap()ing the whole of every .MYI file.
>>
>> This might improve performance but might also make MyISAM more
>> self-tuning (because the OS gets to choose what blocks to keep in ram
>> etc).
>>
>> Initial testing suggests that the mmap method is marginally faster for
>> some (e.g. bulk) operations- but I've not done anything really formal
>> yet.
>>
>> Anyway, it's currently in the concept stage, but I've created a quick
>> and dirty patch for 5.1.22-rc which implements it.
>>
>> As far as I can tell this patch seems to work in a subset of cases at
>> least. There are many caveats:
>>
>> * It effectively behaves as if delay_key_writes=ALL all the time (I think)
>> * Running out of address space is a real possibility on a 32-bit box
>> and will stop tables from being opened etc. There may be OS limits on
>> mmap()s which I haven't hit yet.
>> * Error checking isn't as good as it could be
>> * IO errors or running out of disc space will almost certainly cause
>> something bad to happen (crash?)
>>
>> This option is not currently parameterised (it cannot be turned on/off
>> even at compile time).
>>
>> ---
>>
>> There is a lot of copying of blocks in MyISAM key reads/writes, this
>> could probably mostly be eliminated if mmap() was used for indexes.
>> However, this would require considerable changes to the code base that
>> are currently well beyond my ability to understand the code :)
>>
>> Comments please
>>
>> Mark
>>
>>
>> --
>> MySQL Internals Mailing List
>> For list archives: http://lists.mysql.com/internals
>> To unsubscribe:    http://lists.mysql.com/internals?unsub=1
>>
>
> --
> MySQL Internals Mailing List
> For list archives: http://lists.mysql.com/internals
> To unsubscribe:    http://lists.mysql.com/internals?unsub=1
>
>



-- 
Mark Callaghan
mdcallag@stripped
Thread
Patch for MYISAM to mmap keysMark Robson2 Dec
  • Re: Patch for MYISAM to mmap keysMasood Mortazavi20 Mar
    • Re: Patch for MYISAM to mmap keysMARK CALLAGHAN20 Mar
    • scope of SCA? (was: Re: Patch for MYISAM to mmap keys)Kristian Nielsen20 Mar
      • Re: scope of SCA? (was: Re: Patch for MYISAM to mmap keys)Masood Mortazavi21 Mar
Re: Patch for MYISAM to mmap keysSergey Vojtovich23 Mar
Re: Patch for MYISAM to mmap keysSergey Vojtovich24 Mar