Hi Guilhem
Guilhem Bichot wrote:
> Hello,
>
> Marc Alff a écrit, Le 07.04.2009 23:39:
>> Guilhem Bichot wrote:
>>> Marc Alff a écrit, Le 07.04.2009 18:13:
>>>> Guilhem Bichot wrote:
>>>>> Marc Alff a écrit, Le 23.03.2009 18:34:
>>>>>> Guilhem Bichot wrote:
>>>>>> ./mysqld --performance-schema-max-mutex-instruments=200
>>>>>>
>>>>>> Let's assume 150 mutex instruments have been loaded already.
>>>>>>
>>>>>> Let's assume "a_plugin" contains 40 mutex instruments.
>>>>>> Let's assume "b_plugin" contains 20 mutex instruments.
>>>>>>
>>>>>> INSTALL PLUGIN a_plugin
>>>>>> --> the server now has 150+40 = 190 mutex instruments.
>>>>>>
>>>>>> UNINSTALL PLUGIN a_plugin
>>>>>> --> the server still has 190 instruments
>
>>>
>>>> There is currently no way to prevent a plugin from consuming slots for
>>>> registering the instruments themselves, which is the memory
>>>> allocated by
>>>> --performance-schema-max-mutex-instruments
>>>>
>>>> I don't see this as an issue, since only a few bytes per instrument
>>>> will
>>>> be used, and the number of instruments in a plugin should be very low.
>>>> There are no plans to selectively exclude instrumentation when
>>>> loading a
>>>> plugin with some filtering options.
>>>>
>>>> To see how much memory is used, use:
>>>>
>>>> SHOW ENGINE PERFORMANCE_SCHEMA STATUS
>>>>
>>>> (PFS_MUTEX_INFO).ROW_SIZE is 248 bytes
>>>>
>>>> So, if a plugin has 10 instrumented mutexes, it will "waste" 2480
>>>> bytes.
>>> And in the default config:
>>> --performance-schema-max-rwlock-instruments=<number>
>>> default=20
>>> Maximum number of rwlock instruments.
>>> So, if a plugin has 2 rwlocks, it takes 10% of the allowed space. No
>>> matter if that's only a few hundreds of bytes, it's still 10%.
>>> If it has 10 types of rwlocks (quite possible with a not-trivial
>>> engine), that will take 50%.
>>>
>>
>> That is correct, although I don't understand why it's relevant.
>>
>> The default value is meant to be sufficient for all the MySQL provided
>> plugins, and will be revised when more instrumentation or more plugin
>> are implemented.
>>
>> If a third party plugin needs more slots, that's what the
>> --performance-schema-max-rwlock-instruments=<number> is for.
>>
>> Please describe what the problem is, I just don't see one here.
>
> The command-line option above is used at startup. Setting it to a large
> enough value requires knowing in advance that we'll plug and unplug a
> certain plugin during the lifetime of this mysqld.
> Somehow it requires divination of a problem happening in the future.
>
I see neither a question nor a comment here.
How many instruments are present in each plugin, aka what a plugin
requirements are in terms of instruments, should be part of the plugin
documentation.
This as been explained already.
One of the fundamental design choice of the performance schema is to pre
allocate buffers with a fixed size to never have to deal with resizing
memory or dynamic allocation, which is critical for runtime performances.
There are no requirements to be able to resize dynamically internal
buffers, for this very reason. This as been discussed already, not to
mentioned approved, during the architecture review.
>>>> The best option if space is a concern is to not use space by avoiding
>>>> loading instrumented code. Space can *never* be freed, as doing this
>>>> while the code is running and executing instrumented code will lead to
>>>> crashes.
>>> In my case, the plugin's instrumented code is not running as UNINSTALL
>>> PLUGIN has been executed. So I don't follow the explanation.
>>
>> All the code performing a SELECT from the performance schema can be
>> executed during or after a plugin is uninstalled, and this code relies
>> on the underlying data structures to be valid and stable, for the
>> instruments themselves.
>> I should have said just "while the code is running" then ...
>
> Yes. Makes it clear.
>
>>>> That's not a bug, it's a conscious design choice, to have lock-free
>>>> code.
>>> There is a shortcut above. For example lf_hash is lock-free, though it
>>> reuses memory (recycles) when an element has been deleted and nobody is
>>> looking at it anymore.
>>>
>>
>> Sure.
>>
>> But lf_hash is not the only way to implement lock free code either.
>
> Yes, I didn't say "only". I said "for example".
>
>> The choice here is to have an array as static as possible, which makes
>> it much easier to access elements directly without performance hits.
>>
>> Also, you seem to be assuming that memory should be freed after
>> UNINSTALL plugin, which is not the case.
>>
>> Instruments used by a plugin survive UNINSTALL plugin, for the data in
>> the collected by the performance schema to be still meaningful (_HISTORY
>> tables, aggregates, and later state or lock data).
>
> Assuming a user TRUNCATEs the P_S tables after uninstalling the plugin,
> you will have a hard time explaining her/him that it's necessary that
> the plugin's instruments continue occupying rows somewhere.
Loading / unloading a plugin has nothing to do with the performance
schema data.
If a query executed code in that plugin, it waited on some code in the
plugin.
Even after the plugin is unloaded, the time spent by the query or the
server was still spent, and is still readable in the performance schema
tables, as well as various aggregates, as well as in history tables.
That includes aggregates per instruments.
Also, there is no TRUNCATE in SETUP_INSTRUMENT,
truncate is only to clear recorded data or aggregation counters, not to
remove the counters.
This works as expected.
>
> However, as I can imagine that freeing memory would be hard to do,
> especially as it would create a gap in the array, and as the problems
> above will likely be rare, I won't argue more. I merely needed to hear
> your arguments. Thanks.
>
