Hi Stewart,
Weekly status update;
Still having some problems when the cluster management node goes down.
For some reason the agent doesn't properly respond anymore after this.
From a few traces it seems that the agent fails to auto-reregister with
the snmpd daemon periodically (which it is supposed to do) and the snmpd
daemon gives up on it. At the very least, SNMP requests don't seem to be
forwarded. I need a little more time on that, perhaps I'll try with a
more recent net-snmp (I'm using 5.1.3.1).
Other than that I've added SNMP traps, which are now sent whenever the
agent loses or regains connection with either the cluster or the mysqld
daemon itself. Also a simple trap is sent upon startup or shutdown.
Agent daemonizes too (or not, at the option of the admin).
Code seems mostly clean (I am adding a couple of headers on functions so
they can be more easily reused), and a rudimentary Makefile is in place
(I haven't looked into autoconf/automake which would be fairly useless
if it will end up in the MySQL tree at some point in the future). No
testing as yet on any other version than 4.1.18.
Workarounds necessary:
* Missing include files for mysql/ndb/mgmapi. It seems that at least
ndb_global.h and ndb_net.h are not installed in mysql/ndb when
executing 'make install'. See ndb/include/Makefile.am in the
source tree for 4.1. Unfortunately, mysql/ndb/ndb_types.h, which
is included by mysql/ndb/mgmapi/mgmapi.h #includes ndb_global.h,
which will cause compiles to fail. Any reason why
ndb/include/Makefile.am lists ndb_global.h and ndb_net.h under
noinst_HEADERS? A workaround is copying both from the sourcetree
(ndb/include) into INSTALLPATH/mysql/ndb
* mysql/ndb/mgmapi/mgmapi.h defines 2 const values, namely const
struct Ndb_Mgm_Error_Msg ndb_mgm_error_msgs[] and const int
ndb_mgm_noOfErrorMsgs. If mgmapi.h is included in 2 or more
sourcefiles in a project, it will cause the link stage to fail
with a multiple definition error unless -zmuldefs is specified at
linkstage
* In mysql/ndb/mgmapi/mgmapi.h ndb_mgm_event_category and
ndb_mgm_clusterlog_level are two enums. However; strict C checking
does not allow for these types to be used without the 'enum'
keyword prepended. A workaround is typedef-ing both to 'int' prior
to including the mgmapi.h
I can provide a beta (it is properly running on my system, though I'm
still testing it). Let me know where I should send it to.
Cheers,
Michel
Michel Stam wrote:
>
> Please don't mind me here. Pressed send in mid-edit, so ignore the
> last mail. It seems I'm ready for the weekend ....
>
> Hey Stewart,
>
>> Ok I am trying to use my_sys.h as much as I can. Not sure if I can
>> use NDB portlib if people do not compile in ndb support?
>> not sure if we export it in the ndbapi lib... probably not :)
>
> I'm assuming not, but it can be changed later on. No problems there.
>
>
>>> In the future we'll be exporting more things in the
>>> INFORMATION_SCHEMA.FILES table. currently we have free extents for disk
>>> data. more, per table information is planned for sometime in the
>>> future.
>>>
>>> having all of this from SQL IMHO is a good idea.
>>>
>> Ok, in that case I'll not do anything right now and assume the
>> maximums. This can be added later on, when the information_schema
>> table is available
>> My current thoughts for what to add to the I_S.FILES table are below.
>> Note that this has not gone through *any* review yet (apart from a quick
>> "looks good" from some other devs) :) <SNIP>
>
>
> Loads of data there; Not that familiar with MySQL's internals yet.
> Again, I'll stick with what I have so I can get -a- version out.
> Discussion is far better if you can see what is there, rather than what
> is not.
>
>>> great - this was the purpose of ndb_size.pl too.
>>>
>> I added it to SNMP as I thought it would be a nice feature to have.
>> Especially since some programs, like Zabbix, can plot very nice
>> graphs of SNMP values, thus showing you how fast your database runs
>> full.
>> If it's possible (and easy) having it easy to plug in extra queries to
>> be exported via SNMP could be useful.
>
> Actually that was a design requirement for me; Currently I've
> implemented the queries as a series of const char* arrays. Every part of
> the SNMP tree has one. Also every part of said tree has a seperate
> handler function per part of the tree. This is the way Net-SNMP works.
> It may seem that some of the functions are quite similar (I've tried
> to keep them as much the same as possible, to make adding stuff easy;
> No need to know how the SNMP handling is done that way).
> Also, in the future someone might want to deviate from this and add
> something to one of the tree parts which is not a query in the form of
> 'show x like y' or 'select x from y', but perhaps a readout through a
> socket connection. Hence the redundancy there. Just so it can be
> easily extended.
>
> Adding a variable to the SNMP tree would require you to add an entry to
> the MIB file, edit a header file and one C file (in both cases, adding
> one or two constants should do it). So long as you don't want actual
> SNMP tables adding should be fairly easy. Especially if the data you
> seek is available through a query.
>
> SNMP tables is a different matter; these I've generated with the help of
> Net-SNMP's mib2c Perl script. This takes care of most of the hard work
> and creates a couple of iteration functions I had to fill in. Adding
> tables requires a bit more knowledge about how Net-SNMP does this.
> Still, that is doable too. Currently, there's 3 of these tables;
> - The MySQL processlist
> - The table list
> - NDB cluster status (ndb_mgm -e show), if compiled in
>
> Current status; I've removed all C++ code from the sources so it is
> completely C; comments have been added, I'm currently debugging a few
> minor snags.
> What remains;
> - Daemonize the program after starting
> - Meddle a little with the SNMP types for some of the variables in the
> SNMP tree
> - a bit more logging in case of errors
> - add support for embbeded database server, if compiled in
> - exhaustive testing
>
> I did find one thing;
> See the following link:
> main.c:175: multiple definition of `ndb_mgm_error_msgs'
> engines_ndbcluster.o(.rodata+0x200):engines_ndbcluster.c:192: first
> defined here
> main.o(.rodata+0x2a0): In function `main':
> main.c:199: multiple definition of `ndb_mgm_noOfErrorMsgs'
> engines_ndbcluster.o(.rodata+0x2a0):engines_ndbcluster.c:211: first
> defined here
> /usr/lib/libndbclient.so: undefined reference to `__cxa_pure_virtual'
> collect2: ld returned 1 exit status
>
> I could trace these to /usr/include/mysql/ndb/mgmapi/mgmapi.h.
> const struct Ndb_Mgm_Error_Msg ndb_mgm_error_msgs[] = {
> { NDB_MGM_NO_ERROR, "No error" },
> ..... and so on....
>
> const int ndb_mgm_noOfErrorMsgs =
> sizeof(ndb_mgm_error_msgs)/sizeof(struct Ndb_Mgm_Error_Msg);
>
> Because I include mgmapi.h in 2 different files, the link stage sees 2
> definitions of the same variable. Is it possible to change this into a
> declaration and move the definition into a .c file which is linked
> into the library?
> My only other option is passing -zmuldefs to the linker ... Unless I'm
> missing something, which of course is quite possible :)
>
> Lastly, the __cxa_pure_virtual requires me to link in stdc++, while
> the code is purely C other than that. This is probably unavoidable.
>
> Have a great weekend.
>
> Cheers,
>
> Michel
>
>
> Stewart Smith wrote:
>> On Thu, 2006-06-22 at 21:50 +0200, Michel Stam wrote:
>>
>>> Ok I am trying to use my_sys.h as much as I can. Not sure if I can
>>> use NDB portlib if people do not compile in ndb support?
>>>
>>
>> not sure if we export it in the ndbapi lib... probably not :)
>>
>>> I am probably going to need a few functions in the range of memset,
>>> strstr and strtoll/strtoull.
>>>
>>
>> best judgement - that's what we end up doing sometimes internally.
>>
>>> >>Also, I assume that the MySQL SNMP OID I mentioned in an earlier
>>> mail
>>> >>is
>>> >>currently not used by anyone in MySQL? (this to prevent any nasty
>>> >>conflicts later on). If not, I may need to register a new one
> (which
>>> >>takes approximately 4 weeks).
>>> >>
>>> > I don't know. We can chase that up a bit in the future.
>>>
>>> Ok, I'm using it right now, but it can be changed in under 2 minutes
>>> (its a constant in a header file).
>>>
>>
>> great!
>>
>>
>>>> In the future we'll be exporting more things in the
>>>> INFORMATION_SCHEMA.FILES table. currently we have free extents for
>>>> disk
>>>> data. more, per table information is planned for sometime in the
>>>> future.
>>>>
>>>> having all of this from SQL IMHO is a good idea.
>>>>
>>> Ok, in that case I'll not do anything right now and assume the
>>> maximums. This can be added later on, when the information_schema
>>> table is available
>>>
>>
>> My current thoughts for what to add to the I_S.FILES table are below.
>> Note that this has not gone through *any* review yet (apart from a quick
>> "looks good" from some other devs) :)
>> also, this wouldn't be added to anything before 5.1.
>>
>> Add reporting of IndexMemory and DataMemory usage (per table) to
>> INFORMATION_SCHEMA.FILES.
>>
>> Varsized attributes are allocated in a separate set of pages in 5.1.
>> so there
>> are two places to get usage from.
>>
>> In NDB, IndexMemory and DataMemory usage can be different on each
>> node. This is
>> less of a problem in 5.1 as we don't support initial NR as a form of
>> repacking
>> anymore.
>>
>> See WL#1359 for details of columns.
>>
>> I propose we have the following "files" with the following behaviour:
>>
>> foreach node:
>> FILE_NAME=NULL
>> FILE_TYPE="INDEX"
>> TABLESPACE_NAME="IndexMemory"
>> FREE_EXTENTS= number of free IndexMemory pages for node X
>> TOTAL_EXTENTS= total number of IndexMemory pages
>> EXTENT_SIZE=size of IndexMemory page
>> EXTRA= "NODE=X" (where X is node id) - format as for DATAFILES and
>> UNDOFILES.
>> - other columns are NULL
>>
>> same for DataMemory
>>
>> these rows exist so that you can get reports when there are no tables
>> in NDB
>>
>> for each NDB table:
>> a row for IndexMemory and DataMemory as above, but DataMemory is
>> split into
>> two parts (i.e. THREE rows per table)
>> - FILE_NAME="Fixed Sized Columns", TABLESPACE_NAME="DataMemory"
>> - FILE_NAME="Variable Sized Columns", TABLESPACE_NAME="DataMemory"
>> - TABLE_CATALOG, TABLE_SCHEMA, TABLE_NAME for the table
>> - ROW_FORMAT (fixed for fixed column part, variable for varsized part)
>> - TABLE_ROWS is row count for the table
>> - AVG_ROW_LENGTH (constant for fixed, possibly estimate for varsize)
>> - FREE_EXTENTS if we know we have free extents, number. else NULL
>> - TOTAL_EXTENTS= total number of pages allocated to this table
>> - EXTENT_SIZE=size of page
>> - DATA_LENGTH actual space used, in bytes. NULL if we cannot easily
>> find this out.
>> - MAX_DATA_LENGTH size we can be if we use all pages allocated to
>> us. - INDEX_LENGTH, always NULL - even for indexmemory.
>> - DATA_FREE (in bytes) the free space inside the pages allocated to
>> us.
>> possibly an estimate.
>> - EXTRA: node id as per DATAFILE. also OVERHEAD - the overhead per
>> page.
>>
>> In the case of BLOB tables and Index tables:
>> - another row for index/data memory but with FILE_NAME being the name
>> of the
>> index/blob table
>>
>>
>>>> great - this was the purpose of ndb_size.pl too.
>>>>
>>> I added it to SNMP as I thought it would be a nice feature to have.
>>> Especially since some programs, like Zabbix, can plot very nice
>>> graphs of SNMP values, thus showing you how fast your database runs
>>> full.
>>>
>>
>> If it's possible (and easy) having it easy to plug in extra queries to
>> be exported via SNMP could be useful.
>>
>>
>>> Ok, I'll await the information; Right now it isn't really important
>>> whether its fully integrated or not. Getting a version public is.
>>> Other than a few mysql/ndb/mgmapi/ include files which are missing
>>> by default after a 'make install' on the base mysql source tree
>>> (mysql/ndb/ndb_global.h and mysql/ndb/ndb_types.h if I remember
>>> correctly), the snmp module quite happily compiles outside of the
>>> MySQL source tree right now.
>>> I will look into that a little later on and submit a patch, if its
>>> not already fixed.
>>>
>>
>> great. we should fix this if it's a current problem.
>>
>>
>>> In my case, of course, the module code doesn't touch any of the
>>> server code (monitoring applications should affect the systems they
>>> monitor very little, or preferably, not at all).
>>>
>>
>> great. although we should expand support for monitoring applications
>> where appropriate.
>>
>>
>>> I'd welcome any review of the code, of course.
>>>
>>
>> look forward to reviewing it.
>>
>> thanks,
>> stewart
>>
>
--
Michel Stam ing.
Software Development Engineer
michel@stripped <mailto:michel@stripped>
Houtweg 29
1251 CS Laren
The Netherlands
t: +31 (0)35 548 98 58
f: +31 (0)35 531 65 61
www.teleconcept.nl <http://www.teleconcept.nl>
