From: John Embretsen
Date: January 19 2009 4:45pm
Subject: Re: bzr commit into mysql-6.0-falcon-team branch (john.embretsen:2964)
List-Archive: http://lists.mysql.com/falcon/391
Message-Id: <4974AE45.4010907@Sun.COM>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7BIT

Kevin,

Thanks for taking a look at this. Comments inline.

Kevin Lewis wrote:
> John,
> 
> I was trying you approach out today and found that the pruning cycle
> that allows blob pages to be reused is not called on a load-based event
> for this test.
> 
> The load-based system I implemented will make a call to check whether to
> start the scavenger after every 64 records allocated.  This was to avoid
> doing math constantly when allocating record cache memory.  So after 64
> records have been added to the record cache, it will check how many
> bytes have been added by subtracting the current cache size from the
> cache size the last time the scavenger actually ran.  If this is more
> than 2% of the cache, then it will call the scavenger.

Thanks for describing this...

> Debugging it, I found that each RecordVersion object for this updated
> blob record uses up only 162 bytes (in debug mode) of the record cache.
>    With a default cache size of 250 Mb, it will only call the scavenger
> after every 5 Mb of record cache use.  But these blob records only add
> 162 bytes per update to the cache, while adding 1.1 Mb to new blob pages.

Right, as you mentioned in the bug report, load-based scavenging will
not be triggered until after about 33000 updates in such a scenario.

> So the reason you test seemed to keep the file size down was that a
> regular scheduled scavenge just happened to run.  Those scavenge cycles
> will run no matter how much has recently been added to cache.

I see, thanks for clarifying. (The first scavenge cycle seems to occur
almost immediately after server startup every time, though.)

Question (just trying to understand this better):

The same default value of the falcon_scavenge_schedule variable existed
also with the old scavenger. Are you saying that the old scavenger did
not prune these old records at the regular scavenge cycles _unless_ the
record cache was 75% full?


> This means that the success of your test is timing dependent.
> 
> I think I need to re-open this bug and add some kind of blob page
> tracker that will signal the scavenger after a certain number of blob
> pages have been added (preferably, added by update).

I created a new bug for this, http://bugs.mysql.com/bug.php?id=42202
"Unacceptable tablespace growth when doing rapid BLOB updates". Feel
free to update the bug report if needed.

> As for your test script, you should fill the blob field with values that
> are independent of the filing system;
> 
> INSERT INTO t1 (myblob) VALUES (repeat('a', 1*1024*1024));
> UPDATE t1 SET myblob = (repeat('b', 1*1024*1024));
> 
> A file named 'MYSQL_TEST_DIR/include/UnicodeData.txt' does not work very
> well on Windows.

Good point, I will modify the test accordingly. However, I also use a
file to temporarily store the size of the tablespace, so I need to work
out a solution for that as well. The solution would probably be either
 a) do some file path separator trickery if on Windows
 b) disable the test on Windows

> You could try to set the scavenge schedule to every 2 or 5 seconds for
> this test, and it will probably work pretty well.  But it would still be
> timing dependent.

Yes, but not too badly, I guess, given the sleeps I added between the
BLOB updates. I'll try setting a non-default schedule in the new version
of the test to reduce timing dependence.


thanks,

-- 
John

> John H. Embretsen wrote:
>> #At
>> file:///export/home/tmp/je159969/mysql/bzr-repos/build-mysql-6.0-falcon-team/
>> based on revid:cpowers@stripped
>>
>>  2964 John H. Embretsen    2009-01-16
>>       Regression test for bug 41870 - Unbounded tablespace growth when
>> updating BLOB record
>>             This test measures the size of the default (FALCON_USER)
>> tablespace file
>>       both before and after updating a 1.1 MB BLOB multiple (20) times.
>>       If no or very little space was released and re-used during this
>> time, the
>>       test will fail. File size is measured using Perl code.
>>             Test case may be sensitive to changes in the behavior of
>> the Falcon scavenger, and is a --big-test.
>> added:
>>   mysql-test/suite/falcon/r/falcon_blob_space-big.result
>>   mysql-test/suite/falcon/t/falcon_blob_space-big-master.opt
>>   mysql-test/suite/falcon/t/falcon_blob_space-big.test
>>
>> per-file messages:
>>   mysql-test/suite/falcon/r/falcon_blob_space-big.result
>>     Expected test result. Efforts have been made to make this as
>> informative as
>>     possible while still being relatively robust.
>>   mysql-test/suite/falcon/t/falcon_blob_space-big-master.opt
>>     mysqld options needed for 1.1 MB BLOB updates given the current
>> default mysql-test settings.
>>   mysql-test/suite/falcon/t/falcon_blob_space-big.test
>>     New test case. Involves Perl code and the use/passing of
>> environment variables. Should be able to detect regressions of bug 41870.