From: Vladislav Vaintroub
Date: January 19 2009 5:17pm
Subject: RE: bzr commit into mysql-6.0-falcon-team branch (john.embretsen:2964)
List-Archive: http://lists.mysql.com/falcon/393
Message-Id: <001d01c97a59$d7655470$862ffd50$@com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7BIT

> >
> > A file named 'MYSQL_TEST_DIR/include/UnicodeData.txt' does not work
> very
> > well on Windows.
> 
> Good point, I will modify the test accordingly. However, I also use a
> file to temporarily store the size of the tablespace, so I need to work
> out a solution for that as well. The solution would probably be either
>  a) do some file path separator trickery if on Windows
>  b) disable the test on Windows

John and Kevin,
Perl  should be ok with '/' as path separator. MySQL does only officially
allow '/' as path separator. There are several tests in the test suite that
use external files, and if I remember correctly they all use '/' as path
separator.

In the meanwhile, I'd suggest falcon gets a working Windows machine in the
trd lab, because I have a feeling Norwegian guys are too scared of it and
would better disable everything rather than fix;)




> -----Original Message-----
> From: John.Embretsen@stripped [mailto:John.Embretsen@stripped]
> Sent: Monday, January 19, 2009 5:46 PM
> To: Kevin Lewis
> Cc: commits@stripped; FalconDev
> Subject: Re: bzr commit into mysql-6.0-falcon-team branch
> (john.embretsen:2964)
> 
> Kevin,
> 
> Thanks for taking a look at this. Comments inline.
> 
> Kevin Lewis wrote:
> > John,
> >
> > I was trying you approach out today and found that the pruning cycle
> > that allows blob pages to be reused is not called on a load-based
> event
> > for this test.
> >
> > The load-based system I implemented will make a call to check whether
> to
> > start the scavenger after every 64 records allocated.  This was to
> avoid
> > doing math constantly when allocating record cache memory.  So after
> 64
> > records have been added to the record cache, it will check how many
> > bytes have been added by subtracting the current cache size from the
> > cache size the last time the scavenger actually ran.  If this is more
> > than 2% of the cache, then it will call the scavenger.
> 
> Thanks for describing this...
> 
> > Debugging it, I found that each RecordVersion object for this updated
> > blob record uses up only 162 bytes (in debug mode) of the record
> cache.
> >    With a default cache size of 250 Mb, it will only call the
> scavenger
> > after every 5 Mb of record cache use.  But these blob records only
> add
> > 162 bytes per update to the cache, while adding 1.1 Mb to new blob
> pages.
> 
> Right, as you mentioned in the bug report, load-based scavenging will
> not be triggered until after about 33000 updates in such a scenario.
> 
> > So the reason you test seemed to keep the file size down was that a
> > regular scheduled scavenge just happened to run.  Those scavenge
> cycles
> > will run no matter how much has recently been added to cache.
> 
> I see, thanks for clarifying. (The first scavenge cycle seems to occur
> almost immediately after server startup every time, though.)
> 
> Question (just trying to understand this better):
> 
> The same default value of the falcon_scavenge_schedule variable existed
> also with the old scavenger. Are you saying that the old scavenger did
> not prune these old records at the regular scavenge cycles _unless_ the
> record cache was 75% full?
> 
> 
> > This means that the success of your test is timing dependent.
> >
> > I think I need to re-open this bug and add some kind of blob page
> > tracker that will signal the scavenger after a certain number of blob
> > pages have been added (preferably, added by update).
> 
> I created a new bug for this, http://bugs.mysql.com/bug.php?id=42202
> "Unacceptable tablespace growth when doing rapid BLOB updates". Feel
> free to update the bug report if needed.
> 
> > As for your test script, you should fill the blob field with values
> that
> > are independent of the filing system;
> >
> > INSERT INTO t1 (myblob) VALUES (repeat('a', 1*1024*1024));
> > UPDATE t1 SET myblob = (repeat('b', 1*1024*1024));
> >
> > A file named 'MYSQL_TEST_DIR/include/UnicodeData.txt' does not work
> very
> > well on Windows.
> 
> Good point, I will modify the test accordingly. However, I also use a
> file to temporarily store the size of the tablespace, so I need to work
> out a solution for that as well. The solution would probably be either
>  a) do some file path separator trickery if on Windows
>  b) disable the test on Windows
> 
> > You could try to set the scavenge schedule to every 2 or 5 seconds
> for
> > this test, and it will probably work pretty well.  But it would still
> be
> > timing dependent.
> 
> Yes, but not too badly, I guess, given the sleeps I added between the
> BLOB updates. I'll try setting a non-default schedule in the new
> version
> of the test to reduce timing dependence.
> 
> 
> thanks,
> 
> --
> John
> 
> > John H. Embretsen wrote:
> >> #At
> >> file:///export/home/tmp/je159969/mysql/bzr-repos/build-mysql-6.0-
> falcon-team/
> >> based on revid:cpowers@stripped
> >>
> >>  2964 John H. Embretsen    2009-01-16
> >>       Regression test for bug 41870 - Unbounded tablespace growth
> when
> >> updating BLOB record
> >>             This test measures the size of the default (FALCON_USER)
> >> tablespace file
> >>       both before and after updating a 1.1 MB BLOB multiple (20)
> times.
> >>       If no or very little space was released and re-used during
> this
> >> time, the
> >>       test will fail. File size is measured using Perl code.
> >>             Test case may be sensitive to changes in the behavior of
> >> the Falcon scavenger, and is a --big-test.
> >> added:
> >>   mysql-test/suite/falcon/r/falcon_blob_space-big.result
> >>   mysql-test/suite/falcon/t/falcon_blob_space-big-master.opt
> >>   mysql-test/suite/falcon/t/falcon_blob_space-big.test
> >>
> >> per-file messages:
> >>   mysql-test/suite/falcon/r/falcon_blob_space-big.result
> >>     Expected test result. Efforts have been made to make this as
> >> informative as
> >>     possible while still being relatively robust.
> >>   mysql-test/suite/falcon/t/falcon_blob_space-big-master.opt
> >>     mysqld options needed for 1.1 MB BLOB updates given the current
> >> default mysql-test settings.
> >>   mysql-test/suite/falcon/t/falcon_blob_space-big.test
> >>     New test case. Involves Perl code and the use/passing of
> >> environment variables. Should be able to detect regressions of bug
> 41870.
> 
> --
> Falcon Storage Engine Mailing List
> For list archives: http://lists.mysql.com/falcon
> To unsubscribe:    http://lists.mysql.com/falcon?unsub=wlad@stripped