From: Warren Young
Date: July 18 2007 9:45pm
Subject: Re: Connection class reorg in a testable state
List-Archive: http://lists.mysql.com/plusplus/6828
Message-Id: <469E89F9.2050206@etr-usa.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Joel Fielder wrote:
> 
> If you're planning on introducing more testing like you have done for
> TCPConnection, may I recommend UnitTest++,

Well, I can tell you right now that it can't fully replace dtest. 
UnitTest++ doesn't look like it would be as facile at exercising the 
example programs as dtest.  The most it could do is replace the scheme 
I'm building up in the nascent test subdirectory.

> I would be very pleased if you introduced unit testing 

I thought I already had. :)

> as it would allow
> more freedom for those who want to help develop the library without
> having to worry about whether they've broken anything else

Precisely.  v2.3.2 saw the creation of dtest, and in its first hour of 
existence it pointed out two bugs that were fixed before anyone else 
noticed them.  (At least, no one complained about them on the list.) 
I'm already sold.

> I know there's the
> examples and your new dtest thingy but I'm not sure they offer the same
> level of confidence as a screen which says "Success: 2000 tests passed".

I don't see what it is you're uneasy about.  I think dtest provides the 
same benefit you're talking about, at least potentially.

If your concern is that we don't yet have 2000 tests, get coding. :)

If it's that the output of dtest doesn't have the same form as you're 
used to, we can discuss changing it.

> Plus it would be easier for
> end users to check if the library works on their platform :)  

This is actually one of my objections to UnitTest++: it seems to want to 
make testing mandatory for all builds.  I won't do that unless I'm 
convinced that tests will almost never fail spuriously due to niggly 
platform differences.

It may sound unrealistic to demand that, but I can't abide the 
possibility of noise on the list about spurious UT++ bugs.  The only 
benefit to testing everywhere is that it can catch problems with 
platforms I don't use.  So far, the frequency of such bug reports is so 
low that it wouldn't take much noise to swamp the benefit.

Please consider this a challenge, not a refusal.  Can you convince me 
that UT++ provides enough benefit over dtest to counteract the time 
spent dealing with list noise?

> Where
> appropriate, you could create a mock object to represent the database
> server and then check that mysql++ has done what it should have done

dtest addresses this issue already.  It avoids the repeatability problem 
you get in testing against "real" data by testing against a fixed sample 
database.

You can argue that the current dtest scheme is emaciated, but the 
solution is independent of implementation details: add more sample data.

> this would push the library towards database independence 

I, too, have found having a dummy database driver for the purpose of 
testing valuable, but I don't see it as a good way to make MySQL++ 
database-independent.

I believe it to be an absolute Law of Computing that a component is not 
reusable until it _has been_ reused at least three times in disparate, 
real situations.  A dummy database driver doesn't count towards that. 
Database independence must precede dummy database drivers for testing.