Hi Monty,
<snip>
> We can certainly do this even better, but the truth is simple that some
> bugs are impossible to find without having a lot of test users that
> uses the product in many different ways.
>
> Our main solution to this is to do frequent releases so that a bug
> doesn't live for long. Even with more than 120 releases behind us, we
> haven't had more than 4-5 release that has had a fatal bug that hits
> that would hit a majority of our users (like the ORDER BY bug in
> 3.23.34).
Fair enough, and I'd picked up from posts in that past that you have some
pretty comprehensive testing so I wasn't really having a dig at you guys.
I'm acutely aware of testing issues at the moment though - the small trading
firm that I work for was recently bought by a large Wall Street bank for our
technology, particularly for order execution and electronic trading
machines that just sit there and every day make significant money by
themselves. Yet, despite our testing, we still role out stuff that
sometimes has careless bugs and it just plain sucks. Small company, infinite
drive and 'get it out the door asap' approach to minimise lost trading
opportunities, and where you know that you can fix a bug quickly and role
out a new release, is just no longer appropriate or acceptable in a much
larger corporate environment.
So testing is an issue much on my mind at the moment :-)
Nick
