1. fix all tests
2. testing.drupal.org (or some service) watch cvs commits and upon commit, run tests
3. if testing.drupal.org reports a test failing, someone would roll-back the commit
until it's resolved.

It's really the only fair and sane way IMHO to stay ontop of this. But there is a long way to go to get
there i understand.

It's also important we learn from the failing tests and see if they are failing for good or bad reasons and
if there is something that keeps breaking things over and over that can be fixed once and for all.

Thanks for posting this, you've nailed all the concerns and implications very well!