Re: designing a test suite for multiple implementations (tools thread)

["jerry gay" <jerry.gay@gmail.com>]

On 8/14/06, chromatic <chromatic@wgz.org> wrote:
> Ugh, so this means editing the canonical test repository for every
> status-changing update to every implementation?  Yuck.  This seems like a
> problem for implementation-specific harnesses and reporting tools.

this is what prompted me to start this thread. the problem is years of test
suite design, which point to keeping todo() info with in the testfiles, with
each subtest. moving todo() info out of these test files leads to fragile
test harnesses, as adding a test to the middle of a file will change test
numbers. if test descriptions are used, then unique descriptions for each
test are required. et cetera.

this is all hashed out in the link to the #perl6 conversation larry provided
previously. the conclusion was to keep todo() info with the subtests and
deal with the result (for now.) i *hope* there's a better way. i repost the
link for ease of reference:
http://colabti.de/irclogger/irclogger_log/perl6?date=2006-08-07,Mon&sel=110#l193

~jerry