Re: designing a test suite for multiple implementations (tools thread)

[chromatic <chromatic@wgz.org>]

On Monday 14 August 2006 17:20, jerry gay wrote:

> > I've added the plan for the neutral todo mechanism to Pugs' TASKS
> > file, getting help from many others on #perl6. The new todo marks look
> > like this:
> >
> >    todo :pugs<6.28.0>, :p6p5<0.110>, :parrot<1.00>;
> >    is $got, $expected;  # affected by the preceding todo call
>
> or equivalently,
>
> >     is $got, $expected, todo :pugs<6.2.13>, :p6p5<0.110>;
>
> this is a proposed api change to the test library for perl6. i expect that
> since these are the first tests written in perl6, they'll be used my many
> as a starting point for writing their own tests in the future. i'd like to
> hear from the perl-qa folks on what they think about supporting this
> feature: is this an acceptable proposal, or is there some other (better)
> way?

Ugh, so this means editing the canonical test repository for every 
status-changing update to every implementation?  Yuck.  This seems like a 
problem for implementation-specific harnesses and reporting tools.

-- c