Constraints on Perl 5 (Larry's list)

Larry Wall (the founding father of Perl) offered the following list of constraints he perceived on Perl 5, at a speech at the Atlanta Linux Showcase in October 2000. Emphasised phrases are Larry's; the plain text parts are my own annotations of Larry's points.

• The need for backward compatibility: Perl 5 attempted to be backward compatible with Perl 4, which attempted to be backward compatible with Perl 3. This leads to a situation where legacy or deprecated language constructs are still supported, even if they really should be thrown out.

• My leisurely mental growth: Although most people would be stunned to hear Larry describe his mental capacities as less than godlike, it is a fact that Perl 5 was Larry's rewrite, and needed to fit within the scope of his comprehension.

• Cultural model with too many bottlenecks: Perl 5 development centred around the perl5-porters mailing list ("p5p"), and was presided over by a single "pumpking" (project leader) at any one time.

• Lack of knowledge of the future: When Perl 5 was first envisaged, the World Wide Web was only just gaining in popularity, XML had never been heard of, and numerous other technologies that we now take for granted were only a dim light on the horizon.

• Lack of round tuition: There were a lot of things that might have been done with Perl 5 if Larry and/or the other developers had got a "round tuit".

• Complexity of the core: Perl 5 was a significant departure from Perl 4, and introduced a lot of new concepts. The implementation of many of these turned out to be quite complex, to the point where a potential Perl 5 developer may not be able to understand it.

• Complacency: At the outset of Perl 5, Perl was acknowledged as one of the best and most widely used tools for toolsmithing on Unix systems, and for CGI scripting for the World Wide Web. For some years, Perl rested on its laurels and failed to address new fields of programming as they rose in prominence.

How it used to be (and still is, for Perl 5)

Perl 5 development centred around one mailing list, perl5-porters, aka "p5p". This mailing list was home to a core group of perl hackers and an ever-changing procession of hopeful contributors, with Larry presiding over the whole thing as a benevolent, but mostly uninvolved, dictator.

The p5p list was very high volume, and many subscribers dropped off it when they were unable to keep up with the discussions. For those without the courage to brave the deluge, a weekly summary was posted on the web and available by email. This provided links to threads or specific posts of significant interest, all of which were available via a web archive.

As well as being high volume, p5p had a reputation for unfriendliness and for being the home to some particularly heated flame wars. Certain personalities on the list were renowned for the frequency and vehemence of their flames, and newcomers were often scared away by the argumentative atmosphere or the cold welcome they received.

More recently, a system of "refereeing" has been introduced, in which a small group of trusted regulars are charged with keeping the peace. The means at their disposal include out-of-band warnings to offenders and moderation of posters who repeatedly ignore warnings.

Why change it?

A number of reasons were given for why the Perl development process needed changing. These included:

• Reverse p5p's bad reputation

• Give everyone an opportunity to contribute -- not just the old-timers and Unix users, but also the Windows contingent, the WWW scripting crowd, and those who use Perl for purposes ranging from astronomical data-crunching to mainframe applications development

• Record discussions and decisions for future reference, so that late joiners could read a clear archive of what had gone before

• Apply a bit of real project management

• Foster the kind of working environment that doesn't drive valuable people away

How did we make decisions about this?

Straight after the TPC 4.0 meeting and announcement, a mailing list called perl6-bootstrap was set up, with the goal of determining what shape the subsequent process and community should take.

Unfortuneately, the bootstrap phase was given only a couple of weeks to make decisions, and the first week of that was largely wasted while travelling TPC attendees returned home, recovered from their jetlag, and caught up on their email backlogs.

The result was rushed decisions that upset some people. However, most of the people were upset because the decisions had been rushed, not because they were necessarily bad. Few suggestions were offered that differed significantly from what we ended up with -- most differences were in the naming of mailing lists and other such (relative) trivia.

The shape of Perl 6 development

Here's what the bootstrap list came up with:

• Project manager: Nathan Torkington

• Working groups, lead by WG chairs

  • Language (Kirrily Robert)

  • Internals (Dan Sugalski)

  • Standard Library (Graeme Barr)

  • Build (Jarkko Hietaniemi)

  • QA (Michael Schwern)

  • Licensing (Bradley Kuhn)

  Each working group has a mailing list, and may spawn sub-lists (with chairs for each one) as required.

• Working groups to develop RFCs describing possible changes to make in Perl 6. Note that these are actually "Requests For Comment", not fixed standards as IETF RFCs tend to be.

• Larry would create an overall design for Perl 6 based on the best suggestions to come out of the RFC process.

Constraints on Perl 6 (Larry's list)

• Can Larry understand it?

• Do people really need the new feature?

• Can we implement it efficiently and robustly?

• Does it preclude translation from Perl 5?

• Does the utility grow faster than the complexity?

• Does the size grow slower than Moore's Law

• Must take time to maintain Perl 5

• Must take time to design Perl 6 right

Skud's psychic powers predict the future (with help from Larry's talk)

• Perl will still be Perl

  • TMTOWTDI ("There's more than one way to do it") -- just because one Perl language construct duplicates the functionality of another is not sufficient reason to avoid it.

  • Make easy things easy, hard things possible -- if a feature achieves this, it is in keeping with the Perl spirit and is likely to be considered for inclusion.

  • Perl is not Python, Java, C, BASIC, COBOL, Lisp... and will not become any of these, despite the loud arguments put forward by some proponents of these languages during the RFC process

  • ... but it should be easy for people from those languages to pick up

  • May also be easier to connect directly to those languages (e.g. Inline.pm, cleaner XS replacement, etc.)

• Language changes

  • Death to global variables! (especially punctuation vars such as $/); these are likely to be converted to some kind of OO interface.

  • Deprecated features removed. For instance, the use of ' as a synonym for :: will die a peaceful death of old age

  • Non-critical builtins move out to modules, including formats, shared memory functions, gethostbyname and friends, etc

  • Easier and safer OO programming

  • Easier functional programming

  • No more typeglobs (but none of their functionality lost)

  • Filehandles become objects

  • Easier interpolation of complex expressions

    print "The answer is $obj->answer()";

  • Optional typing of scalars (int, float, etc)

    my int $answer = 42; my int @array = (1, 2, 3);

  • Better subroutine parameter specification mechanism

  • A switch statement, based on Damian Conway's excellent (and very Perlish) suggested implementation

  • wantarray() becomes a more generic want()

• Standard library changes

  • Greater consistency

  • More applications development support: web, XML, CORBA, ???

  • Some stuff moved from the core to the library: date/time, shm*, formats, etc

• Internals changes

  • More modular

  • Easier to maintain

  • Much easier to extend

  • Better garbage collection

  • We'll know more when we start on detailed design and implementation

• Licensing changes

  • Possible rewrite of the Artistic License