Moving printf formats forward

One of the advantages of RakuAST is that you can write executable code in an object-oriented style. One area that really gets your attention in this area is handling sprintf Formats

Historically, the C programming language provided the first implementation printf Logic:

The printf family of functions in the C programming language is a set of functions that take a format string as input among a variable-sized list of other values ​​and produce a string as output that matches the format specifier and the input values. given matches.

And in Raku?

Of course, the Raku programming language also has an implementation printf, with a set of supported options. However, the Rakudo implementation does not implement the Raku programming language printf It has its own formats, but it borrows from it NQP. which has its advantages and disadvantages.

Of course, the positives are that NQP is generally much faster than Raku (since NQP is sort of the “assembly language” of Rakudo). However, NQP implementation of sprintf Written using grammar (written in NQP). And Each When a call is made to this sprintf function (for example, when printing a line in a report using sprintf), the entire syntax is re-executed, with the associated actions that construct the string to be returned. And the implementation of the grammar is expensive in terms of memory And CPU.

This causes rakudo sprintf and related functions (eg .fmt) One of the slowest raku features in Rakudu. Surely there must be a better way to do this with RakuAST?

Enter the formatter

Almost two years ago, your journey truly began at RakuAST. Try RakuAST’s features later by writing a test first. And one of the first things that was done was porting the original NQP grammar sprintf To Raku and create a separate set of actions, which becomes No produces a string, but produces an AST instead.

Such an AST can then be converted to executable code, and then executed whenever a set of arguments needs to be converted to a string. So instead of needing to execute the same grammar, just a few (static) codes can be called, which is then optimized for better performance at runtime, just like any other piece of code in Raku.

This was then converted Formatter class and since this was a brand new feature, it was only made available by using it 6.e.PREVIEW The language version was then largely neglected for the next 1.5 years. As it is clear, the time is not yet right for that.

obstacles

One of the obstacles he encountered while working Formatterwas there No Comprehensive tests for sprintf Functionality for Raku. yes there was A A test file that tested some properties, but there was no comprehensive test for all possible combinations of template properties, along with the possible values ​​they should work on.

This was one of the first things that had to be done, so that the new implementation could be said to match the old. During the development of these tests, it became clear that there were some inconsistencies in the existing implementation, and worse: obvious bugs.

So the question arose: should the new implementation follow the behavior of the old implementation or not?

Raku nuclear meeting

This then became part of the discussion at the first Raku Nuclear Summit. There it was decided that the new implementation would not adhere to the previous implementation because the new sprintf However, the function needs a language version. And this flow sprintf Tests should be fixed for language level 6.d. And so it was done.

Another decision that was made was that the performance provided by Formatter class (which converts a format string to a Callable) must be embedded in a Format A class that should normally act as a string, but when used as a Callable It should perform the processing according to the given format. This is now implemented. That means you can now do:

use v6.e.PREVIEW;
my $f = Format.new("%5s");
dd $f;              # Format.new("%5s")
say $f;             # %5s
dd $f("foo");       # "  foo"
say "'$f'";         # '%5s'
say "'$f("foo")'";  # '  foo'

As you see, Format The object acts much like a string. Which is not so strange when you notice the method separation order Format:

use v6.e.PREVIEW;
say Format.^mro;  # ((Format) (Str) (Cool) (Any) (Mu))

Only when as a Callable (by placing () after that with arguments), does it show its special function.

A new quotation adverb

In RCS it was suggested that Format.new("%5s") In normal use it would be too bad. More seriously, it prevents compile-time generation Format Object because lookup methods are a runtime operation in Raku (in general). A way around it is to introduce a new construct/adverb/processor with string quotes. it became "format" Adverb (or "o" For the short version):

use v6.e.PREVIEW;  # RAKUDO_RAKUAST=1 also required for now
my $format = q:format/%5s/;
say "'$format("foo")'";   # '  foo'

or more directly and shorter by using the alternative adverb “o”:

use v6.e.PREVIEW;
dd q:o/%5s/("foo");  # "  foo"

Why "o" For the short version? Because "f" Previously taken to enable/disable *f*function interpolation. And q:o Visually closer to fo than anything else

Performance

So what does this new method of formatting values ​​into a string do?

Without any optimization of the RakuAST version, up to 30x speedup has been reported. So this is quite an improvement. And potentially better than many other programming languages ​​that also depend on some sort of runtime mode for thread creation, rather than on executable code.

Result

A new way to create strings from a given set of values ​​and a formatting string (ie printf performance) is implemented using RakuAST in the Raku programming language, making it up to 30 times faster.

This feature will be available from Rakudo version 2023.06 and its selection 6.e Language level by specifying use v6.e.PREVIEW. The new quote adverb will now only be available when compiled with the new RakuAST grammar, which can be specified by specifying RAKUDO_RAKUAST environmental variable.