The IronMeta Language

This section is an informal introduction to the features of the IronMeta language.

It uses the following IronMeta file named Calc.ironmeta, which is included in the IronMeta distribution.

The Calc grammar is much more complex than it needs to be, in order to demonstrate some of the advanced functionality of IronMeta.

// IronMeta Calculator Example

using System;
using System.Linq;

ironMeta Calc<char, int> : IronMeta.CharacterMatcher<int>
{
    Expression = Additive;
    
    Additive = Add | Sub | Multiplicative;
        
    DecimalDigit = .:c ?( c >= '0' && c <= '9' ) -> { return (int)c - '0'; };
    
    Add = BinaryOp(Additive, '+', Multiplicative) -> { return _IM_Result.Results.Aggregate((total, n) => total + n); };
    Sub = BinaryOp(Additive, '-', Multiplicative) -> { return _IM_Result.Results.Aggregate((total, n) => total - n); };

    Multiplicative = Multiply | Divide;
    Multiplicative = Number(DecimalDigit);
    
    Multiply = BinaryOp(Multiplicative, "*", Number, DecimalDigit) 
            -> { return _IM_Result.Results.Aggregate((p, n) => p * n); };
    Divide = BinaryOp(Multiplicative, "/", Number, DecimalDigit) 
            -> { return _IM_Result.Results.Aggregate((q, n) => q / n); };
        
    BinaryOp :first :op :second .?:type = first:a KW(op) second(type):b -> { return new List<int> { a, b }; };
    
    Number :type = Digits(type):n Whitespace* -> { return n; };
    
    Digits :type = Digits(type):a type:b -> { return a*10 + b; };
    Digits :type = type;
            
    KW :str = str Whitespace*;
}

We will go through this example line by line to introduce the IronMeta language:

Comments

// IronMeta Calculator Example

You may include comments anywhere in the IronMeta file. They may also be in the C-style form:

/* C-Style Comment */

Preamble

using System;
using System.Linq;

You can include C# using statements at the beginning of an IronMeta file. IronMeta will automatically add using statements to its output to include the namespaces it needs.

Parser Declaration

ironMeta Calc<char, int> : IronMeta.CharacterMatcher<int>

An IronMeta parser always starts with the keyword ironMeta. Then comes the name of the parser (Calc, in this case), and the input and output types. The generated parser will take as input an IEnumerable of the input type, and return as output an IEnumerable of the output type.

In this case, the Calc parser will operate on a stream of char values, and output a stream of int values. We will define the parser so that the output list only includes one value.

Note:

You must always include the input and output types.

You may also optionally include a base class:

 : IronMeta.CharacterMatcher<int>

If you do not include a base class, your parser will inherit directly from IronMeta.Matcher. The IronMeta.CharacterMatcher class provides some specialized functionality for dealing with streams of characters.

Rules

    Expression = Additive;

An IronMeta rule consists of a name, an pattern for matching parameters, "=", a pattern for matching against the main input, and a terminating semicolon ";" (for folks used to C#) or comma "," (for folks used to OMeta):

    IronMetaRule = "ironMeta" Pattern "=" Pattern (";" | ",");

In this case, the rule Expression has no parameters, and matches by calling another rule, Additive.

Matching Input

You can use the period "." to match any item of input, or you can use arbitrary C# expressions. The C# expressions may be a string literal, a character literal, or any other expression that is surrounded by curly braces:

    MyPattern = 'a' "b" {3.14159} {new MyClass()};

IronMeta will use the standard C# Equals() method to match the items.

The pattern literal can also be an IEnumerable of the input type -- IronMeta does not contain any special string-matching functionality; the fact that string implements IEnumerable<char> allows us to use C# strings directly.

This eliminates the need for the OMeta token function; just use a string literal, or if you are matching on something other than characters, use a list:

    MyPattern = {new List<MyInputType>{ a, b, c }};

Sequence and Disjunction

    Additive = Add | Sub | Multiplicative;

As is probably obvious from the other rules, you write a sequence of patterns by simply writing them one after the other, separated by whitespace.

To specify a choice between alternatives, separate them with "|".

Note:

Unlike in other parser generator formalisms, separating expressions with a carriage return does NOT mean they are alternatives! You must always use the "|".

Other Operators

You can modify the meaning of patterns with the following operators:

• "?" as a suffix will match zero or one times.
• "*" as a suffix will match zero or more times.
• "+" as a suffix will match one or more times.

In strict PEG mode these operators are all greedy -- they will match as many times a possible and then return that result. If you want your parsers to be able to backtrack, you will need to disable strict PEG mode.

• "&" as a prefix will match an expression but NOT advance the match position. This allows for unlimited lookahead.
• "~" as a prefix will match if the expression does NOT match. It will not advance the match position.

Conditions and Actions

    DecimalDigit = .:c ?( c >= '0' && c <= '9' ) -> { return (int)c - '0'; };

Here things get more interesting. This rule has only one expression, the period ".". This will match a single item of input. It is then bound to the variable c by means of the colon ":".

Note:

You can leave out the period if you are binding to a variable; that is, ":c" is equivalent to ".:c".

However, this rule will not actually match any character, because it contains a condition. A condition is written with "?" followed immediately by a C# expression in parentheses. The C# expression must evaluate to a bool value. Once the expression matches (in this case it will match anything), it is bound to the variable c, which is then available for use in your C# code.

The rule also contains an action. Actions are written with "-\>" followed by a C# block surrounded by curly braces. This block must contain a return statement that returns a value of the output type, or a List<> of the output type.

In this case, the variable c is explicitly cast to an int in order to force the variable to return its result, because otherwise C# would implicitly cast it to char because of the '0' in the expression.

Note:

If you do not provide an action for the expression, it will simply return the results of its patterns, as a list. Matching a single item will return default(TResult) by default, or you can pass a delegate or lambda function to the matcher when you create it that will convert values of the input type to the output type.

Be aware that an action only applies to the last expression in an OR expression. So the action in the following:

    MyRule = One | Two | Three -> { my action };

will only run if the expression Three matches! If you want an action to apply on an OR, use parentheses:

    MyRule = (One | Two | Three) -> { my action };

Variables

    Digits :type = Digits(type):a type:b -> { return a*10 + b; };

Upon a successful match, variables will contain information about the results of the match of the expression they are bound to. In this example, because a & b are used in an expression containing an integer, they will automatically evaluate to the results of their expressions, because the result type of the Calc grammar is int.

IronMeta variables are very flexible. They contain implicit cast operators to:

• A single value of the input type: this will return the last item in the list of results of the expression that the variable is bound to.
• A single value of the output type.
• A List<> of the input type.
• A List<> of the output type.

If your input and output types are the same, the implicit cast operators will only return the inputs, and you will need to use the explicit variable properties:

• c.Inputs returns the list of inputs that the parse pattern matched.
• c.Results returns the list of results that resulted from the match.
• c.StartIndex returns the index in the input stream at which the pattern started matching.
• c.NextIndex returns the first index in the input stream after the pattern match ended.

You can also use variables in a pattern, in which case they will match whatever input they matched when they were bound. Or, if they were bound to a rule in a parameter pattern (see below), they will call that rule. You can even pass parameters to them.

Built-In Variables

IronMeta automatically defines some variables for use in your C# code:

• _IM_Result: bound to the entire expression that your condition or action applies to.
• _IM_StartIndex: an int that holds the index at which the match starts.
• _IM_NextIndex: an int that holds the index after the match ends.

Multiple Rule Bodies

    Multiplicative = Multiply | Divide;
    Multiplicative = Number(DecimalDigit);

You can have multiple rule bodies; their patterns will be combined in one overall OR when that rule is called.

Parameters

    Add = BinaryOp(Additive, '+', Multiplicative) -> { return _IM_Result.Results.Aggregate((total, n) => total + n); };

This rule shows that you can pass parameters to a rule. You can pass literal match patterns, rule names, or variables.

    BinaryOp :first :op :second .?:type = first:a KW(op) second(type):b -> { return new List<int> { a, b }; };

This rule demonstrates how to match parameters. The parameter part of a rule is actually a matching pattern no different than that on the right-hand side of the "=" ! Using this fact, plus the ability to specify multiple rules with the same name, you can write rules that match differently depending on the number and kind of parameters they are passed.

Rules as Arguments

    Add = BinaryOp(Additive, '+', Multiplicative) -> { return _IM_Result.Results.Aggregate((total, n) => total + n); };
    BinaryOp :first :op :second .?:type = first:a KW(op) second(type):b -> { return new List<int> { a, b }; };

These rules show that you can pass rules as parameters to other rules. To match against them, just capture them in a variable in your parameter pattern, and then use the variable as an expression in your pattern. You can pass parameters as usual.

Pattern Matching Arguments

The example above shows that you can define different behavior depending on the parameters to a rule.

List Folding

    KW :str = str Whitespace*;

If you look at the rules that call this rule (indirectly through the BinaryOp rule), you'll see that they pass both a single character and a string:

    Sub = BinaryOp(Additive, '-', Multiplicative) -> { return _IM_Result.Results.Aggregate((total, n) => total - n); };
    Divide = BinaryOp(Multiplicative, "/", Number, DecimalDigit) -> { return _IM_Result.Results.Aggregate((q, n) => q / n); };

When matching against variables captured in parameters, variables containing single items or variables containing lists will match correctly.

Rule Inheritance

    Number :type = type+:digits Whitespace* -> { return digits.Results.Aggregate(0, (sum, n) => sum*10 + n); };

You may have noticed that there is no rule in the file called Whitspace! Because IronMeta matchers are C# classes, and their rules are methods, they can inherit rules from base classes. You can call these rules just like any others.

In this case, the CharacterMatcher class contains a rule called Whitespace which matches any whitespace character, so the parser compiles and runs just fine.

CharacterMatcher

The CharacterMatcher class provides the following rules:

• Whitespace: matches a whitespace character.
• EOL: matches end-of-line. This function is useful in that it records the positions of the ends of lines for use later.
• EOF: matched end-of-file.

It also provides the following utility functions:

• _IM_GetText: you can use this in a condition or action to get a string corresponding to the input matched by a pattern expression.
• GetLineNumber(): you can use this after you have finished parsing to get the line number that a character at the given index is in.
• GetLine(): you can use this after you have finished parsing to get a particular line of text from your input stream.