VisualLangLab - Using the API

The VisualLangLab API is used by client programs to load a grammar file (created in and saved by the VisualLangLab GUI), and create a parser from it. The API is written in Java, and may be used by client programs in any JVM language. The API is quite small and simple, and is best understood in the context of a real example. Using the API requires a good knowledge of the AST Structure and action-code design.

The rest of this article describes a compact example based on one of the sample grammars bundled with the VisualLangLab GUI. Another, somewhat larger, example can be found in Evaluating Expressions Using an AST.

An Arithmetic Expression Parser

The following example uses the API to load the PS2E-ArithExpr sample grammar (see Sample Grammars) into a client Java program. The grammar's top-level rule Expr is then fetched and applied to the string "(3 + 5) / (8 - 4)" to obtain an AST which is then interpreted to obtain the value of the expression. For better clarity, the program is organized as 4 functions with distinct roles:

• The main() function loads the grammar from a file, fetches the top-level rule, and applies the parser to the string. It checks the result, and in case of success passes the AST to the function evalExprAST()
• The function evalExprAST() interprets the complete expression's AST
• The function evalTermAST() interprets the AST produced by the term rule
• The function evalFactorAST() interprets the AST from the factor rule

The main() function is shown first below. The colored parts depend on the VisualLangLab API as described after the code below.

import java.io.File;
import java.io.IOException;
import java.util.List;
import javax.xml.parsers.ParserConfigurationException;
import org.xml.sax.SAXException;
import vll4j.Vll4j;

public static void main(String[] args) throws 
    ParserConfigurationException, SAXException, IOException {
    String input = "(3 + 5) * (8 - 4)";
    Vll4j vll = Vll4j.fromFile(new File("PS2E-ArithExpr.vll"));
    Vll4j.Parser exprParser = vll.getParserFor("Expr");
    Vll4j.ParseResult parseResult = vll.parseAll(exprParser, input);
    if (parseResult.successful()) {
        Object ast = parseResult.get();
        Float result = evalExprAST(ast);
        System.out.println(result);
    } else {
        System.out.println(parseResult);
    }
}

The colored parts are all dependent on the VisualLangLab API. The entities in red are entirely VisualLangLab specific, but those in blue follow the design of synonymous entities in the Scala parser combinator package. Although all of the API is implemented in Java, the design of a small part of the Scala API has been adapted to leverage the advantages of a proven, well-understood and documented design. The purpose and use of the entities highlighted above is explained in the table below:

Most API users will never need to use other features of the classes descrbed above. More detailed information about the design of these classes can be obtained from the hyperlinks (to the Scala API) given above. The Java source-code of these classes (and other utility classes like Reader) can be found in the vll4j.core package.

The main() function (above) passes the entire AST to another function. The other functions shown below do not need the VisualLangLab API—they merely process the AST (which contains standard Java/JVM data types).

Handling the Complete AST

The evalExprAST() function processes the entire AST (from main()). The table below places the function's code alongside the AST structure to help you understand the logic better.

Array(
|  @term,
|  List(
|  |  Choice(
|  |  |  Array(0, @term),
|  |  |  Array(1, @term)
|  |  )
|  )
)

01 static Float evalExprAST(Object ast) {
02     Object[] array = (Object[]) ast;
03     Float term = evalTermAST(array[0]);
04     for (Object pair[] : (List<Object[]>) array[1]) {
05         if (pair[0].equals(0)) {
06             term += evalTermAST(pair[1]);
07         } else if (pair[0].equals(1)) {
08             term -= evalTermAST(pair[1]);
09         }
10     }
11     return term;
12 }

The variable array on line-2 corresponds to the AST's outermost layer (Array). Observe how the two elements of this array are used in line-3 (to define the variable term), and on line-4 (within the for to iterate the contained list). The elements of the list (each a 2-element array) are successively assigned to the variable pair (in line-4). The first element of this array is tested by the if (lines 5 and 7), and the second element of the array is respectively used to add to or subtract from the running total.

As shown by the AST, this rule references rule term at two places, so those portions of the AST must be handed off to another function for further processing. That function evalTermAST() is described below.

Handling Rule term's AST

The evalTermAST() function processes the AST of the rule term. The table below places the function's code alongside the AST structure to help you understand the logic better.

Array(
|  @factor,
|  List(
|  |  Choice(
|  |  |  Array(0, @factor),
|  |  |  Array(1, @factor)
|  |  )
|  )
)

01 static Float evalTermAST(Object ast) {
02     Object[] array = (Object[]) ast;
03     Float factor = evalFactorAST(array[0]);
04     for (Object pair[] : (List<Object[]>) array[1]) {
05         if (pair[0].equals(0)) {
06             factor *= evalFactorAST(pair[1]);
07         } else if (pair[0].equals(1)) {
08             factor /= evalFactorAST(pair[1]);
09         }
10     }
11     return factor;
12 }

The AST of this rule (term) is structurally identical to the previous case (Expr), so the code is also structurally identical. The only difference is in the identity of the rule (factor in this case) to which the sub-AST is handed off for further processing, and the arithmetic operators (*= and /= in this case) used for updating the cumulated value.

Handling Rule factor's AST

The evalFactorAST() function processes the AST of the rule factor. The table below places the function's code alongside the AST structure to help you understand the logic better.

Choice(
|  Array(0, [floatingPointNumber]),
|  Array(1, @Expr)
)

01 static Float evalFactorAST(Object ast) {
02     Object[] pair = (Object[]) ast;
03     Float factorResult = -1f;
04     if (pair[0].equals(0)) {
05         factorResult = Float.parseFloat((String) pair[1]);
06     } else if (pair[0].equals(1)) {
07         factorResult = evalExprAST(pair[1]);
08     }
09     return factorResult;
10 }

The top-level of this AST is a Choice, meaning that one of the two array objects defined in the Choice will be received. Line-2 of the code therefore assigns the AST as received to pair which is an array of two Objects. It then proceeds to test the first element of the array to help it to decide how to process the second element.

Boxing and Unboxing

The last example above (funcion evalFactorAST()) makes it clear that autoboxing is at work on the API side. How else would an array of Object's carry an integer? The API user should be aware of this and exploit Java's unboxing capabilities appropriately.

Running the Code

To compile and run the code, proceed as follows.

1. Assemble the functions and includes shown above into a single Java class
2. In the VisualLangLab GUI, save the PS2E-ArithExpr sample grammar to a file by invoking File -> SaveAs, and enter PS2E-ArithExpr.vll as the file-name (this name is used in the main() function)
3. To compile the program you must have VLL4J.jar on the classpath
4. To run the program, use the Java launcher with both, VLL4J.jar and the directory containing the class-files, on the class path. The file PS2E-ArithExpr.vll created above must also be in the working directory.

The program should print out 32 when run as-is. You should change the expression parsed by the program (assigned to the variable input in the main() function) to check out the parser completely.

Code with Embedded Grammar

It is sometimes required to completely embed all resources (including the grammar) within a program. A couple of options are available for this:

1. Extract the contents of the grammar-file, and use it as a long string inside the program. A grammar can be loaded from a String by using the static Vll4j Vll4j.fromString(String) function. This approach is convenient only for small grammars
2. Package the client program as well as the grammar-file in a JAR file. Use the Java ClassLoader's getResourceAsStream(String) method to get an InputStream, and load the grammar by passing the stream to the static Vll4j Vll4j.fromStream(InputStream) function