Lecture 30 | Compiler I

Building the Jack Syntax Analyzer

From the Jack grammar to structured XML.

Jack Grammar

Three Layers of Syntax

Program structure

Classes, class variables, subroutines, parameters, and local variables.

Statements

let, if, while, do, and return.

Expressions

Terms, operators, array access, subroutine calls, constants, and variables.

The compilation unit is one class stored in one .jack file.

Program Structure

The Shape of a Jack Class

class:
  'class' className '{'
    classVarDec*
    subroutineDec*
  '}'

classVarDec:
  ('static' | 'field') type varName (',' varName)* ';'

All class-variable declarations precede all subroutine declarations; the grammar makes this ordering explicit.

Exercise 1

Match Source Lines to Grammar Elements

class Counter {
  field int value;
  constructor Counter new() { ... }
  method void increment() { ... }
}
CounterclassName
field int value;classVarDec
constructor ...subroutineDec
method ...subroutineDec
Statements

Five Alternatives, Recognized by One Token

statements: statement*

statement:
  letStatement | ifStatement | whileStatement |
  doStatement  | returnStatement
letifwhiledoreturn

These keywords allow compileStatements() to dispatch directly to the correct routine and to stop when the next token is not a statement keyword.

Exercise 2

Where Does the Statement Sequence End?

{
  let x = x + 1;
  do Output.printInt(x);
  return;
}
  1. Which routine handles each statement?
  2. What token tells compileStatements() to stop?
compileLet(), compileDo(), and compileReturn(); the closing } ends the sequence.
Expressions

An Expression Is a Sequence of Terms

expression: term (op term)*

term: integerConstant | stringConstant | keywordConstant |
      varName | varName '[' expression ']' | subroutineCall |
      '(' expression ')' | unaryOp term

Jack's grammar does not encode conventional operator precedence. Parentheses must express the required grouping.

Exercise 3

Distinguish Identifier-Led Terms

For each token prefix, choose the grammatical form.

x +variable
a [array entry
move (subroutine call
Math .qualified subroutine call

This is the principal place where the compilation engine needs one-token lookahead.

Analyzer Output

XML Is a Textual Parse Tree

Jack source code transformed by the syntax analyzer into nested XML elements

Nested XML elements make the parser's structural decisions visible and testable before code generation is added.

Analyzer Output

Terminals and Non-Terminals Use Different Tags

Terminal token
<keyword> return </keyword>
<symbol> ; </symbol>

Its tag records one of the five token types.

Non-terminal construct
<returnStatement>
  ... recursive contents ...
</returnStatement>

Its tag encloses the recursively parsed structure.

Exercise 4

Write the XML for a Return Statement

return;
<returnStatement>
  <keyword> return </keyword>
  <symbol> ; </symbol>
</returnStatement>

No <expression> appears because the expression in the grammar is optional.

Software Architecture

Three Modules, Three Responsibilities

JackAnalyzerfind inputs and coordinate the work
→
JackTokenizercharacters → typed tokens
→
CompilationEnginetokens → structured XML

Keeping these responsibilities separate allows each part to be implemented and tested independently.

JackAnalyzer

The Top-Level Driver

  1. Accept one Xxx.jack file or a directory containing Jack files.
  2. For each source file, create a JackTokenizer.
  3. Create the corresponding Xxx.xml output file.
  4. Create a CompilationEngine and begin with compileClass().

Xxx.jack → Xxx.xml

CompilationEngine

Every Routine Has the Same Contract

Precondition: construct xxx is next in the token stream.

Work: recognize terminals, recursively compile its non-terminals, and emit XML tags.

Postcondition: the tokenizer is positioned exactly after xxx.

compileClass()compileStatements()compileWhile()compileExpression()compileTerm()
Exercise 5

Plan compileWhile()

whileStatement:
  'while' '(' expression ')' '{' statements '}'

Write the sequence of emit, consume, and recursive-call operations.

  1. Open <whileStatement>; consume while and (.
  2. Call compileExpression(); consume ) and {.
  3. Call compileStatements(); consume }.
  4. Close </whileStatement>.
Implementation Strategy

Build and Test in Two Stages

1Tokenizer

Generate XxxT.xml. Compare it with the supplied token files before beginning the parser.

2Parser

Begin with expressionless programs, then add expressions and test complete Jack programs.

Suggested progressionExpressionlessSquare→Square→ArrayTest

Use the supplied comparison files as the contract. Indentation is for readability; the XML structure and tokens must match.

Summary of This Week

The Syntax Analyzer Makes Structure Explicit

CharactersJack source
→
Tokenslexical analysis
→
Grammar structurerecursive descent
→
XMLtestable parse tree

Next chapter: keep the parser, add semantic information, and generate executable VM code.