Parsing Templates and AST

Next, we'll look at the parser and AST implemented in compiler-core.
The SFC parser uses this.

AST

AST stands for Abstract Syntax Tree.

It is probably the most complex intermediate object that the Vue.js compiler has.
Information such as directives, mustaches, slots, etc., are represented as ASTs here.

The implementation is in ast.ts of compiler-core.

packages/compiler-core/src/ast.ts

Let's read through the overall structure.

Looking at the types of Nodes, we can see that there are several categories.

vuejs/core-vapor/packages/compiler-core/src/ast.tsLines 29 to 61 in 30583b9ee1

29export enum NodeTypes {
30  ROOT,
31  ELEMENT,
32  TEXT,
33  COMMENT,
34  SIMPLE_EXPRESSION,
35  INTERPOLATION,
36  ATTRIBUTE,
37  DIRECTIVE,
38  // containers
39  COMPOUND_EXPRESSION,
40  IF,
41  IF_BRANCH,
42  FOR,
43  TEXT_CALL,
44  // codegen
45  VNODE_CALL,
46  JS_CALL_EXPRESSION,
47  JS_OBJECT_EXPRESSION,
48  JS_PROPERTY,
49  JS_ARRAY_EXPRESSION,
50  JS_FUNCTION_EXPRESSION,
51  JS_CONDITIONAL_EXPRESSION,
52  JS_CACHE_EXPRESSION,
53
54  // ssr codegen
55  JS_BLOCK_STATEMENT,
56  JS_TEMPLATE_LITERAL,
57  JS_IF_STATEMENT,
58  JS_ASSIGNMENT_EXPRESSION,
59  JS_SEQUENCE_EXPRESSION,
60  JS_RETURN_STATEMENT,
61}

• Plain
• Containers
• Codegen
• SSR Codegen

To conclude, codegen and ssr codegen are not related to the Vapor compiler.
This relates to concepts we'll explain later, such as IR and transform, but in Vapor Mode, information for codegen is aggregated in IR.
However, in traditional Vue.js (non-Vapor Mode), there is no concept of IR, and even the output code was represented as AST.
In Vapor Mode, the transformer converts the AST into IR, but in Vue.js (non-Vapor Mode), the AST (Plain, Containers) is converted into AST (Codegen, SSR Codegen) and passed to the codegen.

This time, to explain the design of the Vapor Mode compiler, we won't touch on codegen and ssr codegen.
Let's look at the others!

Plain

First, the basic AST Node types without any specific category.

vuejs/core-vapor/packages/compiler-core/src/ast.tsLines 29 to 37 in 30583b9ee1

29export enum NodeTypes {
30  ROOT,
31  ELEMENT,
32  TEXT,
33  COMMENT,
34  SIMPLE_EXPRESSION,
35  INTERPOLATION,
36  ATTRIBUTE,
37  DIRECTIVE,

Root

As the name suggests, Root represents the root of the template.
It has Nodes in its children.

vuejs/core-vapor/packages/compiler-core/src/ast.tsLines 111 to 128 in 30583b9ee1

111export interface RootNode extends Node {
112  type: NodeTypes.ROOT
113  source: string
114  children: TemplateChildNode[]
115  helpers: Set<symbol>
116  components: string[]
117  directives: string[]
118  hoists: (JSChildNode | null)[]
119  imports: ImportItem[]
120  cached: (CacheExpression | null)[]
121  temps: number
122  ssrHelpers?: symbol[]
123  codegenNode?: TemplateChildNode | JSChildNode | BlockStatement
124  transformed?: boolean
125
126  // v2 compat only
127  filters?: string[]
128}

Element

Element is a Node representing an element.
Elements like <p> or <div> correspond to this.
Components and slots also correspond to this.

These also have Nodes in their children.
They also have attribute information and directive information.

vuejs/core-vapor/packages/compiler-core/src/ast.tsLines 136 to 175 in 30583b9ee1

136export interface BaseElementNode extends Node {
137  type: NodeTypes.ELEMENT
138  ns: Namespace
139  tag: string
140  tagType: ElementTypes
141  props: Array<AttributeNode | DirectiveNode>
142  children: TemplateChildNode[]
143  isSelfClosing?: boolean
144  innerLoc?: SourceLocation // only for SFC root level elements
145}
146
147export interface PlainElementNode extends BaseElementNode {
148  tagType: ElementTypes.ELEMENT
149  codegenNode:
150    | VNodeCall
151    | SimpleExpressionNode // when hoisted
152    | CacheExpression // when cached by v-once
153    | MemoExpression // when cached by v-memo
154    | undefined
155  ssrCodegenNode?: TemplateLiteral
156}
157
158export interface ComponentNode extends BaseElementNode {
159  tagType: ElementTypes.COMPONENT
160  codegenNode:
161    | VNodeCall
162    | CacheExpression // when cached by v-once
163    | MemoExpression // when cached by v-memo
164    | undefined
165  ssrCodegenNode?: CallExpression
166}
167
168export interface SlotOutletNode extends BaseElementNode {
169  tagType: ElementTypes.SLOT
170  codegenNode:
171    | RenderSlotCall
172    | CacheExpression // when cached by v-once
173    | undefined
174  ssrCodegenNode?: CallExpression
175}

Text

Text is, as the name suggests, Text.
In <p>hello</p>, hello corresponds to this.

vuejs/core-vapor/packages/compiler-core/src/ast.tsLines 183 to 186 in 30583b9ee1

183export interface TextNode extends Node {
184  type: NodeTypes.TEXT
185  content: string
186}

Comment

Comment is a comment.
<!-- comment --> corresponds to this.

vuejs/core-vapor/packages/compiler-core/src/ast.tsLines 188 to 191 in 30583b9ee1

188export interface CommentNode extends Node {
189  type: NodeTypes.COMMENT
190  content: string
191}

SimpleExpression

SimpleExpression is a simple expression that appears in the template. It's a bit difficult to explain what is simple and what is not, but for example, a and o.a are simple, while (() => 42)() is not simple.

The foo in {{ foo }} and the handlers.onClick in <button @click="handlers.onClick"> correspond to this.

vuejs/core-vapor/packages/compiler-core/src/ast.tsLines 232 to 254 in 30583b9ee1

232export interface SimpleExpressionNode extends Node {
233  type: NodeTypes.SIMPLE_EXPRESSION
234  content: string
235  isStatic: boolean
236  constType: ConstantTypes
237  /**
238   * - `null` means the expression is a simple identifier that doesn't need
239   *    parsing
240   * - `false` means there was a parsing error
241   */
242  ast?: BabelNode | null | false
243  /**
244   * Indicates this is an identifier for a hoist vnode call and points to the
245   * hoisted node.
246   */
247  hoisted?: JSChildNode
248  /**
249   * an expression parsed as the params of a function will track
250   * the identifiers declared inside the function body.
251   */
252  identifiers?: string[]
253  isHandlerKey?: boolean
254}

Interpolation

This is a mustache.
{{ foo }} corresponds to this.

vuejs/core-vapor/packages/compiler-core/src/ast.tsLines 256 to 259 in 30583b9ee1

256export interface InterpolationNode extends Node {
257  type: NodeTypes.INTERPOLATION
258  content: ExpressionNode
259}

Attribute

This corresponds to attributes (not directives).
In <div id="app">, id="app" corresponds to this.

vuejs/core-vapor/packages/compiler-core/src/ast.tsLines 193 to 198 in 30583b9ee1

193export interface AttributeNode extends Node {
194  type: NodeTypes.ATTRIBUTE
195  name: string
196  nameLoc: SourceLocation
197  value: TextNode | undefined
198}

Directive

This is a directive.

v-on:click="handler" and v-for="item in items" correspond to this.
Of course, shorthand notations like @click="handler" and #head are also included.

vuejs/core-vapor/packages/compiler-core/src/ast.tsLines 200 to 218 in 30583b9ee1

200export interface DirectiveNode extends Node {
201  type: NodeTypes.DIRECTIVE
202  /**
203   * the normalized name without prefix or shorthands, e.g. "bind", "on"
204   */
205  name: string
206  /**
207   * the raw attribute name, preserving shorthand, and including arg & modifiers
208   * this is only used during parse.
209   */
210  rawName?: string
211  exp: ExpressionNode | undefined
212  arg: ExpressionNode | undefined
213  modifiers: string[]
214  /**
215   * optional property to cache the expression parse result for v-for
216   */
217  forParseResult?: ForParseResult
218}

Containers

Containers are Nodes with specific structures.

vuejs/core-vapor/packages/compiler-core/src/ast.tsLines 39 to 43 in 30583b9ee1

39  COMPOUND_EXPRESSION,
40  IF,
41  IF_BRANCH,
42  FOR,
43  TEXT_CALL,

The order might be a bit out of sequence, but let's look at the easier ones first.

If, IfBranch

If and IfBranch are Nodes represented by v-if, v-else-if, and v-else.

vuejs/core-vapor/packages/compiler-core/src/ast.tsLines 286 to 298 in 30583b9ee1

286export interface IfNode extends Node {
287  type: NodeTypes.IF
288  branches: IfBranchNode[]
289  codegenNode?: IfConditionalExpression | CacheExpression // <div v-if v-once>
290}
291
292export interface IfBranchNode extends Node {
293  type: NodeTypes.IF_BRANCH
294  condition: ExpressionNode | undefined // else
295  children: TemplateChildNode[]
296  userKey?: AttributeNode | DirectiveNode
297  isTemplateIf?: boolean
298}

Structurally, an IfNode has multiple IfBranchNodes, and an IfBranchNode has a condition (condition) and children (Nodes when that condition is met).
In the case of v-else, condition becomes undefined.

For

This is the Node represented by v-for.

vuejs/core-vapor/packages/compiler-core/src/ast.tsLines 300 to 309 in 30583b9ee1

300export interface ForNode extends Node {
301  type: NodeTypes.FOR
302  source: ExpressionNode
303  valueAlias: ExpressionNode | undefined
304  keyAlias: ExpressionNode | undefined
305  objectIndexAlias: ExpressionNode | undefined
306  parseResult: ForParseResult
307  children: TemplateChildNode[]
308  codegenNode?: ForCodegenNode
309}

vuejs/core-vapor/packages/compiler-core/src/ast.tsLines 311 to 317 in 30583b9ee1

311export interface ForParseResult {
312  source: ExpressionNode
313  value: ExpressionNode | undefined
314  key: ExpressionNode | undefined
315  index: ExpressionNode | undefined
316  finalized: boolean
317}

In <div v-for="it in list">, source becomes list, and value becomes it.

CompoundExpression

This is a somewhat hard-to-understand concept.

compound means "composite", and this Node is composed of multiple Nodes.

vuejs/core-vapor/packages/compiler-core/src/ast.tsLines 261 to 284 in 30583b9ee1

261export interface CompoundExpressionNode extends Node {
262  type: NodeTypes.COMPOUND_EXPRESSION
263  /**
264   * - `null` means the expression is a simple identifier that doesn't need
265   *    parsing
266   * - `false` means there was a parsing error
267   */
268  ast?: BabelNode | null | false
269  children: (
270    | SimpleExpressionNode
271    | CompoundExpressionNode
272    | InterpolationNode
273    | TextNode
274    | string
275    | symbol
276  )[]
277
278  /**
279   * an expression parsed as the params of a function will track
280   * the identifiers declared inside the function body.
281   */
282  identifiers?: string[]
283  isHandlerKey?: boolean
284}

Examples like {{ foo }} + {{ bar }} correspond to this.

Intuitively, this might seem like a structure of Interpolation + Text + Interpolation, but
the Vue.js compiler treats these together as a CompoundExpression.

The noteworthy point is that types like string and Symbol can be seen in children.

vuejs/core-vapor/packages/compiler-core/src/ast.tsLines 269 to 275 in 30583b9ee1

269  children: (
270    | SimpleExpressionNode
271    | CompoundExpressionNode
272    | InterpolationNode
273    | TextNode
274    | string
275    | symbol

This is a mechanism to treat partial strings not as some AST Node but as literals for simplicity.

Between {{ foo }} + {{ bar }}, the + part of the string is more efficient to treat as a literal " + " rather than expressing it as a Text Node.

It's like an AST as follows:

{
  "type": "CompoundExpression",
  "children": [
    { "type": "Interpolation", "content": "foo" },
    " + ",
    { "type": "Interpolation", "content": "bar" }
  ]
}

TextCall

vuejs/core-vapor/packages/compiler-core/src/ast.tsLines 319 to 323 in 30583b9ee1

319export interface TextCallNode extends Node {
320  type: NodeTypes.TEXT_CALL
321  content: TextNode | InterpolationNode | CompoundExpressionNode
322  codegenNode: CallExpression | SimpleExpressionNode // when hoisted
323}

This is a Node used when expressing Text as a function call createText.
For now, you don't need to worry too much about it.

---

So far, we've looked at the necessary AST Nodes.
From here, let's look at the implementation of the parser that generates these ASTs!