ANTLR4 parser for Python 3.13.1  

About files:

• PythonParser.g4 is the ANTLR4 parser grammar that based on the official Python PEG grammar

• PythonLexerBase class

  • handles the Python indentations
  • creates encoding token
  • tokenizes fstring literals
  • and manage many other things

• Example files from: Python 3.13.1 Standard Lib
  
  

Recent changes:

• parser grammar update for Python 3.13.1
• added ENCODING token
• complete rewrite of fstring tokenizer in lexer grammar and PythonLexerBase class
  • now correctly tokenizes the followings in fstring:
    • escape sequences
    • walrus operator
    • dictionary comprehension
    • set comprehension
• soft keywords changes:
  • no embedded code (semantic predicates) in parser grammar for soft keywords
  • no need for PythonParserBase class
  • no need for transformGrammar.py
  • BREAKING CHANGES:
    • dedicated tokens for soft keywords instead of NAME token:
      • NAME_OR_TYPE
      • NAME_OR_MATCH
      • NAME_OR_CASE
      • NAME_OR_WILDCARD

Previous changes

Related links:

ANTLR 4

ANTLR4-tools

ANTLR 4 Documentation

ANTLR 4 Runtime API

Python 3.13 Lexical Analysis

cpython

ANTLR4 parser for Python 2.7.18

ANTLR4 parser for Python 3.8