djthoms djthoms - 5 months ago 56
Java Question

Create Java 8 AST with ANTLR4 programmatically

I'm trying to figure out just how exactly to use ANTLR, but I'm having a really difficult time digesting the things I've found. So far, here are my resources:



A little bit of background



I'm experimenting with moving from JavaParse to ANTLR because I want to handle ASTs of languages besides Java. My understanding of ANTLR and the predefined grammars (linked above) is that this is feasible.

Setup



I created a very simple and standard gradle project in IntelliJ and I'm still encountering issues:

The problem



I'm missing the
Java8Lexer
and
Java8Parser
classes. I have no idea where to find these.

build.gradle


group 'com.antlr-demo'
version '1.0-SNAPSHOT'

apply plugin: 'java'

sourceCompatibility = 1.8

repositories {
mavenCentral()
}

dependencies {
compile 'org.antlr:antlr4-master:4.5'
testCompile group: 'junit', name: 'junit', version: '4.11'
}


Test.java


Even in this very trivialized example, none of the two classes I need are not being imported.

public static void parseFile(String f) { // found in Test.java:257
try {
if ( !quiet ) System.err.println(f);
// Create a scanner that reads from the input stream passed to us
Lexer lexer = new Java8Lexer(new ANTLRFileStream(f)); // missing

CommonTokenStream tokens = new CommonTokenStream(lexer);

// Create a parser that reads from the scanner
Java8Parser parser = new Java8Parser(tokens); // missing
if ( diag ) parser.addErrorListener(new DiagnosticErrorListener());
if ( bail ) parser.setErrorHandler(new BailErrorStrategy());
if ( SLL ) parser.getInterpreter().setPredictionMode(PredictionMode.SLL);

// start parsing at the compilationUnit rule
ParserRuleContext t = parser.compilationUnit();
if ( notree ) parser.setBuildParseTree(false);
if ( gui ) t.inspect(parser);
if ( printTree ) System.out.println(t.toStringTree(parser));
}
catch (Exception e) {
System.err.println("parser exception: "+e);
e.printStackTrace(); // so we can get stack trace
}
}


It's not described in the pom file...

Answer

You are missing one step. You got the java8 grammar but you haven't yet created a a parser from it. This usually involves running the antlr4 jar on the grammar file (FAQs and more), which is very simple (example taken from the Getting Started page):

$ antlr4 Hello.g4
$ javac Hello*.java