CG Compilation Framework

Compilation Framework

Once the code is generated, it must be compiled. Compilation is a bit of an involved process. Basic flow:

• The operator invokes FragmentContext.getImplementationClass() (which really gets the implementation object.)
• The FragmentContext invokes the global CodeCompiler.getImplementationClass() method.
• CodeCompiler looks up the code in the code cache.
• If not found, CodeCompiler creates a QueryClassLoader to hold the generated class.
• Next CodeCompiler calls ClassTransformer.getImplementationClass( ) to get the generated class (the actual class this time.)
• ClassTransformer calls QueryClassLoader.getClassByteCode( ) to compile the code.
• ClassTransformer then merges the generated code with the template to create the actual class byte codes.
• This code is placed into the cache.
• The CodeCompiler uses the QueryClassLoader to create an instance of the class.

The actual compilation is controlled by the ClassCompilerSelector which selects either the Janio or JDK compilers.

JDK Compiler

The "JDK compiler" uses the built-in Java compiler (which is, in modern Java, available in the JRE itself.) Java compilation in Java itself is based on the JavaCompiler class. The javax.tools provides a dynamic compilation framework. The IBM article provides a sample app that compiles Java classes from in-memory sources.

Drill's implementation is similar to the approach outlined in the IBM article. The DrillJavaFileManager manages the (virtual) source and class files, DrillJavaFileObject manages each file.

Debugging Hints

You can view the generated code in one of three ways:

• Enable debug logging for AbstractClassCompiler. The generated code is written to the log file. Set the config option drill.exec.compile.debug to true if you want line numbers added to each line.
• Uncomment the obvious block of code in AbstractClassCompiler to write each generated class to /tmp. This version is handy as it uses the class name to name each file.
• Pass the following to the command line when starting Drill: -Dorg.codehaus.janino.source_debugging.enable=true -Dorg.codehaus.janino.source_debugging.dir=/tmp. (See this post for more information.) This version writes all files using generic temporary names, making it hard to find the particular file of interest.

You can gain access to the generated class files by uncommenting the obvious lines near line 256 in MergeAdapter.