Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Compile-time annotation-processor tool to clientcore #43457

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

samvaity
Copy link
Member

@samvaity samvaity commented Dec 18, 2024

Description

Creates a new /sdk/clientcore/tools package named annotation-processor, which serves as a shippable replacement for the in-house RestProxy annotation processing utilities used to generate HTTP client service implementations.

This package addresses common challenges with service client implementations relying on the intermediate RestProxy layer, which introduces reflection-based inefficiencies, negatively impacts performance, and produces less readable stack traces. Instead, annotation-processor uses JavaPoet to process annotations and generate service client implementations. Currently, it generates the respective service client implementation in ${project.build.directory}/generated-sources/.

Remaining Work:

  • Enhance response body handling to manage complex cases.
  • Address minor issues in public API generation.
  • Generate comprehensive Javadocs for generated code.

This implementation currently supports end-to-end functionality for the OpenAI getChatCompletions API as a proof of concept.

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

eng/versioning/external_dependencies.txt Outdated Show resolved Hide resolved
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>io.clientcore</groupId>
<artifactId>clientcore-tools-service</artifactId>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clientcore-tools-parent or something might be more appropriate - but more generally, do we need to have a parent for tools, or can we just put the codegen a level higher?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was just thinking if we add any more tooling specifically for clientcore this would be a good place to club it. I will rename it to clientcore-tools-parent for now (easy to take it away if we dont need it later).

sdk/clientcore/tools/sdk-codegen-tool/README.md Outdated Show resolved Hide resolved
sdk/clientcore/tools/sdk-codegen-tool/README.md Outdated Show resolved Hide resolved
sdk/clientcore/tools/sdk-codegen-tool/README.md Outdated Show resolved Hide resolved
sdk/clientcore/tools/sdk-codegen-tool/README.md Outdated Show resolved Hide resolved
@samvaity samvaity force-pushed the compile-time-codegen branch from cadcf9b to 149a750 Compare December 19, 2024 22:45
@samvaity samvaity changed the title Add compile time sdk-codegen-tool Add Compile-time annotation-processor tool to clientcore Dec 19, 2024
@@ -0,0 +1,135 @@
# Client Core Compile-Time Annotation Processor

The client-core annotation processor for introducing compile-time code generation for libraries based on client core
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be a note here, at least temporarily, that "this project is for experimentation and exploring new ideas that may or may not make it into a supported GA release."

Comment on lines +107 to +120
templateInput.addImport("io.clientcore.core.util.Context");
templateInput.addImport("io.clientcore.core.util.binarydata.BinaryData");
templateInput.addImport("io.clientcore.core.http.models.HttpHeaders");
templateInput.addImport("io.clientcore.core.http.pipeline.HttpPipeline");
templateInput.addImport("io.clientcore.core.http.models.HttpHeaderName");
templateInput.addImport("io.clientcore.core.http.models.HttpMethod");
templateInput.addImport("io.clientcore.core.http.models.HttpResponse");
templateInput.addImport("io.clientcore.core.http.models.HttpRequest");
templateInput.addImport("io.clientcore.core.http.models.Response");
templateInput.addImport("java.util.Map");
templateInput.addImport("java.util.HashMap");
templateInput.addImport("java.util.Arrays");
templateInput.addImport("java.lang.Void");
templateInput.addImport("java.util.List");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given we have a dependency on io.clientcore:core could we change all these to being something like Content.class.getName() as core is still in development and if these move it'll be obvious as compilation here will break.

Comment on lines +86 to +99
templateInput.setHttpRequestContexts(serviceInterface.getEnclosedElements().stream()
.filter(element -> element.getKind() == ElementKind.METHOD)
.filter(element -> element.getAnnotation(HttpRequestInformation.class) != null)
.map(ExecutableElement.class::cast)
.map(e -> createHttpRequestContext(e, templateInput))
.collect(Collectors.toList()));

// template input set UnexpectedResponseExceptionDetails
templateInput.setUnexpectedResponseExceptionDetails(serviceInterface.getEnclosedElements().stream()
.filter(element -> element.getKind() == ElementKind.METHOD)
.filter(element -> element.getAnnotation(HttpRequestInformation.class) != null)
.map(ExecutableElement.class::cast)
.map(e -> e.getAnnotation(UnexpectedResponseExceptionDetail.class))
.collect(Collectors.toList()));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the streams here have the same initial filters and mapping to ExecutableElement could we create an initial list and work from there?

final String serviceInterfaceImplFQN = serviceInterfaceFQN + "Impl";
final String serviceInterfaceImplShortName = serviceInterfaceImplFQN.substring(lastDot + 1);

templateInput.setPackageName(packageName);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if packageName is null at this point?

Comment on lines +67 to +69
final String serviceInterfaceShortName = serviceInterfaceFQN.substring(lastDot + 1);
final String serviceInterfaceImplFQN = serviceInterfaceFQN + "Impl";
final String serviceInterfaceImplShortName = serviceInterfaceImplFQN.substring(lastDot + 1);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
final String serviceInterfaceShortName = serviceInterfaceFQN.substring(lastDot + 1);
final String serviceInterfaceImplFQN = serviceInterfaceFQN + "Impl";
final String serviceInterfaceImplShortName = serviceInterfaceImplFQN.substring(lastDot + 1);
final String serviceInterfaceShortName = serviceInterfaceFQN.substring(lastDot + 1);
final String serviceInterfaceImplShortName = serviceInterfaceShortName + "Impl";

No need for serviceInterfaceImplFQN as it just gets thrown away and we're repeating the logic to substring.

method.addHeader(headerParam.value(), param.getSimpleName().toString());
} else if (queryParam != null) {
// we do not support query param substitutions, so we just add the query param name and value
method.addQueryParam(queryParam.value(), param.getSimpleName().toString());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we missing support for encoded and multipleQueryParams?


final boolean hasQueryParams = !method.getQueryParams().isEmpty();

Pattern pattern = Pattern.compile("\\{(.+?)\\}");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Make this a constant instead of creating a new Pattern each time as it never changes.

Comment on lines +35 to +36
String paramName = matcher.group(1);
Substitution substitution = method.getSubstitution(paramName);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Real niche corner case if I'm understanding this code correctly. If this is handling host and path substitutions at the same time it's possible that the host and path have substitutions with the same name:

@ServiceInterface(name = "HostSubstitutionMethods", host = "https://{sub1}.host.com")
interface WeirdDesign {
  @HttpRequestInformation(method = HttpMethod.GET, path = "{sub1}")
  void substitution(@HostParam("sub1") String hostSub1, @PathParam("sub1") String pathSub1);
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me add this to the test case and validate.


if (hasQueryParams) {
buffer.append("?");
for (Map.Entry<String, String> entry : method.getQueryParams().entrySet()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will need to handle encoded and multi for path params. May want to create utility methods in core which this can use to handle that. That way any time there's an update to any handling in core we don't need to update here and core and core can just self-encompass the update.

buffer.append(entry.getKey()).append("=\" + ").append(entry.getValue()).append(" + \"&");
}
// Remove the last "&\""
buffer.delete(buffer.length() - 5, buffer.length());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're just removing &\" why are we deleting 5 characters?


// strip out unnecessary `+ ""` in the buffer
String result = buffer.toString();
result = result.replaceAll(" \\+ \"\"", "");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we just handle this while generating the buffer?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants