Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for dynamic message types #640

Merged
merged 15 commits into from
Nov 27, 2023
Merged

Conversation

emcfarlane
Copy link
Contributor

Adds support for dynamic clients and servers with options to initialize messages on receives. See #523 for more details. Two new options WithRequestInitializer and WithResponseInitializer provide initializer functions that can construct messages before unmarshalling. Proto based schemas can use the new Schema field on Spec to introspect the method types. Other IDLs can set their own schema using the WithSchema option.

Dynamic Message Initializer

As an example to support dynamicpb.Message we will inspect the schema and set the message descriptor. An intializer func can be run on the client or handller. This func is provided to the examples below as an option:

// dynamicMessageInitialzier inits a dynamicpb.Message descriptor to the correct
// input or output type for the method.
func dynamicMessageInitializer(spec Spec, msg any) error {
	dynamic, ok := msg.(*dynamicpb.Message)
	if !ok {
		return nil
	}
	desc, ok := spec.Schema.(protoreflect.MethodDescriptor)
	if !ok {
		return fmt.Errorf("unexpected schema type %T for %T message", spec.Schema, dynamic)
	}
	// Client initializer is run on response, Server on request.
	if spec.IsClient {
		*dynamic = *dynamicpb.NewMessage(desc.Output())
	} else {
		*dynamic = *dynamicpb.NewMessage(desc.Input())
	}
	return nil
}

Dynamic Clients

Construct a client using dynamicpb.Message. Ensure the URL includes the suffix of the handler method to invoke and to include WithSchema and WithResponseInitializer options:

desc, _ = protoregistry.GlobalFiles.FindDescriptorByName("connect.ping.v1.PingService.Ping")
methodDesc := desc.(protoreflect.MethodDescriptor)
// Create a client with [dynamicpb.Message] as the request and response message types.
client := connect.NewClient[dynamicpb.Message, dynamicpb.Message](
	http.DefaultClient,
	serverURL + "/connect.ping.v1.PingService/Ping",
	connect.WithSchema(methodDesc),
	connect.WithResponseInitializer(dynamicMessageInitializer),
	connect.WithIdempotency(connect.IdempotencyNoSideEffects),
)
// Build the message using the input descriptor.
msg := dynamicpb.NewMessage(methodDesc.Input())
msg.Set(
	methodDesc.Input().Fields().ByName("number"),
	protoreflect.ValueOfInt64(42),
)
res, _ := client.CallUnary(context.Background(), connect.NewRequest(msg))
// Get values from the message using the output descriptor.
got := res.Msg.Get(
	methodDesc.Output().Fields().ByName("number")
).Int()

Dynamic Handlers

Create the server func with a signature that includes the dynamicpb.Message. Then construct the handler specifying the procedure URL and ensure to include WithSchema and WithRequestInitializer options:

// Create a handler func with [dynamicpb.Message] as the request and response message types.
dynamicPing := func(_ context.Context, req *connect.Request[dynamicpb.Message]) (*connect.Response[dynamicpb.Message], error) {
	got := req.Msg.Get(methodDesc.Input().Fields().ByName("number")).Int()
	msg := dynamicpb.NewMessage(methodDesc.Output())
	msg.Set(
		methodDesc.Output().Fields().ByName("number"),
		protoreflect.ValueOfInt64(got),
	)
	return connect.NewResponse(msg), nil
}
mux := http.NewServeMux()
mux.Handle("/connect.ping.v1.PingService/Ping",
	connect.NewUnaryHandler(
		"/connect.ping.v1.PingService/Ping",
		dynamicPing,
		connect.WithSchema(methodDesc),
		connect.WithRequestInitializer(dynamicMessageInitializer),
		connect.WithIdempotency(connect.IdempotencyNoSideEffects),
	),
)

Fixes #523

New field Schema of type any on Spec objects. For proto based schemas
the type will be of protoreflect.MethodDescriptor. This allows for easy
introspection to interceptors.
An Initializer helps to construct dynamic messages on Receive. This lets
clients and servers use dynamic messages. A default initializer
for dynamicpb.Message is provided. Other IDLs can provide custom
Initializers using the WithInitializer option.
Copy link
Member

@jhump jhump left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of small suggestions. Otherwise LGTM

client_stream.go Outdated
Comment on lines 120 to 125
if s.initializer != nil {
if err := s.initializer(s.conn.Spec(), s.msg); err != nil {
s.receiveErr = err
return false
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This nil checks is made in 8 separate places. What if instead you added an unexported initialize function to config and then pass config.initialize instead of config.Initializer? That way, that one method could do the nil check (and just return nil error if the func is nil).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of having the function branch off the two configs I've added a maybeInitializer that wraps the func and does the nil check. Wdyt?

option.go Outdated Show resolved Hide resolved
@jhump jhump merged commit 739280b into connectrpc:main Nov 27, 2023
8 checks passed
Copy link
Member

@akshayjshah akshayjshah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exported APIs LGTM.

@emcfarlane emcfarlane deleted the ed/dynamic branch November 27, 2023 19:50
@jhump jhump added bug Something isn't working enhancement New feature or request and removed bug Something isn't working labels Dec 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for dynamic or reflected types for client
3 participants