Any hints on how to use the TCP API?

[RudolfVonKrugstein]

Hi,

The TCP API seems to be not documented (yet). Any hints, or simple example on how to use it? I would like to use onnxruntimer-server for serving silero VAD and using it via TCP api.

Thanks you!

[kibae]

Hi, @RudolfVonKrugstein

You're right, there's no documentation for the TCP API yet. I created it to work with the pg_onnx project, I didn't think anyone would use it.
I'll try to explain it briefly and write some documentation after my vacation.

onnxruntime-server/src/transport/tcp/tcp_server.hpp

Lines 21 to 26 in c2b2141

	struct protocol_header {
	int16_t type;
	int64_t length;
	int64_t json_length;
	int64_t post_length;
	} __attribute__((packed));

The structure of the protocol headers. The header contains the kind of operation you are requesting, the length of the entire request (JSON+POST), the length of the JSON data, and the length of the POST data (non-JSON data).
The types of operations are defined here.

onnxruntime-server/src/onnxruntime_server.hpp

Lines 181 to 187 in c2b2141

	enum type : int16_t {
	CREATE_SESSION = 1,
	EXECUTE_SESSION = 5,
	DESTROY_SESSION = 9,
	LIST_SESSION = 21,
	GET_SESSION = 22,
	};

After the header, you can send JSON data and after that POST data. Attach some of the E2E test code.

onnxruntime-server/src/test/e2e/e2e_test_tcp_server.cpp

Lines 16 to 27 in c2b2141

	auto json_data = json.dump();
	struct onnxruntime_server::transport::tcp::protocol_header header = {};
	header.type = htons(type);
	header.length = HTONLL(json_data.size() + post_size);
	header.json_length = HTONLL(json_data.size());
	header.post_length = HTONLL(post_size);
	std::vector<boost::asio::const_buffer> buffers;
	buffers.emplace_back(&header, sizeof(header));
	buffers.emplace_back(json_data.c_str(), json_data.size());
	if (post != nullptr)
	buffers.emplace_back(post, post_size);

Have a great holiday season! 🥳

[RudolfVonKrugstein]

Thank you! I will try it after the holidays. And I will leave the issue open for you to decide if you want to close it now or when there is documentation.