feat: general service discovery for farm testnets

[NikolaMilosa]

This service will be used for all sorts of farm testnets which will monitor arbitrary services. The idea is to use this to register testnet machines and then use our vector stack to scrape logs

[github-actions]

MSD Diff

No diff

[Copilot]

PR Overview

This PR introduces a general service discovery service for farm testnets that registers target machines and integrates with the vector stack for log scraping. Key changes include the implementation of a server module with endpoints for adding and retrieving targets, new storage implementations (in-memory and file), and updates to metrics, supervisor, and configuration builder modules to support the service discovery workflow.

Reviewed Changes

File	Description
src/server/mod.rs	Implements the server with axum routes and a graceful shutdown.
src/storage/in_memory.rs and src/storage/file.rs	Introduces in-memory and file storage mechanisms for target persistence.
src/metrics.rs	Adds metrics instrumentation for monitoring target statuses.
src/supervisor.rs	Implements target supervision with target polling, error logging, and GC handling.
src/server/add_targets.rs & get_targets.rs	Define HTTP endpoints for target management.
Cargo.toml	Updates dependencies and workspace members.
multiservice-discovery-shared/builders/general_exec.rs	Adds a configuration builder for general service discovery.
multiservice-discovery-downloader	Updates configuration generation logic and target conversion functions.
multiservice-discovery-shared/builders/exec_log_config_structure.rs	Exports builder types for command execution configurations.

Copilot reviewed 24 out of 24 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

rs/ic-observability/multiservice-discovery-downloader/src/downloader_loop.rs:107

• Using unwrap on cli.output_dir.parent() may panic if the output directory has no parent; add a proper check or use a fallback value.

fs_err::create_dir_all(cli.output_dir.parent().unwrap()).unwrap();

rs/ic-observability/multiservice-discovery-downloader/src/downloader_loop.rs:190

• Deserialization with unwrap in convert_and_filter_general_targets can lead to a runtime panic if the input does not match the expected schema; consider error handling to gracefully skip or report faulty entries.

values.into_iter().map(|value| serde_json::from_value(value).unwrap())

[Copilot]

Using unwrap on duration_since can cause a panic if the system clock is adjusted or if an error occurs; consider handling the Err case explicitly.

Suggested change

	if SystemTime::now().duration_since(self.last_successful_sync).unwrap() > self.gc_timeout {
	self.info("GC elapsed, removing target...");
	(self.remove_self)().await;
	self.metrics.remove_observed_value(&self.target.name);
	break;
	match SystemTime::now().duration_since(self.last_successful_sync) {
	Ok(duration) => {
	if duration > self.gc_timeout {
	self.info("GC elapsed, removing target...");
	(self.remove_self)().await;
	self.metrics.remove_observed_value(&self.target.name);
	break;
	}
	}
	Err(e) => {
	self.warn(format!("Failed to calculate duration since last successful sync: {:?}", e));
	}

[NikolaMilosa]

Doesn't make any sense to make the code uglier since the time is never set to anything but SystemTime::now. If this happens ever I would want the app to completely crash since we have a bug and not to just print the error and try and recover.

This should never happen.

[DFINITYManu]

If the system time is a problem, can you change it to use monotonic time? std::time::Instant. They are guaranteed to be increasing all the time. Plsfix.

[sasa-tomic]

Should the in-memory storage be the default? What happens with the list of testnets if the service is updated/restarted?

[NikolaMilosa]

Where ever we deploy it it will be deployed with a file. This is the default just so that locally when testing something you can quickly do a cargo run and be off to testing and not have to provide the files all the time

[DFINITYManu]

I love sane dev defaults.

[sasa-tomic]

LGTM, thanks!

[DFINITYManu]

avoid system clock, use std time instant. These can be compared and have no risk of falling backwards.