Skip to content

Feat: add mithril-client CLI command for UTxO-HD ledger state snapshot conversion #2518

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
146 changes: 146 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions mithril-client-cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,15 @@ chrono = { workspace = true }
clap = { workspace = true }
cli-table = "0.5.0"
config = { workspace = true }
flate2 = "1.1.1"
fs2 = "0.4.3"
futures = "0.3.31"
human_bytes = { version = "0.4.3", features = ["fast"] }
indicatif = { version = "0.17.11", features = ["tokio"] }
mithril-cli-helper = { path = "../internal/mithril-cli-helper" }
mithril-client = { path = "../mithril-client", features = ["fs", "unstable"] }
mithril-doc = { path = "../internal/mithril-doc" }
reqwest = { workspace = true }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workspace only specify the minimal set of shared features for reqwest since we don't want to force them on downstream library users.
For our binaries we needs a bit more:

Suggested change
reqwest = { workspace = true }
reqwest = { workspace = true, features = [
"default",
"gzip",
"zstd",
"deflate",
"brotli"
] }

serde = { workspace = true }
serde_json = { workspace = true }
slog = { workspace = true, features = [
Expand All @@ -47,8 +49,11 @@ slog = { workspace = true, features = [
slog-async = { workspace = true }
slog-bunyan = { workspace = true }
slog-term = { workspace = true }
tar = "0.4.44"
thiserror = { workspace = true }
tokio = { workspace = true, features = ["macros", "rt-multi-thread"] }
zip = "4.0.0"

[dev-dependencies]
mithril-common = { path = "../mithril-common", features = ["test_tools"] }
mockall = { workspace = true }
1 change: 1 addition & 0 deletions mithril-client-cli/src/commands/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ pub mod cardano_stake_distribution;
pub mod cardano_transaction;
mod deprecation;
pub mod mithril_stake_distribution;
pub mod tools;

pub use deprecation::{DeprecatedCommand, Deprecation};

Expand Down
16 changes: 16 additions & 0 deletions mithril-client-cli/src/commands/tools/archive_unpacker/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
mod tar_gz_unpacker;
mod zip_unpacker;

pub use tar_gz_unpacker::*;
pub use zip_unpacker::*;

use mithril_client::MithrilResult;
use std::path::Path;

#[cfg_attr(test, mockall::automock)]
pub trait ArchiveUnpacker {
fn unpack(&self, archive_path: &Path, unpack_dir: &Path) -> MithrilResult<()>;

#[cfg(test)]
fn as_any(&self) -> &dyn std::any::Any;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
use std::{fs::File, path::Path};

use anyhow::Context;
use flate2::read::GzDecoder;
use tar::Archive;

use mithril_client::MithrilResult;

use super::ArchiveUnpacker;

#[derive(Debug, Eq, PartialEq)]
pub struct TarGzUnpacker;

impl ArchiveUnpacker for TarGzUnpacker {
fn unpack(&self, archive_path: &Path, unpack_dir: &Path) -> MithrilResult<()> {
let archive = File::open(archive_path)
.with_context(|| format!("Could not open archive file '{}'", archive_path.display()))?;
let gzip_decoder = GzDecoder::new(archive);
let mut file_archive = Archive::new(gzip_decoder);
file_archive.unpack(unpack_dir).with_context(|| {
format!(
"Could not unpack '{}' with 'Gzip' to directory '{}'",
archive_path.display(),
unpack_dir.display()
)
})?;

Ok(())
}

#[cfg(test)]
fn as_any(&self) -> &dyn std::any::Any {
self
}
}

#[cfg(test)]
mod tests {
use std::fs::{self, File};

use flate2::{write::GzEncoder, Compression};
use tar::{Builder, Header};

use mithril_common::temp_dir_create;

use super::*;

#[test]
fn unpack_tar_archive_extracts_all_files() {
let temp_dir = temp_dir_create!();
let archive_path = temp_dir.join("archive.tar.gz");

{
let tar_gz_file = File::create(&archive_path).unwrap();
let encoder = GzEncoder::new(tar_gz_file, Compression::default());
let mut tar_builder = Builder::new(encoder);

let content = b"root content";
let mut header = Header::new_gnu();
header.set_size(content.len() as u64);
header.set_cksum();
tar_builder
.append_data(&mut header, "root.txt", &content[..])
.unwrap();

let content = b"nested content";
let mut header = Header::new_gnu();
header.set_size(content.len() as u64);
header.set_cksum();
tar_builder
.append_data(&mut header, "nested/dir/nested-file.txt", &content[..])
.unwrap();

tar_builder.finish().unwrap();
}

TarGzUnpacker.unpack(&archive_path, &temp_dir).unwrap();

let root_file = temp_dir.join("root.txt");
assert!(root_file.exists());
let root_file_content = fs::read_to_string(&root_file).unwrap();
assert_eq!(root_file_content, "root content");

let nested_file = temp_dir.join("nested/dir/nested-file.txt");
assert!(nested_file.exists());
let nested_file_content = fs::read_to_string(&nested_file).unwrap();
assert_eq!(nested_file_content, "nested content");
Comment on lines +79 to +87
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: You could use assert_dir_eq for the existence check.

}
}
Loading