[move-vm][closures] Type and Value representation and serialization

[wrwg]

Description

[PR 4/n vm closures]

This implements types and values for functions and closures, as well as serialization. For a detailed discussion, see the design doc. It does not yet implement the interpreter and runtime verification.

Overview:

• MoveValue and MoveTypeLayout are extended to represent closures and functions. A closure carries the function name, the closure mask, the layout of the captured arguments, and their values. The layout enables serialization of the captured arguments without any context.
• Runtime Type is extended by functions.
• ValueImpl is extended by a closure variant. This representation uses trait AbstractFunction to represent the function value and implement core functionality like comparison and printing. The existing trait FunctionValueExtension is used to create AbstractFunction from the serialized representation.
• Similar as with MoveValue, the serialized representation of ValueImpl::Closure contains the captured parameter layouts, enabling to deserialize the captured arguments context independently. This design has been chosen for robustness, avoiding a dependency of serialization from loaded functions, and to enable fully 'lazy' deserialization of closures. The later allows a contract to store hundreds of function values and on loading them from storage, avoiding loading the associated code until the moment the closure is actually executed.
• AbstractFunction is implemented in the loader such that it can be either in 'unresolved' or in `resolved' state.

How Has This Been Tested?

Testing postponed until e2e wiring is up

Type of Change

• New feature
• Bug fix
• Breaking change
• Performance improvement
• Refactoring
• Dependency update
• Documentation update
• Tests

Which Components or Systems Does This Change Impact?

• Validator Node
• Full Node (API, Indexer, etc.)
• Move/Aptos Virtual Machine
• Aptos Framework
• Aptos CLI/SDK
• Developer Infrastructure
• Move Compiler
• Other (specify)

[trunk-io]

⏱️ 1h 6m total CI duration on this PR

Job	Cumulative Duration	Recent Runs
rust-move-tests	13m	🟩
rust-move-tests	13m	🟩
rust-cargo-deny	9m	🟩 ⬜ 🟩 🟩 🟩 (+1 more)
rust-move-tests	8m	🟥
rust-move-tests	8m	🟥
check-dynamic-deps	6m	🟩 🟩 🟩 🟩 🟩 (+1 more)
general-lints	3m	🟩 🟩 🟩 🟩 🟩 (+1 more)
rust-move-tests	2m	🟥
semgrep/ci	2m	🟩 🟩 🟩 🟩 🟩 (+1 more)
file_change_determinator	1m	🟩 🟩 🟩 🟩 🟩 (+1 more)
rust-move-tests	29s	⬜
permission-check	17s	🟩 🟩 🟩 🟩 🟩 (+1 more)
permission-check	12s	🟩 🟩 🟩 🟩 🟩 (+1 more)
check-branch-prefix	1s	🟩

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

[wrwg]

• [move-vm][closures] Interpreter #15680
• [move-vm][closures] Type and Value representation and serialization #15670 👈 (View in Graphite)
• [move-cm][closures] Refactor: Move type conversions out of Loader into a trait #15669
• [move-vm][closures] Refactor: move abilities and closure mask into core types #15668
• [move-vm][closures] Types and opcodes for closures #15667
• [move-vm] Increase bytecode VERSION_MAX to v8 #15777
• [move-prover] Sharding feature + re-animate Prover.toml #15775
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[georgemitenkov]

First almost full pass, need to look more, but there are already plenty of comments

[georgemitenkov]

What if we have a resolution_error: Some(..) stored here? Do we just ignore it?

[georgemitenkov]

You should call

Type::verify_ty_arg_abilities(function.ty_param_abilities(), &ty_args)
            .map_err(|e| e.finish(Location::Module(module_id.clone())))?;

here?

[georgemitenkov]

Also, Loader has load_function(...) -> VMResult<LoadedFunction>, can we reuse it? Just move this as default impl to ModuleStorage (I did not d it because I hoped to remove V1 faster, but it can go to module storage easily).

[georgemitenkov]

Add more info to debug?

[georgemitenkov]

I do not think this even works:

impl TryInto<StructTag> for &MoveStructLayout {
    type Error = anyhow::Error;

    fn try_into(self) -> Result<StructTag, Self::Error> {
        use MoveStructLayout::*;
        match self {
            Runtime(..) | RuntimeVariants(..) | WithFields(..) | WithVariants(..) => bail!(
                "Invalid MoveTypeLayout -> StructTag conversion--needed MoveLayoutType::WithTypes"
            ),
            WithTypes { type_, .. } => Ok(type_.clone()),
        }
    }
}

because here we have runtime layouts?

[georgemitenkov]

If you have a generic type parameter in the function, this fails:

// from ty_to_ty_tag implementation

// References and type parameters cannot be converted to tags.
            Type::Reference(_) | Type::MutableReference(_) | Type::TyParam(_) => {
                return Err(
                    PartialVMError::new(StatusCode::UNKNOWN_INVARIANT_VIOLATION_ERROR)
                        .with_message(format!("No type tag for {:?}", ty)),
                );
            },

[runtian-zhou]

Unneeded change

[runtian-zhou]

This should more like lazy loaded closure?

[runtian-zhou]

Do you need to double the length here as you would be serializing tuples

[runtian-zhou]

Was wondering why the interior mutability is needed here for error. If the resolution failed this function would fail and you don't need to store the error msg here?s