0.6.4

Swift 5.4-development updates

May have some issues on linux + 5.4 still.

0.6.3

• Swift 5.4 compatibility

0.6.2

Bugfix

• =serialization rm 5.2.4 workaround, fix LWWMap with Codable Value (Codable in Proto nesting) #782
  • this would manifest when using deeply nested CRDTs such as LWWMap which stores an LWWRegister internally, both being encoded using protocol buffers, and using a value which is Codable/JSON encoded -- it would have wrong assumptions about how it should store the serialized bytes.

Stats

A total of 1 tickets (including 1 bug 🐞) were resolved in this patch release.
You can view the full changes made on the 0.6.2 milestone.

0.6.1 - Cluster/SWIM metrics

Swift Metrics support

SWIM

• Updated to 0.3.0 of swift-cluster-membership which includes cluster metrics (directly reported by the swim algorithm instance) https://github.com/apple/swift-cluster-membership/releases/tag/0.3.0
• Implemented metrics for round trip time of ping and pingRequest messages #780

Available metrics are:

// actor lifecycle counters
first.actors.lifecycle["root:/user", "event:start"] // counter
first.actors.lifecycle["root:/user", "event:stop"] // counter
first.actors.lifecycle["root:/system", "event:start"] // counter
first.actors.lifecycle["root:/system", "event:stop"] // counter

// swim metrics
first.cluster.swim.message.count["direction:out"] // counter

first.cluster.swim.probe.ping["type:successful"] // counter
first.cluster.swim.probe.ping["type:failed"] // counter
first.cluster.swim.probe.pingRequest["type:successful"] // counter
first.cluster.swim.probe.pingRequest["type:failed"] // counter

first.cluster.swim.lha[] // gauge

first.cluster.swim.roundTripTime.pingRequest[] // timer
first.cluster.swim.roundTripTime.pingRequest["type:all"] // timer
first.cluster.swim.roundTripTime.pingRequest["type:firstAck"] // timer

// cluster/swim membership

first.cluster.swim.members["status:alive"] // gauge
first.cluster.swim.members["status:suspect"] // gauge
first.cluster.swim.members["status:unreachable"] // gauge
first.cluster.swim.removedMemberTombstones[] // counter

Opt-in per actor (group) metrics

It is now possible to enable metrics per actor and report them as a groups etc.

For example, one may want to monitor a specific actor and how long deserialization of the messages it receives takes, as well as monitor it's mailbox count while it is active. This can be now done via:

        let one: ActorRef<String> = try first.spawn(
            "measureMe",
            props: .metrics(group: "measureMe", measure: [.mailbox, .deserialization]),
            someBehaviorOrActorable
        )

resulting in metrics like:

actorSystemName.measureMe.mailbox.count // gauge
actorSystemName.measureMe.deserialization.size // gauge (bytes)
actorSystemName.measureMe.deserialization.time // timer (nanoseconds)

Refer to:

• https://github.com/apple/swift-distributed-actors/blob/main/Sources/DistributedActors/Metrics/Metrics+Actor.swift
• https://github.com/apple/swift-cluster-membership/blob/main/Sources/SWIM/Metrics.swift

for detailed documentation on the metrics.

Stats

A total of 4 tickets (including 1 bug 🐞) were resolved in this patch release.
You can view the full changes made on the 0.6.1 milestone.

commits  added  removed
      2   1079      253 Konrad `ktoso` Malawski
      1      0        4 Tomer Doron

Thanks as usual for all the feedback and help to our users and contributors!

0.6.0 - OSS SWIM

Use OSS Swift Cluster Membership's SWIM

The SWIM+Lifeguard cluster membership was extracted from Distributed Actors and released as a stand-alone module as part of the Swift Cluster Membership.

See the blog announcement here: https://swift.org/blog/swift-cluster-membership/

Other Notable Changes

New API

• Implement Actorable version of ActorTestKit.ensureRegistered() ga:test-kit t:actorable t:testkit #742

General

• Patterns around storing Actor relates-to-swift-feature t:actor t:docs user-feedback #618

• ActorSystem.shutdown() does not invoke callback if system is already shutdown bug 🐞 #760, thank you @budde!

• +test actorable call during shutdown should not crash ga:actorable-feature-parity t:actorable #730

• =test ensureNodes is too strict; a down may already become a removal ga:cluster t:tests#732

• exact nio ssl version for compat with specific project which requires it #738

• Actorable and generic actorable types source-gen breaks down 3 - in progress ga:actorable-feature-parity relates-to-swift-feature t:actorable #658

• (Make it) Crash process if failure escalates to a Guardian (esp. /system) t:actor t:isolation #31

• =build bump to using stable 5.3 swift-syntax #777

• SWIM: Do not ensureAssociated, just get associations for unknown nodes 1 - triaged ga:swim t:cluster:swim #428

Test fixes and hardening

• Ensure we close all associations when we terminate / receive RIP ga:cluster t:cluster:all #600

• [radar 68467435] Swift 5.3-dev compilation broken on Linux bug 🐞 failed 💥 relates-to-swift-feature

• Foundation.JSONEncoder issue in Swift 5.2.4 Linux bug 🐞 t:serialization #729

• !serialization assume _getMangledTypeName is not available #776

• =test harden test_gossip_shouldEventuallyStopSpreading failed 💥 t:crdt #771

• FAILED: SWIMShellClusteredTests.test_swim_shouldIncreasePingTimeout_whenHighMultiplier failed 💥 ga:swim t:cluster:swim #727

• flaky test: 'ActorableExampleRouterTests.test_refRouter_shouldWork' failed 💥 #740

• SKIP one integration test because #758 compilation breaks until swift… #773

• FAILED: test_interceptor_shouldInterceptSignals failed 💥 t:actor #614

• FAILED: ActorableExampleRouterTests test_actorRouter_shouldWork failed 💥 #736

• FAILED: DistributedActorsTests.ShootTheOtherNodeClusteredTests test_shootOtherNodeShouldTerminateIt failed 💥 t:cluster:all #390

• !!tmp disable test_actorOwned_ORSet_add_many_times #747

• =ios fix availability requirements for iOS #731

Stats

A total of 27 issues was closed since the 0.5.1 release, including 4 bugs 🐞.

You can view all resolved tickets on the 0.6.0 milestone.

commits  added  removed
     27   4890     5354 Konrad `ktoso` Malawski
      2     90        0 Adam Budde
      1    130       47 Yim Lee

0.5.1

Bug Fixes

CRDT

• Fixed a bug with unfortunate interplay of CRDT.ORMap serialization and Foundation's JSONEncoder as implemented in 5.2.4 on Linux (only) #720
  • a proper fix to Foundations encoders is underway to bring them in-sync with behavior on macOS as well as Swift 5.3: swiftlang/swift-corelibs-foundation#2840
  • that patch would mean the workaround be not necessary anymore.

Stats

A total of 1 issues was closed since the 0.5.1 release, including 1 bug 🐞.

Thank you to Yim Lee for spotting and fixing the issue!

You can view all resolved tickets on the 0.5.1 milestone.

commits  added  removed
      1    130       47 Yim Lee
      1     27        1 Konrad `ktoso` Malawski

0.5.0

New Features

Core / Actors

• Support for Swift 5.3 (and 5.2)
  • 5.1 support was dropped
• Minor: New GCD Dispatcher, meaning that actors can run on specified dispatch queues #703
• Ability to DeathWatch using a Message that the actor can receive directly, rather than having to implement .receiveSignal
  • This is not only a "nice to have" but enables expressing some patterns which otherwise would require storing mappings in dictionaries (and there having trouble with typesafely implementing those contexts)
• Minor: ActorSystem gained .park() call, allowing for simpler "wait until terminated" and building simple 1 file applications. #459
• Minor: new shutdown() overloads to integrate nicer with swift-service-lifecycle #230

Serialization

• Support for plist (xml or binary) message encoding #557 (per-message configurable) based on user feedback

Actorable

• 🎉 Actorables can now be generic types
  • for example, struct Proxy<Message: Codable>: Actorable { ... }
  • Actorable protocols with associated types are not yet supported, see: #656

(Reusable) Gossiper / GossipLogic

• Extracted shared needs of various gossip implementations into Gossiper imeplementation, and re-used it in CRDT gossip as well as re-phrasing Membership gossip in it's terms. This already spotted minor issues which we would have missed in the ad-hoc implementations otherwise. #28
• Gossiper can in the near future become a public type offering "if you need any gossip protocol, just plug in your GossipLogic and it just works" for end-users, including peer discovery, gossip round management etc.

CRDTs

• 📘 Introduced high-level documentation for the feature #638
• 🎉 Implemented (naive) gossip replication; currently remains somewhat naive, however it is a good enough step 1 (our of 3 planned ones). #28
  • Gossiper is re-using new Gossiper type.

Cluster Singleton Actors

• 📘 Added documentation #468 and polished API #396 based on user-feedback

Instrumentation and Instruments(.app) instruments

• Introduced a few initial Instrumentation cut-points in the system #475
  • These will be used for future tracing efforts as well as general measurements and other instrumentation (eagerly awaiting swift-baggage-context). See Sources/DistributedActors/Instrumentation, for the current Instrumentation types.
• Implemented need-driven custom instruments for the following use-cases:
  • ReceptionistInstrumentation,ActorInstrumentation, ActorTransportInstrumentation (incl. Serialization events), and ActorMailboxInstrumentation; See example screenshot.
  • These tools are becoming tremendously helpful in deciding about implementation tradeoffs and inspecting runtime behavior, we expect users to value them as well, in order to avoid treating the actor system as a black box.
• ActorInstruments package definition is fully implemented in type-safe Swifty Instruments DSL, meaning there is no possibility to mis-configure the instrumentation for non-existing fields/tables/schemas:
  • See ActorInstrumentsPackageDefinition.swift, for the complete package definition example
• Introduced SwiftyInstruments DSL, which enabled efficiently building specialized custom instruments for Instruments.app using plain Swift.
  • Definitions cover most of the Instruments package definition schema, some missing bits easy to fill-in in the upcoming future on an as-needed basis. See: SwiftyInstruments, discussed with Instruments team a little bit.
  • Brought up some function-builder limitations which are a bit annoying for this API on the forums:

ActorTestKit

• Introduced fishForMessage which allows hooking into event streams and fishing for messages which satisfy some criteria. #474

Improvements

Core

• ⚠️ Removed ClassBehavior in favor of making Actorable a first-class citizen in all APIs #562
• Logging: Ability to pass in fully constructed Logger for the system to use as base logger: #662 after multiple design discussions and improvements made to swift-log
• 🚀 Avoid un-necessary CoW on DeathWatch container causing performance issues when spawning thousands of child actors. 0cf3dc0

Transport

• 🔥 Hardened and re-worked the Association design as well as connection management
  • This was a long outstanding task which we knew was important to revisit; New implementation guarantees no dropped messages even during racy connection establishments (e.g. nodes A and B joining "eachother" simultaniously -- neither A's or B's sent messages will be dropped when they face such racy handshake and multiple similar races were fixed).
• 🔥 Hardened RemotePersonality for ActorRefs, lessening the use of locks tremendously as well as improving correctness at the same time #382 & #383
• 🔥 Hardened Handshake handling with non-existing or dead nodes, cluster performs exponential backoff when attempting to join such nodes and eventually gives up (rathe than trying to join a dead node forever) #389

Membership Gossip

• 🚀 Vastly improved Membership Gossip implementation including stop conditions and improved ACK handling #699 Resulting in correct and reliable membership gossip.
• 🚀 Drastically improved membership's network utilization (more to be done here in the future), by "smart" encoding schemes used to serialize the membership state, resulting in 4~6x payload size decrease of gossip payloads (measured for 20 node clusters). #676

SWIM Membership / Failure Detector

• Implemented Local Health Aware Probe extension, as defined in the Lifeguard SWIM extensions paper #352
• Implemented Buddy System extension, as defined in the Lifeguard SWIM extensions paper, see: #180 for details

Receptionist

• 🎉 Cluster Receptionist: dropped previous placeholder implementation (full-state sync based) and replaced with operation sync based one. Similar to op-based CRDTs however not using the CRDT infra (long story why not, happy to discuss).
• 🚀 Performance work resulting in API redesign for greater reusability and exploiting CoW semantics correctly for Receptionist.Listing avoiding copying of read-only data to subscribers. Greatly improved scalability of some sample apps, speeding up spawning/registering/looking up 15,000 actors to ~300ms, from previous patological cases of >1minute.

Serialization

• Serialization infrastructure re-design around Codable (docs: #538)
  • This means that Actor messages MUST be Codable, leading to an easier way between "hello world" and "now my hello world is distributed!"
  • Introduced NonTransportableActorMessage as way to "cheat" when non-codable types intended for local only actor messaging need to be sent around.
  • We are aware of some usability issues and limitations with that approach, e.g. the lack of automatically synthesized Codable conformances for enums with associated values, and we intend to fix those limitations in the language itself.
• New serialization infrastructure makes use of Serialization.Manifest and (optionally, when available) _getMangledTypeName ⚠️ Need to discuss future direction with security aspects of this feature and alternative tradeoffs we can take. Had discussions planned but things got rescheduled a bit.

CRDT

• Serialization implementations for LWWRegister, ORMap #509 #598 and Codable conformances for all CRDT types #507

Actorable

• ⚠️ To re-gain the fine-grained control over what functions should be exposed as actor messages we introduced the "comment annotation" of // @actor.
  • Previously magic method names were used for this, resulting in terribly looking code like __notAnActorFunction(hello: Hello)
• Continued work on full feature-parity with Behavior API
  • Actorab...

Cluster Hardening, new Membership, Plugins and Singletons

Main features

• Introduction of Membership and Cluster.Event types and infrastructure
• Tons of cluster hardening and edge case handling
• Cluster documentation and initial feature set
• New feature: Actor (Cluster) Singleton
• DiningPhilosophers also now in Actorable APIs, thanks to Erik Hornberger! 🎉

Stats

A total of 41 issues was closed since the 0.3.0 release 🎉
You can view all resolved tickets on the 0.4.0 milestone.

commits  added  removed
     72  21653    16625 Konrad `ktoso` Malawski
     11   2449      620 Yim Lee
      1    566        0 Erik Hornberger
      1      4        1 Tom Doron

0.3.0 PoC XPC transport and Actorable interop

Main features

Developed an XPC integration Proof of Concept (NOT production ready!), showing how we could utilize the Actor concept's similarity to XPC Services as well as the GenActors infrastructure to generate statically type checked Actor<>s for interacting and defining XPC services using Actorables.

If or not this PoC will become something serious remains to be seen however -- if this seems compelling (esp. the ability to easily move or interact with code in XPC Services, Remote Nodes and/or Linux isolated processes) all using the same programming model, let us know.

• Introduced XPCTransport in DistributedActorsXPC module
• Introduced ActorableTestProbe
• Expanded capabilities of GenActors to generate Stubs and Codables.

Stats

A total of 9 issues was closed since the previous minor, all of them here: https://github.com/apple/swift-distributed-actors/milestone/7?closed=1

0.2.4 Make settings public

Main features

• SWIM settings were internal but should have been public -- fixed.

Stats

A total of 2 issues were resolved since 0.2.3, see here for details: https://github.com/apple/swift-distributed-actors/milestone/10?closed=1