Pull out Cardano specifics to different modules

There's a cyclic dependency issue in this commit that is fixed in the next one. The problem is that I pulled out `ConsensusModePeerTargets` which depend on `PeerSelectionTargets` and then the module that defines `PeerSelectionTargets` requires `ConsensusModePeerTargets`. This is fixed by abstracting over Cardano Node specific types such as `ConsensusModePeerTargets`. Add extension points to Diffusion data structures Reorganised and refactored Diffusion modules Moved Diffusion Cardano specifics to their own folder, separating utilities and functions accordingly. Changed Diffusion selection by splitting P2P in two: P2PCardano and P2P. The former instantiates diffusion using Cardano Node specifics. The latter is able to accommodate with other use cases. Added a Minimal/Node.hs that is incomplete but will hold the minimal Node diffusion instantiation example that highlights how one can use their own custom types. Refactor localRootPeers and PeerSelectionActions - Removed PeerSelectionActionsArgs: This data type was bit redundant. - Reorganized DNSActions and LedgerPeersArgs - Improved API for PeerSelectionActions Renamed Cardano.Node to Cardano.Network Refactored ConsensusModePeerTargets Refactor PeerSelection{Views, Counters} Make PeerSelection{Views, Counters} extensible by third party users. Generalised PublicRootPeers First refactor of Outbound Governor Make Diffusion Module Fully General and Polymorphic - Generalize MinimalP2P Module - `MinimalP2P` is _the_ module for diffusion initialization. - Updated parameter plumbing to support generalization and polymorphism. - Enhance ArgumentsExtra - Added additional parameters to `ArgumentsExtra` to facilitate a more general and polymorphic diffusion setup. This includes `requestPublicRootPeers`, `peerChurnGovernor`, etc.. - Refactor PeerSelection.Governor.Monitor - Moved Cardano-specific monitoring actions to a separate module. - Implemented general `localRoots` and `targetPeers` monitoring actions. - Pulled out Cardano specific implementation details that allowed to generalise the minimal set of monitoring actions. Added fields to `ExtraGuardedDecisions` so that 3rd party users can pass their own `localRoots` and `targetPeers` actions and also backdoors/callbacks like `abortGovernor`, `updateWithState` to not only support Cardano specific needs but to give more power/flexibility to 3rd party users. - Polymorphize Diffusion Module - Removed `CardanoP2P` dependencies from the `Diffusion` module. - Ensured the `Diffusion` module remains fully polymorphic and adaptable to different use cases. - Refactored test suite to compile with new changes. Update CHaP and Fix build Remove daBlockFetchMode Address Review Feedback Fix Churn no timeouts test Moved Cardano folder to Ouroboros folder Increase shortDelay when churning to fix test Recover sigusr1 signal handler Make addresses polymorphic Refactor extraDebugState appropriately Added P2PDecisionType and Refactored requestPublicRootPeers type Address Review Feedback
IntersectMBO · Feb 6, 2025 · e1b0745 · e1b0745
1 parent a933425
commit e1b0745
Show file tree

Hide file tree

Showing 68 changed files with 7,143 additions and 3,870 deletions.
diff --git a/cabal.project b/cabal.project
@@ -18,7 +18,7 @@ index-state:
   , hackage.haskell.org 2024-12-10T16:20:07Z
 
   -- Bump this if you need newer packages from CHaP
-  , cardano-haskell-packages 2024-09-26T15:16:07Z
+  , cardano-haskell-packages 2024-11-26T16:00:26Z
 
 packages: ./cardano-ping
           ./monoidal-synchronisation

diff --git a/flake.lock b/flake.lock
diff --git a/ouroboros-network-api/ouroboros-network-api.cabal b/ouroboros-network-api/ouroboros-network-api.cabal
@@ -23,28 +23,29 @@ flag asserts
 library
   hs-source-dirs: src
   exposed-modules:
+    Cardano.Network.ConsensusMode
+    Cardano.Network.PeerSelection.Bootstrap
+    Cardano.Network.PeerSelection.LocalRootPeers
+    Cardano.Network.PeerSelection.PeerTrustable
+    Cardano.Network.Types
     Ouroboros.Network.AnchoredFragment
     Ouroboros.Network.AnchoredSeq
     Ouroboros.Network.Block
     Ouroboros.Network.BlockFetch.ConsensusInterface
     Ouroboros.Network.CodecCBORTerm
-    Ouroboros.Network.ConsensusMode
     Ouroboros.Network.ControlMessage
     Ouroboros.Network.Handshake
     Ouroboros.Network.Handshake.Acceptable
     Ouroboros.Network.Handshake.Queryable
     Ouroboros.Network.Magic
     Ouroboros.Network.NodeToClient.Version
     Ouroboros.Network.NodeToNode.Version
-    Ouroboros.Network.PeerSelection.Bootstrap
     Ouroboros.Network.PeerSelection.LedgerPeers.Type
     Ouroboros.Network.PeerSelection.LedgerPeers.Utils
-    Ouroboros.Network.PeerSelection.LocalRootPeers
     Ouroboros.Network.PeerSelection.PeerAdvertise
     Ouroboros.Network.PeerSelection.PeerMetric.Type
     Ouroboros.Network.PeerSelection.PeerSharing
     Ouroboros.Network.PeerSelection.PeerSharing.Codec
-    Ouroboros.Network.PeerSelection.PeerTrustable
     Ouroboros.Network.PeerSelection.RelayAccessPoint
     Ouroboros.Network.Point
     Ouroboros.Network.Protocol.Limits

diff --git a/...pi/src/Ouroboros/Network/ConsensusMode.hs → ...-api/src/Cardano/Network/ConsensusMode.hs b/...pi/src/Ouroboros/Network/ConsensusMode.hs → ...-api/src/Cardano/Network/ConsensusMode.hs
@@ -1,7 +1,7 @@
 {-# LANGUAGE DeriveAnyClass #-}
 {-# LANGUAGE DeriveGeneric  #-}
 
-module Ouroboros.Network.ConsensusMode where
+module Cardano.Network.ConsensusMode where
 
 import Data.Aeson
 import GHC.Generics (Generic)

diff --git a/...oboros/Network/PeerSelection/Bootstrap.hs → ...ardano/Network/PeerSelection/Bootstrap.hs b/...oboros/Network/PeerSelection/Bootstrap.hs → ...ardano/Network/PeerSelection/Bootstrap.hs
@@ -1,16 +1,15 @@
 {-# LANGUAGE DeriveGeneric     #-}
 {-# LANGUAGE OverloadedStrings #-}
 
-module Ouroboros.Network.PeerSelection.Bootstrap
+module Cardano.Network.PeerSelection.Bootstrap
   ( UseBootstrapPeers (..)
   , isBootstrapPeersEnabled
   , requiresBootstrapPeers
   , isNodeAbleToMakeProgress
   ) where
 
+import Cardano.Network.Types (LedgerStateJudgement (..))
 import GHC.Generics (Generic)
-import Ouroboros.Network.PeerSelection.LedgerPeers.Type
-           (LedgerStateJudgement (..))
 import Ouroboros.Network.PeerSelection.RelayAccessPoint (RelayAccessPoint)
 
 data UseBootstrapPeers = DontUseBootstrapPeers

diff --git a/...s/Network/PeerSelection/LocalRootPeers.hs → ...o/Network/PeerSelection/LocalRootPeers.hs b/...s/Network/PeerSelection/LocalRootPeers.hs → ...o/Network/PeerSelection/LocalRootPeers.hs
@@ -1,6 +1,6 @@
 {-# LANGUAGE DeriveGeneric #-}
 
-module Ouroboros.Network.PeerSelection.LocalRootPeers (OutboundConnectionsState (..)) where
+module Cardano.Network.PeerSelection.LocalRootPeers (OutboundConnectionsState (..)) where
 
 import GHC.Generics
 import NoThunks.Class

diff --git a/...os/Network/PeerSelection/PeerTrustable.hs → ...no/Network/PeerSelection/PeerTrustable.hs b/...os/Network/PeerSelection/PeerTrustable.hs → ...no/Network/PeerSelection/PeerTrustable.hs
@@ -1,6 +1,6 @@
 {-# LANGUAGE DeriveGeneric #-}
 
-module Ouroboros.Network.PeerSelection.PeerTrustable (PeerTrustable (..)) where
+module Cardano.Network.PeerSelection.PeerTrustable (PeerTrustable (..)) where
 
 import GHC.Generics (Generic)
 

diff --git a/ouroboros-network-api/src/Cardano/Network/Types.hs b/ouroboros-network-api/src/Cardano/Network/Types.hs
@@ -0,0 +1,29 @@
+{-# LANGUAGE DeriveGeneric              #-}
+{-# LANGUAGE DerivingStrategies         #-}
+{-# LANGUAGE GeneralizedNewtypeDeriving #-}
+
+module Cardano.Network.Types where
+
+import Data.Aeson (FromJSON)
+import GHC.Generics (Generic)
+import NoThunks.Class (NoThunks)
+
+-- | Wether the node is caught up or fell too far behind the chain
+data LedgerStateJudgement = YoungEnough | TooOld
+  deriving (Eq, Show, Generic)
+
+instance NoThunks LedgerStateJudgement
+
+-- | Minimum number of hot big ledger peers in Genesis mode
+--   for trusted state to be signalled to Consensus. This number
+--   should be smaller than the `targetNumberOfActiveBigLedgerPeers`
+--   but greater than 1. In Genesis, we may demote a big ledger peer
+--   for underperformance, but not promote a replacement immediately
+--   to guard against adversaries which may want to slow down our
+--   progress.
+--
+newtype NumberOfBigLedgerPeers =
+  NumberOfBigLedgerPeers { getNumberOfBigLedgerPeers :: Int }
+  deriving stock (Eq, Show)
+  deriving newtype (FromJSON)
+
diff --git a/ouroboros-network-api/src/Ouroboros/Network/BlockFetch/ConsensusInterface.hs b/ouroboros-network-api/src/Ouroboros/Network/BlockFetch/ConsensusInterface.hs
@@ -23,11 +23,10 @@ import GHC.Generics (Generic)
 import GHC.Stack (HasCallStack)
 import NoThunks.Class (NoThunks)
 
+import Cardano.Network.ConsensusMode (ConsensusMode (..))
+import Cardano.Network.Types (LedgerStateJudgement (..))
 import Ouroboros.Network.AnchoredFragment (AnchoredFragment)
 import Ouroboros.Network.Block
-import Ouroboros.Network.ConsensusMode (ConsensusMode (..))
-import Ouroboros.Network.PeerSelection.LedgerPeers.Type
-           (LedgerStateJudgement (..))
 import Ouroboros.Network.SizeInBytes (SizeInBytes)
 
 data PraosFetchMode =

diff --git a/ouroboros-network-api/src/Ouroboros/Network/PeerSelection/LedgerPeers/Type.hs b/ouroboros-network-api/src/Ouroboros/Network/PeerSelection/LedgerPeers/Type.hs
@@ -18,13 +18,12 @@ module Ouroboros.Network.PeerSelection.LedgerPeers.Type
   ( PoolStake (..)
   , AccPoolStake (..)
   , IsBigLedgerPeer (..)
-  , LedgerStateJudgement (..)
   , LedgerPeersConsensusInterface (..)
+  , mapExtraAPI
   , UseLedgerPeers (..)
   , AfterSlot (..)
   , LedgerPeersKind (..)
   , LedgerPeerSnapshot (.., LedgerPeerSnapshot)
-  , MinBigLedgerPeersForTrustedState (..)
   , isLedgerPeersEnabled
   , compareLedgerPeerSnapshotApproximate
   ) where
@@ -46,19 +45,6 @@ import Data.Aeson.Types
 import NoThunks.Class
 import Ouroboros.Network.PeerSelection.RelayAccessPoint
 
--- | Minimum number of hot big ledger peers in Genesis mode
---   for trusted state to be signalled to Consensus. This number
---   should be smaller than the `targetNumberOfActiveBigLedgerPeers`
---   but greater than 1. In Genesis, we may demote a big ledger peer
---   for underperformance, but not promote a replacement immediately
---   to guard against adversaries which may want to slow down our
---   progress.
---
-newtype MinBigLedgerPeersForTrustedState =
-  MinBigLedgerPeersForTrustedState { getMinBigLedgerPeersForTrustedState :: Int }
-  deriving stock (Eq, Show)
-  deriving newtype (FromJSON)
-
 -- |The type of big ledger peers that is serialised or later
 -- provided by node configuration for the networking layer
 -- to connect to when syncing.
@@ -237,18 +223,19 @@ data IsBigLedgerPeer
    | IsNotBigLedgerPeer
   deriving Eq
 
--- | Wether the node is caught up or fell too far behind the chain
-data LedgerStateJudgement = YoungEnough | TooOld
-  deriving (Eq, Show, Generic, NoThunks)
-
 -- | Return ledger state information and ledger peers.
 --
-data LedgerPeersConsensusInterface m = LedgerPeersConsensusInterface {
-    lpGetLatestSlot           :: STM m (WithOrigin SlotNo),
-    lpGetLedgerStateJudgement :: STM m LedgerStateJudgement,
-    lpGetLedgerPeers          :: STM m [(PoolStake, NonEmpty RelayAccessPoint)]
+data LedgerPeersConsensusInterface extraAPI m = LedgerPeersConsensusInterface {
+    lpGetLatestSlot  :: STM m (WithOrigin SlotNo)
+  , lpGetLedgerPeers :: STM m [(PoolStake, NonEmpty RelayAccessPoint)]
+    -- | Extension point so that third party users can add more actions
+  , lpExtraAPI       :: extraAPI
   }
 
+mapExtraAPI :: (a -> b) -> LedgerPeersConsensusInterface a m -> LedgerPeersConsensusInterface b m
+mapExtraAPI f lpci@LedgerPeersConsensusInterface{ lpExtraAPI = api } =
+  lpci { lpExtraAPI = f api }
+
 instance ToJSON RelayAccessPointCoded where
   toJSON (RelayAccessPointCoded (RelayAccessDomain domain port)) =
     object

diff --git a/ouroboros-network/ouroboros-network.cabal b/ouroboros-network/ouroboros-network.cabal
@@ -55,6 +55,20 @@ library
   -- This has to be tidied up once the design becomes clear.
   exposed-modules:
     Control.Concurrent.Class.MonadSTM.Strict.TMergeVar
+    Ouroboros.Cardano.Diffusion.Configuration
+    Ouroboros.Cardano.Diffusion.Handlers
+    Ouroboros.Cardano.Diffusion.Policies
+    Ouroboros.Cardano.Network.ArgumentsExtra
+    Ouroboros.Cardano.Network.LedgerPeerConsensusInterface
+    Ouroboros.Cardano.Network.PeerSelection.Churn.ExtraArguments
+    Ouroboros.Cardano.Network.PeerSelection.Governor.Monitor
+    Ouroboros.Cardano.Network.PeerSelection.Governor.PeerSelectionActions
+    Ouroboros.Cardano.Network.PeerSelection.Governor.PeerSelectionState
+    Ouroboros.Cardano.Network.PeerSelection.Governor.Types
+    Ouroboros.Cardano.Network.PublicRootPeers
+    Ouroboros.Cardano.Network.Types
+    Ouroboros.Cardano.PeerSelection.Churn
+    Ouroboros.Cardano.PeerSelection.PeerSelectionActions
     Ouroboros.Network.BlockFetch
     Ouroboros.Network.BlockFetch.Client
     Ouroboros.Network.BlockFetch.ClientRegistry
@@ -66,6 +80,7 @@ library
     Ouroboros.Network.BlockFetch.State
     Ouroboros.Network.DeltaQ
     Ouroboros.Network.Diffusion
+    Ouroboros.Network.Diffusion.Common
     Ouroboros.Network.Diffusion.Configuration
     Ouroboros.Network.Diffusion.NonP2P
     Ouroboros.Network.Diffusion.P2P
@@ -74,7 +89,10 @@ library
     Ouroboros.Network.KeepAlive
     Ouroboros.Network.NodeToClient
     Ouroboros.Network.NodeToNode
+    Ouroboros.Network.PeerSelection.Churn
     Ouroboros.Network.PeerSelection.Governor
+    Ouroboros.Network.PeerSelection.Governor.Monitor
+    Ouroboros.Network.PeerSelection.Governor.Types
     Ouroboros.Network.PeerSelection.LedgerPeers
     Ouroboros.Network.PeerSelection.PeerMetric
     Ouroboros.Network.PeerSelection.PeerSelectionActions
@@ -96,16 +114,12 @@ library
     Ouroboros.Network.TxSubmission.Outbound
 
   other-modules:
-    Ouroboros.Network.Diffusion.Common
     Ouroboros.Network.Diffusion.Utils
-    Ouroboros.Network.PeerSelection.Churn
     Ouroboros.Network.PeerSelection.Governor.ActivePeers
     Ouroboros.Network.PeerSelection.Governor.BigLedgerPeers
     Ouroboros.Network.PeerSelection.Governor.EstablishedPeers
     Ouroboros.Network.PeerSelection.Governor.KnownPeers
-    Ouroboros.Network.PeerSelection.Governor.Monitor
     Ouroboros.Network.PeerSelection.Governor.RootPeers
-    Ouroboros.Network.PeerSelection.Governor.Types
     Ouroboros.Network.PeerSelection.LedgerPeers.Common
     Ouroboros.Network.PeerSelection.RootPeersDNS.LedgerPeers
 
@@ -244,33 +258,35 @@ library sim-tests-lib
   exposed-modules:
     Ouroboros.Network.BlockFetch.Examples
     Ouroboros.Network.MockNode
-    Test.Ouroboros.Network.BlockFetch
-    Test.Ouroboros.Network.KeepAlive
+    Test.Ouroboros.Network.Diffusion.Node.ChainDB
+    Test.Ouroboros.Network.Diffusion.Node.Kernel
+    Test.Ouroboros.Network.Diffusion.Node.MiniProtocols
+    Test.Ouroboros.Network.Diffusion.Policies
+    Test.Ouroboros.Network.Diffusion.Testnet.Cardano
+    Test.Ouroboros.Network.Diffusion.Testnet.Cardano.Simulation
+    Test.Ouroboros.Network.Diffusion.Testnet.P2P.Node
     Test.Ouroboros.Network.LedgerPeers
+    Test.Ouroboros.Network.MiniProtocols.BlockFetch
+    Test.Ouroboros.Network.MiniProtocols.KeepAlive
+    Test.Ouroboros.Network.MiniProtocols.TxSubmission
     Test.Ouroboros.Network.MockNode
     Test.Ouroboros.Network.Mux
     Test.Ouroboros.Network.NodeToClient.Version
     Test.Ouroboros.Network.NodeToNode.Version
     Test.Ouroboros.Network.Orphans
     Test.Ouroboros.Network.PeerSelection
+    Test.Ouroboros.Network.PeerSelection.Cardano.Instances
+    Test.Ouroboros.Network.PeerSelection.Cardano.LocalRootPeers
+    Test.Ouroboros.Network.PeerSelection.Cardano.MockEnvironment
+    Test.Ouroboros.Network.PeerSelection.Cardano.PublicRootPeers
     Test.Ouroboros.Network.PeerSelection.Instances
     Test.Ouroboros.Network.PeerSelection.Json
     Test.Ouroboros.Network.PeerSelection.KnownPeers
     Test.Ouroboros.Network.PeerSelection.LocalRootPeers
-    Test.Ouroboros.Network.PeerSelection.MockEnvironment
     Test.Ouroboros.Network.PeerSelection.PeerGraph
     Test.Ouroboros.Network.PeerSelection.PeerMetric
-    Test.Ouroboros.Network.PeerSelection.PublicRootPeers
     Test.Ouroboros.Network.PeerSelection.RootPeersDNS
     Test.Ouroboros.Network.PeerState
-    Test.Ouroboros.Network.Testnet
-    Test.Ouroboros.Network.Testnet.Internal
-    Test.Ouroboros.Network.Testnet.Node
-    Test.Ouroboros.Network.Testnet.Node.ChainDB
-    Test.Ouroboros.Network.Testnet.Node.Kernel
-    Test.Ouroboros.Network.Testnet.Node.MiniProtocols
-    Test.Ouroboros.Network.Testnet.Policies
-    Test.Ouroboros.Network.TxSubmission
     Test.Ouroboros.Network.Version
 
 -- Simulation tests, and IO tests which don't require native system calls.

diff --git a/...Ouroboros/Network/Testnet/Node/ChainDB.hs → ...roboros/Network/Diffusion/Node/ChainDB.hs b/...Ouroboros/Network/Testnet/Node/ChainDB.hs → ...roboros/Network/Diffusion/Node/ChainDB.hs
@@ -1,4 +1,4 @@
-module Test.Ouroboros.Network.Testnet.Node.ChainDB
+module Test.Ouroboros.Network.Diffusion.Node.ChainDB
   ( ChainDB (..)
   , SelectChain (..)
   , newChainDB

diff --git a/.../Ouroboros/Network/Testnet/Node/Kernel.hs → ...uroboros/Network/Diffusion/Node/Kernel.hs b/.../Ouroboros/Network/Testnet/Node/Kernel.hs → ...uroboros/Network/Diffusion/Node/Kernel.hs
@@ -6,7 +6,7 @@
 
 -- | Node kernel which does chain selection and block production.
 --
-module Test.Ouroboros.Network.Testnet.Node.Kernel
+module Test.Ouroboros.Network.Diffusion.Node.Kernel
   ( -- * Common types
     NtNAddr
   , NtNAddr_ (..)
@@ -86,8 +86,8 @@ import Ouroboros.Network.PeerSelection.RelayAccessPoint (RelayAccessPoint (..))
 import Ouroboros.Network.PeerSharing (PeerSharingAPI, PeerSharingRegistry (..),
            newPeerSharingAPI, newPeerSharingRegistry,
            ps_POLICY_PEER_SHARE_MAX_PEERS, ps_POLICY_PEER_SHARE_STICKY_TIME)
-import Test.Ouroboros.Network.Testnet.Node.ChainDB (ChainDB (..))
-import Test.Ouroboros.Network.Testnet.Node.ChainDB qualified as ChainDB
+import Test.Ouroboros.Network.Diffusion.Node.ChainDB (ChainDB (..))
+import Test.Ouroboros.Network.Diffusion.Node.ChainDB qualified as ChainDB
 import Test.QuickCheck (Arbitrary (..), choose, chooseInt, frequency, oneof)