Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data model for freezing UTXOs #376

Merged
merged 67 commits into from
Jul 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
3f0e06d
feat: add address, isFrozen and tag
markettes Jun 20, 2024
53e49ac
feat: filter frozen utxos
markettes Jun 20, 2024
74df0da
feat: utxo migration
markettes Jun 20, 2024
5ddea6d
feat: add postgres for nbxplorer
markettes Jun 21, 2024
c7f246d
fix: add migration for optional address and tag
markettes Jun 21, 2024
0feff81
feat: RegisterUTXOs job
markettes Jun 21, 2024
6f156ee
feat: add wallet to FMUTXO
markettes Jun 24, 2024
a87309f
feat(FMUTXO.cs): add IsSpent property to determine if UTXO is spent b…
markettes Jun 24, 2024
bdc206b
feat(FMUTXO.cs): add NBitcoin and NBXplorer dependencies for UTXO mod…
markettes Jun 24, 2024
5cc7705
feat: consolidate migrations
markettes Jun 24, 2024
728e024
fix: migration
markettes Jun 24, 2024
78920cf
refactor(ChannelOperationRequestRepositoryTests): add additional null…
markettes Jun 24, 2024
bd5dca9
Bump deps
Jossec101 Jun 24, 2024
b4ed8a4
Fix problem with new change in nbxplorer when tracking btc addresses
Jossec101 Jun 24, 2024
48196b5
feat: rebuild of the migration
markettes Jun 25, 2024
456200c
Merge branch 'update-deps' into utxo-freezing-data-model
markettes Jun 25, 2024
f8240e5
Merge branch 'main' into utxo-freezing-data-model
markettes Jun 25, 2024
190554c
EF Core already does execute the migration in a transaction scope, th…
Jossec101 Jun 25, 2024
b636c9e
feat: changes for deps update
markettes Jun 25, 2024
55a381e
Fix AddRangeAsync not using a generic function type making EF Core fa…
Jossec101 Jun 25, 2024
ba242c6
Fix RegisterUTXOsJob to use WalletId instead of Wallet property
Jossec101 Jun 25, 2024
bc80a8d
Merge branch 'main' into utxo-freezing-data-model
markettes Jun 25, 2024
2f95e5f
fix: rollback changes
markettes Jun 25, 2024
1efb82a
feat(FMUTXO.cs): add Blazorise.Extensions for enhanced functionality
markettes Jun 25, 2024
1ebcf1a
fix: remove error and set the creation time and the update time
markettes Jun 25, 2024
3b8efdd
refactor(FUTXORepository.cs): optimize UTXO retrieval logic to enhanc…
markettes Jun 25, 2024
711fcd2
fix: rollback the port
markettes Jun 26, 2024
19faae7
fix: remove isSpent
markettes Jun 26, 2024
c7cad38
refactor(FUTXORepository): replace GetFromUTXOs with GetByOutpoint fo…
markettes Jun 26, 2024
db218ec
fix(docker-compose.yml): change PostgreSQL service port mapping to av…
markettes Jun 26, 2024
1d4690d
fix: fix method type
markettes Jun 26, 2024
7e79ce6
fix: remove unused imports
markettes Jun 26, 2024
50ce070
fix: comments
markettes Jun 26, 2024
c0bee08
feat: UTXOTag in another file
markettes Jun 26, 2024
ca965cd
fix: set to dbtrie again
markettes Jun 26, 2024
79573e5
feat: explanation of the job
markettes Jun 26, 2024
f510933
Update src/Services/CoinSelectionService.cs
markettes Jun 26, 2024
32b7cb9
Update src/Services/CoinSelectionService.cs
markettes Jun 26, 2024
89df184
feat(RegisterUTXOsJob.cs): skip processing wallets without derivation…
markettes Jun 26, 2024
b51cd1a
fix: tags in the relations
markettes Jun 26, 2024
98236b4
fix: add throw
markettes Jun 26, 2024
69c1acf
chore: comments
markettes Jun 26, 2024
a38f089
refactor: remove job to add utxos
markettes Jun 27, 2024
f3a9a46
refactor: remove properties to rollback changes
markettes Jun 27, 2024
a69cf2d
refactor: remove unused methods
markettes Jun 27, 2024
78a5851
chore(FMUTXO.cs): remove unused using directives to clean up code
markettes Jun 27, 2024
8164d37
style(FUTXORepository.cs): remove extra whitespace for cleaner code f…
markettes Jun 27, 2024
8cc2a1b
refactor(Constants.cs): remove REGISTER_UTXOS_CRON constant and its e…
markettes Jun 27, 2024
bd6fd5d
style(FMUTXO.cs): remove extra whitespace for cleaner code formatting
markettes Jun 27, 2024
1dd3fa2
refactor(FMUTXO.cs): remove unused NBitcoin and NBXplorer imports to …
markettes Jun 27, 2024
54cd988
refactor: remove logic for FMUTXOs
markettes Jun 27, 2024
34565cb
refactor: remove spaces
markettes Jun 27, 2024
3592a36
refactor: rollback logic changes
markettes Jun 27, 2024
9a11cda
style(CoinSelectionService.cs): add missing line break for better cod…
markettes Jun 27, 2024
d7294ec
refactor: remove migration
markettes Jun 27, 2024
218f52e
feat: utxotag repository
markettes Jun 28, 2024
39ef3d4
refactor(UTXOTag.cs): remove FMUTXO navigation properties to decouple…
markettes Jun 28, 2024
51a664a
feat(IUTXOTagRepository, UTXOTagRepository): add GetTagByKeyAndOutpoi…
markettes Jun 28, 2024
eb8f9f7
style(UTXOTagRepository.cs): fix indentation for better code readability
markettes Jun 28, 2024
1ff1f20
refactor(UTXOTag): make Outpoint property non-nullable to enforce dat…
markettes Jun 28, 2024
b61e449
feat: exclude frozen utxos
markettes Jun 28, 2024
a81730f
fix: add repository
markettes Jun 28, 2024
a27b906
feat: migration
markettes Jun 28, 2024
063f64d
refactor(NodeGuardServiceTests.cs): add missing null parameter to con…
markettes Jun 28, 2024
e9d7315
feat: add unique constraint for key outpoint in the utxotag table
markettes Jun 28, 2024
8fb6d7c
test: fix tests and add one more
markettes Jun 28, 2024
750f09a
Update src/Helpers/Constants.cs
RodriFS Jul 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions docker/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ version: '3.4'
services:
nodeguard_postgres:
container_name: nodeguard_postgres
image: postgres:13
image: postgres:16
restart: always
environment:
POSTGRES_DB: nodeguard
Expand All @@ -29,7 +29,7 @@ services:
NBXPLORER_SIGNALFILESDIR: /datadir
#Keeping dbtrie for dev until it is fully removed since we would need to modify nbxplorer docker image to wait for the db to be ready
NBXPLORER_DBTRIE: 1
# NBXPLORER_POSTGRES: Host=nodeguard_postgres;Port=5432;Database=nbxplorer;Username=rw_dev;Password=rw_dev
# NBXPLORER_POSTGRES: Host=nbxplorer_postgres;Port=5432;Database=nbxplorer;Username=rw_dev;Password=rw_dev
NBXPLORER_CHAINS: "btc"
NBXPLORER_BTCRPCUSER: "polaruser"
NBXPLORER_BTCRPCPASSWORD: "polarpass"
Expand All @@ -38,11 +38,26 @@ services:
command: ["--noauth"]
volumes:
- "bitcoin_datadir:/root/.bitcoin"

nbxplorer_postgres:
container_name: nbxplorer_postgres
image: postgres:16
restart: always
environment:
POSTGRES_DB: nbxplorer
POSTGRES_USER: rw_dev
POSTGRES_PASSWORD: rw_dev
TZ: Europe/Madrid
volumes:
- nbxplorer_postgres_data:/var/lib/postgresql/data
ports:
- 35432:5432



volumes:
nodeguard_postgres_data:
bitcoin_datadir:
nbxplorer_datadir:
nbxplorer_postgres_data:
nodeguard_data_keys_dir:
2 changes: 2 additions & 0 deletions src/Data/ApplicationDbContext.cs
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,8 @@ protected override void OnModelCreating(ModelBuilder modelBuilder)
public DbSet<InternalWallet> InternalWallets { get; set; }

public DbSet<FMUTXO> FMUTXOs { get; set; }

public DbSet<UTXOTag> UTXOTags { get; set; }

public DbSet<LiquidityRule> LiquidityRules { get; set; }

Expand Down
3 changes: 3 additions & 0 deletions src/Data/Models/FMUTXO.cs
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,11 @@ public class FMUTXO : Entity, IEquatable<FMUTXO>

#region Relationships

// M-N Because if the UTXO is used in a request that gets cancelled,
// the UTXO should be unlocked and assigned to another request
public List<ChannelOperationRequest> ChannelOperationRequests { get; set; }

// Idem as ChannelOperationRequests
public List<WalletWithdrawalRequest> WalletWithdrawalRequests { get; set; }

#endregion Relationships
Expand Down
11 changes: 11 additions & 0 deletions src/Data/Models/UTXOTag.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
namespace NodeGuard.Data.Models;

public class UTXOTag : Entity
{
public string Key { get; set; }

public string Value { get; set; }

// Outpoint of the UTXO in format "hash-index"
public string Outpoint { get; set; }
}
2 changes: 1 addition & 1 deletion src/Data/Repositories/FUTXORepository.cs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
*
*/

using NodeGuard.Data.Models;
using NodeGuard.Data.Models;
using NodeGuard.Data.Repositories.Interfaces;
using Microsoft.EntityFrameworkCore;

Expand Down
2 changes: 1 addition & 1 deletion src/Data/Repositories/Interfaces/IFMUTXORepository.cs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
*
*/

using NodeGuard.Data.Models;
using NodeGuard.Data.Models;

namespace NodeGuard.Data.Repositories.Interfaces;

Expand Down
2 changes: 1 addition & 1 deletion src/Data/Repositories/Interfaces/IRepository.cs
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@
/// </summary>
/// <param name="T"></param>
/// <returns>A tuple (bool, string). The bool represents the call success and the string any possible message.</returns>
public Task<(bool, string?)> AddRangeAsync(List<T> T, ApplicationDbContext applicationDbContext);
public Task<(bool, string?)> AddRangeAsync<T>(List<T> entities, ApplicationDbContext applicationDbContext) where T : class;

Check warning on line 52 in src/Data/Repositories/Interfaces/IRepository.cs

View workflow job for this annotation

GitHub Actions / build-and-test

Type parameter 'T' has the same name as the type parameter from outer type 'IRepository<T>'
RodriFS marked this conversation as resolved.
Show resolved Hide resolved

/// <summary>
///Removes an existing entity of <see cref="IRepository{T}"></see>
Expand Down
22 changes: 22 additions & 0 deletions src/Data/Repositories/Interfaces/IUTXOTagRepository.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
using NodeGuard.Data.Models;

namespace NodeGuard.Data.Repositories.Interfaces;

public interface IUTXOTagRepository
{
Task<UTXOTag?> GetByOutpoint(string outpoint);

Task<UTXOTag?> GetByKeyAndOutpoint(string key, string outpoint);

Task<List<UTXOTag>> GetByKeyValue(string key, string value);

Task<(bool, string?)> AddAsync(UTXOTag type);

Task<(bool, string?)> AddRangeAsync(List<UTXOTag> type);

(bool, string?) Remove(UTXOTag type);

(bool, string?) RemoveRange(List<UTXOTag> types);

(bool, string?) Update(UTXOTag type);
}
7 changes: 4 additions & 3 deletions src/Data/Repositories/Repository.cs
Original file line number Diff line number Diff line change
Expand Up @@ -65,19 +65,20 @@
return (rowsChanged, null);
}

public async Task<(bool, string?)> AddRangeAsync(List<T> T, ApplicationDbContext applicationDbContext)
public async Task<(bool, string?)> AddRangeAsync<T>(List<T> entities, ApplicationDbContext applicationDbContext) where T : class

Check warning on line 68 in src/Data/Repositories/Repository.cs

View workflow job for this annotation

GitHub Actions / build-and-test

Type parameter 'T' has the same name as the type parameter from outer type 'Repository<T>'
RodriFS marked this conversation as resolved.
Show resolved Hide resolved
{
if (T == null) throw new ArgumentNullException(nameof(T));
if (entities == null) throw new ArgumentNullException(nameof(entities));

var rowsChanged = false;
try
{
await applicationDbContext.AddRangeAsync(T);
await applicationDbContext.AddRangeAsync(entities);
rowsChanged = await applicationDbContext.SaveChangesAsync() > 0;
}
catch (Exception e)
{
_logger.LogError(e, "Error on repository");
return (false, e.Message);
}

return (rowsChanged, null);
Expand Down
82 changes: 82 additions & 0 deletions src/Data/Repositories/UTXOTagRepository.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
using Microsoft.EntityFrameworkCore;
using NodeGuard.Data.Models;
using NodeGuard.Data.Repositories.Interfaces;

namespace NodeGuard.Data.Repositories;

public class UTXOTagRepository : IUTXOTagRepository
{
private readonly IRepository<UTXOTag> _repository;
private readonly ILogger<UTXOTagRepository> _logger;
private readonly IDbContextFactory<ApplicationDbContext> _dbContextFactory;

public UTXOTagRepository(IRepository<UTXOTag> repository,
ILogger<UTXOTagRepository> logger,
IDbContextFactory<ApplicationDbContext> dbContextFactory)
{
_repository = repository;
_logger = logger;
_dbContextFactory = dbContextFactory;
}

public async Task<UTXOTag?> GetByOutpoint(string outpoint)
{
await using var applicationDbContext = await _dbContextFactory.CreateDbContextAsync();

return await applicationDbContext.UTXOTags.FirstOrDefaultAsync(x => x.Outpoint == outpoint);
}

public async Task<UTXOTag?> GetByKeyAndOutpoint(string key, string outpoint)
{
await using var applicationDbContext = await _dbContextFactory.CreateDbContextAsync();

return await applicationDbContext.UTXOTags.FirstOrDefaultAsync(x => x.Key == key && x.Outpoint == outpoint);
}

public async Task<List<UTXOTag>> GetByKeyValue(string key, string value)
{
await using var applicationDbContext = await _dbContextFactory.CreateDbContextAsync();

return await applicationDbContext.UTXOTags
.Where(x => x.Key == key && x.Value == value).ToListAsync();
}

public async Task<(bool, string?)> AddAsync(UTXOTag type)
{
await using var applicationDbContext = await _dbContextFactory.CreateDbContextAsync();

type.SetCreationDatetime();

return await _repository.AddAsync(type, applicationDbContext);
}

public async Task<(bool, string?)> AddRangeAsync(List<UTXOTag> type)
{
await using var applicationDbContext = await _dbContextFactory.CreateDbContextAsync();

return await _repository.AddRangeAsync(type, applicationDbContext);
}

public (bool, string?) Remove(UTXOTag type)
{
using var applicationDbContext = _dbContextFactory.CreateDbContext();

return _repository.Remove(type, applicationDbContext);
}

public (bool, string?) RemoveRange(List<UTXOTag> types)
{
using var applicationDbContext = _dbContextFactory.CreateDbContext();

return _repository.RemoveRange(types, applicationDbContext);
}

public (bool, string?) Update(UTXOTag type)
{
using var applicationDbContext = _dbContextFactory.CreateDbContext();

type.SetUpdateDatetime();

return _repository.Update(type, applicationDbContext);
}
}
4 changes: 3 additions & 1 deletion src/Helpers/Constants.cs
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,9 @@
/// Max ratio of the tx total input sum that could be used as fee
/// </summary>
public static decimal MAX_TX_FEE_RATIO =0.5m;


public const string IsFrozenTag = "frozen";

private static string? GetEnvironmentalVariableOrThrowIfNotTesting(string envVariableName, string? errorMessage = null)
{
// If it is a command from ef or a test, ignore the empty env variables
Expand Down Expand Up @@ -116,7 +118,7 @@
// Connections
POSTGRES_CONNECTIONSTRING = Environment.GetEnvironmentVariable("POSTGRES_CONNECTIONSTRING") ?? POSTGRES_CONNECTIONSTRING;

NBXPLORER_URI = GetEnvironmentalVariableOrThrowIfNotTesting("NBXPLORER_URI");

Check warning on line 121 in src/Helpers/Constants.cs

View workflow job for this annotation

GitHub Actions / build-and-test

Possible null reference assignment.

NBXPLORER_BTCRPCURL = Environment.GetEnvironmentVariable("NBXPLORER_BTCRPCURL");

Expand Down
Loading
Loading