Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1936 add api log to log all api calls #2319

Open
wants to merge 27 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
5617f6f
Add logging classes for HTTP transactions and exceptions
pournasserian Nov 29, 2024
b450d67
Merge branch 'dev' into 1936-add-api-log-to-log-all-api-calls
pournasserian Nov 29, 2024
3075ea1
Refactor logging and add HTTP logging middleware
pournasserian Nov 29, 2024
698ad63
Refactor logging and exception handling
pournasserian Nov 29, 2024
6807cb9
Refactor logging and add ApiTokenKey support
pournasserian Nov 29, 2024
3fe77ca
Ensure HttpLogRepository initializes database before logging
pournasserian Nov 29, 2024
c813d3c
Suppress IDE0290 warning in JwtAuthorizationMiddleware
pournasserian Nov 29, 2024
515add3
Refactor logging and repository initialization
pournasserian Nov 29, 2024
1e95ace
Temporarily disable request/response body logging
pournasserian Nov 30, 2024
bcd0981
Fix collection name typo in HttpLogRepository.cs
pournasserian Nov 30, 2024
3099298
Add _isInitialized flag to HttpLogRepository
pournasserian Nov 30, 2024
6c2df24
Refactor HTTP logging and configuration
pournasserian Nov 30, 2024
77ac979
Refactor HttpLog namespace and clean up directives
pournasserian Nov 30, 2024
1741d7f
Add detailed logging documentation and expand HttpLog class
pournasserian Nov 30, 2024
ec4edce
Add HttpLogs table for detailed HTTP request/response logging
pournasserian Nov 30, 2024
40d8751
Add HttpLogs table to Setup.sql for detailed HTTP logging
pournasserian Nov 30, 2024
95c890c
Add JSON conversion for HttpLog properties in DbContext
pournasserian Dec 1, 2024
1b8efcc
Switch to SQLite and add HTTP logging support
pournasserian Dec 1, 2024
fd25890
Refactor logging and update database configuration
pournasserian Dec 1, 2024
c98aeae
Refactor HttpLogService to use background processing queue
pournasserian Dec 1, 2024
a1a1f37
Refactor HTTP logging and add background processing
pournasserian Dec 1, 2024
c176340
Restore HTTP logging functionality
pournasserian Dec 1, 2024
c07d078
Refactor HttpLogConfig and update logging classes
pournasserian Dec 1, 2024
a2e7cf4
Update HTTP logging configuration documentation
pournasserian Dec 1, 2024
20fb534
Refactor API context and add new API result classes
pournasserian Dec 5, 2024
a07b14e
Refactor ApiServiceExtensions DI and using directives
pournasserian Dec 5, 2024
3993e86
Merge branch 'dev' into 1936-add-api-log-to-log-all-api-calls
pournasserian Dec 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 127 additions & 0 deletions docs/Logging.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# Logging

## HTTP Logging for API

The HTTP Logging Middleware `HttpLoggingMiddleware.cs` is responsible for capturing detailed information about every HTTP request and response passing through the application. This includes logging the request and response metadata, body (if enabled), and any exceptions that occur during processing. The logged data can be stored in a database for auditing, debugging, or analytical purposes.

### Key Features

- Logs HTTP request and response metadata such as headers, method, URL, and body.
- Measures the duration of request processing.
- Captures exception details if an error occurs during processing.
- Configurable options to enable or disable logging of request and response bodies.

### How It Works

1. The middleware intercepts all HTTP requests and responses.
2. If configured, it reads the request and response bodies.
3. Exceptions encountered during request processing are captured and logged.
4. Asynchronous logging ensures minimal performance impact.
5. The collected data is structured into an `HttpLog` object and saved to the database.

---

### Configuration Properties

The `HttpLogging` section in the appsettings.json file allows you to customize the behavior of the HTTP logging middleware and background processor. Below are the available configuration properties:

| Property | Type | Default Value | Description |
|------------------------|---------|---------------|-----------------------------------------------------------------------------|
| `Enable` | `bool` | `false` | Toggles the logging functionality on or off. |
| `EnableRequestBody` | `bool` | `false` | Enables or disables logging of the HTTP request body. |
| `EnableResponseBody` | `bool` | `false` | Enables or disables logging of the HTTP response body. |
| `BatchSize` | `int` | `50` | Specifies the number of log entries to process in a single batch. |

### Example `appsettings.json` Configuration

```json
"HttpLogging": {
"Enable": true,
"EnableRequestBody": false,
"EnableResponseBody": false,
"BatchSize": 50
}
```

## HttpLog Details

The `HttpLog` class is used to store detailed information about HTTP requests, responses, and exceptions encountered during processing. This data can be used for debugging, auditing, or performance analysis.

### General Information

| Property | Type | Description |
|-----------------------|--------|---------------------------------------------------------------|
| `StatusCode` | `int` | The HTTP status code returned in the response. |
| `Duration` | `long` | The total time taken to process the request, in milliseconds. |
| `AssemblyName` | `string` | The name of the assembly handling the request. |
| `AssemblyVersion` | `string` | The version of the assembly handling the request. |
| `ProcessId` | `int` | The process ID of the application handling the request. |
| `ProcessName` | `string` | The name of the process handling the request. |
| `ThreadId` | `int` | The thread ID of the thread handling the request. |
| `MemoryUsage` | `long` | The memory usage (in bytes) at the time of logging. |
| `MachineName` | `string` | The name of the machine handling the request. |
| `EnvironmentName` | `string` | The environment name (e.g., Development, Production). |
| `EnvironmentUserName` | `string` | The username of the user running the application. |
| `IsAuthenticated` | `bool` | Indicates whether the request is authenticated. |
| `Language` | `string` | The language preference of the user making the request. |
| `SessionId` | `string` | The session ID associated with the request. |
| `StartDate` | `DateTime` | The start date and time of the request. |
| `TraceId` | `string` | A unique identifier for tracing the request. |
| `UniqueId` | `string` | A unique identifier associated with the request. |
| `UserId` | `Guid` | The ID of the user making the request. |
| `UserIp` | `string` | The IP address of the user making the request. |
| `Username` | `string` | The username of the authenticated user. |
| `ApiTokenKey` | `string` | The API token key used for authentication (if applicable). |

---

### Request Details

| Property | Type | Description |
|--------------------|-------------------------|--------------------------------------------------------------|
| `ReqUrl` | `string` | The full URL of the request. |
| `ReqProtocol` | `string` | The protocol used in the request (e.g., HTTP/1.1). |
| `ReqMethod` | `string` | The HTTP method of the request (e.g., GET, POST). |
| `ReqScheme` | `string` | The scheme used in the request (e.g., http, https). |
| `ReqPathBase` | `string` | The base path of the request. |
| `ReqPath` | `string` | The path of the request. |
| `QueryString` | `string` | The query string of the request. |
| `ReqContentType` | `string` | The content type of the request. |
| `ReqContentLength` | `long?` | The content length of the request body (if available). |
| `ReqBody` | `string?` | The body of the request (if enabled). |
| `ReqHeaders` | `Dictionary<string, string>` | The HTTP headers of the request. |

---

### Response Details

| Property | Type | Description |
|--------------------|-------------------------|--------------------------------------------------------------|
| `ResContentType` | `string` | The content type of the response. |
| `ResContentLength` | `long?` | The content length of the response body (if available). |
| `ResBody` | `string?` | The body of the response (if enabled). |
| `ResHeaders` | `Dictionary<string, string>` | The HTTP headers of the response. |

---

### Exception Details

| Property | Type | Description |
|----------------|-------------|---------------------------------------------------------------|
| `ExData` | `IDictionary?` | Additional data associated with the exception. |
| `ExHelpLink` | `string?` | A link to documentation or support for the exception. |
| `ExHResult` | `int?` | The HRESULT code for the exception. |
| `ExMessage` | `string?` | The message describing the exception. |
| `ExSource` | `string?` | The source of the exception (e.g., assembly or method name). |
| `ExStackTrace` | `string?` | The stack trace for the exception. |

---

### Usage

The `HttpLog` class provides a structured format for capturing request, response, and exception data. It is typically populated by the HTTP Logging Middleware and stored in a database or other logging infrastructure for further analysis.

This data is useful for:
- Debugging and resolving issues in the application.
- Auditing HTTP interactions for compliance or monitoring.
- Analyzing application performance and identifying bottlenecks.
9 changes: 7 additions & 2 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,7 @@
# FluentCMS.Docs
FluentCMS documentation
# FluentCMS

## Documentation

## Logging

For detailed information about HTTP Logging functionality, configuration, and usage, refer to the [Logging Documentation](Logging.md).
231 changes: 231 additions & 0 deletions src/Backend/FluentCMS.Entities/HttpLog.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,231 @@
using System.Collections;

namespace FluentCMS.Entities;

/// <summary>
/// Represents an HTTP log entry that captures detailed information about an HTTP request, response, and any associated exceptions.
/// </summary>
public sealed class HttpLog : Entity
{
/// <summary>
/// Gets or sets the HTTP status code of the response.
/// </summary>
public int StatusCode { get; set; }

/// <summary>
/// Gets or sets the duration of the request processing in milliseconds.
/// </summary>
public long Duration { get; set; }

/// <summary>
/// Gets or sets the name of the assembly handling the request.
/// </summary>
public string AssemblyName { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the version of the assembly handling the request.
/// </summary>
public string AssemblyVersion { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the process ID of the application handling the request.
/// </summary>
public int ProcessId { get; set; }

/// <summary>
/// Gets or sets the name of the process handling the request.
/// </summary>
public string ProcessName { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the thread ID where the request is being handled.
/// </summary>
public int ThreadId { get; set; }

/// <summary>
/// Gets or sets the memory usage in bytes at the time of logging.
/// </summary>
public long MemoryUsage { get; set; }

/// <summary>
/// Gets or sets the name of the machine handling the request.
/// </summary>
public string MachineName { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the name of the environment (e.g., Development, Production).
/// </summary>
public string EnvironmentName { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the name of the user running the environment.
/// </summary>
public string EnvironmentUserName { get; set; } = string.Empty;

/// <summary>
/// Gets or sets a value indicating whether the request is authenticated.
/// </summary>
public bool IsAuthenticated { get; set; }

/// <summary>
/// Gets or sets the language of the request.
/// </summary>
public string Language { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the session ID associated with the request.
/// </summary>
public string SessionId { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the start date and time of the request.
/// </summary>
public DateTime StartDate { get; set; }

/// <summary>
/// Gets or sets the trace ID for distributed tracing.
/// </summary>
public string TraceId { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the unique ID for the request.
/// </summary>
public string UniqueId { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the user ID associated with the request.
/// </summary>
public Guid UserId { get; set; }

/// <summary>
/// Gets or sets the IP address of the user making the request.
/// </summary>
public string UserIp { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the username of the user making the request.
/// </summary>
public string Username { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the API token key used for authentication.
/// </summary>
public string ApiTokenKey { get; set; } = string.Empty;

#region Request

/// <summary>
/// Gets or sets the URL of the request.
/// </summary>
public string ReqUrl { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the protocol of the request (e.g., HTTP/1.1).
/// </summary>
public string ReqProtocol { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the HTTP method of the request (e.g., GET, POST).
/// </summary>
public string ReqMethod { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the scheme of the request (e.g., http, https).
/// </summary>
public string ReqScheme { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the base path of the request.
/// </summary>
public string ReqPathBase { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the path of the request.
/// </summary>
public string ReqPath { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the query string of the request.
/// </summary>
public string QueryString { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the content type of the request.
/// </summary>
public string ReqContentType { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the content length of the request body.
/// </summary>
public long? ReqContentLength { get; set; }

/// <summary>
/// Gets or sets the body of the request.
/// </summary>
public string? ReqBody { get; set; }

/// <summary>
/// Gets or sets the headers of the request.
/// </summary>
public Dictionary<string, string> ReqHeaders { get; set; } = new();

#endregion

#region Response

/// <summary>
/// Gets or sets the content type of the response.
/// </summary>
public string ResContentType { get; set; } = string.Empty;

/// <summary>
/// Gets or sets the content length of the response body.
/// </summary>
public long? ResContentLength { get; set; }

/// <summary>
/// Gets or sets the body of the response.
/// </summary>
public string? ResBody { get; set; }

/// <summary>
/// Gets or sets the headers of the response.
/// </summary>
public Dictionary<string, string> ResHeaders { get; set; } = new();

#endregion

#region Exception

/// <summary>
/// Gets or sets additional data associated with an exception.
/// </summary>
public IDictionary? ExData { get; set; }

/// <summary>
/// Gets or sets the help link for the exception.
/// </summary>
public string? ExHelpLink { get; set; }

/// <summary>
/// Gets or sets the HResult code for the exception.
/// </summary>
public int? ExHResult { get; set; }

/// <summary>
/// Gets or sets the message of the exception.
/// </summary>
public string? ExMessage { get; set; }

/// <summary>
/// Gets or sets the source of the exception.
/// </summary>
public string? ExSource { get; set; }

/// <summary>
/// Gets or sets the stack trace of the exception.
/// </summary>
public string? ExStackTrace { get; set; }

#endregion
}
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
namespace FluentCMS;
namespace FluentCMS.Entities;

// TODO: move this to a project which include server side shared resources which will be accessible in any other server projects

Expand Down Expand Up @@ -62,4 +62,10 @@ public interface IApiExecutionContext
/// Returns an empty string if the user is not authenticated.
/// </summary>
string Username { get; }


/// <summary>
/// Gets the API token key associated with the current request, which can be used for
/// </summary>
string ApiTokenKey { get; }
}
Loading