Support for custom filters

[vukispasic]

Hello,

We are considering to use Gridify in our Web API project instead Sieve but we are missing possibility to configure custom filters.
For example, Sieve library supports registration of custom filters as following:

public IQueryable<Post> IsNew(IQueryable<Post> source, string op, string[] values) // The method is given the {Operator} & {Value}
{
    var result = source.Where(p => p.LikeCount < 100 &&
                                    p.CommentCount < 5);

    return result; // Must return modified IQueryable<TEntity>
}

public IQueryable<T> DateRange<T>(IQueryable<T> source, string op, string[] values) where T : BaseEntity // Generic functions are allowed too
{
    //limit range to 1 year
    var dateFrom = DateTimeOffset.Parse(values[0]);
    var dateTo = dateFrom.AddYears(1);
    var result = source.Where(c => c.DateCreated >= dateFrom && c.DateCreated < dateTo);
    return result;
}

Basically idea is to expose some custom filters for API consumers so they can get required data in simpler way.

Is there a possibility to generate this kind of filters with Gridify?

Second possibility is support to validate filter values from query. For example we want to validate if consumer query is valid in a way that if one filter parameter is used then other is required. Let me explain in more details:

We have a model:

    public class Appointment
    {
        public Guid PatientId { get; set; }
        public Guid PhysicianId { get; set; }
        public DateTimeOffset DateTime { get; set; }
    }

In case filter query contains PatientId (ex. filter=PatientId=3F5CF33E-FB79-4DF9-BE55-B1AA008F2146 ) then DateTime filter is not mandatory, but if query contains ResourceId then DateTime range is mandatory and can't be longer then 1 month (ex. filter=PhysicianId=2041f0a7-edc8-4ad2-9132-8025cb0a0461,DateTime>=2024-07-01,DateTime<2024-08-01 )

We could manually parse filter query string and implement validation logic, but we would like to know is there a neater way with Gridify?

Thank you for your answer

[alirezanet]

Hi @vukispasic,

Is there a possibility to generate this kind of filters with Gridify?

With Gridify, custom filters are rarely necessary as it supports nearly all logical scenarios right out of the box, so no Gridify doesn't have this feature yet.

We could manually parse filter query string and implement validation logic, but we would like to know is there a neater way with Gridify?

The latest versions of Gridify include a SyntaxTree feature, which allows you to view your queries in a tree structure. This can be useful for creating custom validation logic. I've experimented with your example and developed the following code. While it may not be the best implementation, it should provide a clear concept of how to navigate through the tree.

Validate("PatientId=3F5CF33E-FB79-4DF9-BE55-B1AA008F2146");   // it is valid
Validate("PhysicianId=2041f0a7-edc8-4ad2-9132-8025cb0a0461,DateTime>=2024-07-01,DateTime<2024-08-01");  // it is valid
Validate("PhysicianId=2041f0a7-edc8-4ad2-9132-8025cb0a0461");   // throws Invalid query, DateTime is not specified
Validate("PhysicianId=2041f0a7-edc8-4ad2-9132-8025cb0a0461,DateTime>=2023-07-01,DateTime<2024-08-01"); // throws Invalid query, DateTime range exceeds one month


void Validate(string filter)
{
	var tree = SyntaxTree.Parse(filter); // access to the whole query Syntax Tree
	var fields = GetFields(tree.Root).ToList();
	
	// check whether query has PhysicianId
	if (fields.Any(f => f.FieldToken.Text.Equals(nameof(Appointment.PhysicianId), StringComparison.InvariantCultureIgnoreCase)))
	{
		var dateTimeFields = fields.Where(f => f.FieldToken.Text.Equals(nameof(Appointment.DateTime), StringComparison.InvariantCultureIgnoreCase));
		
		// make sure we only have two dateTime conditions
		if (dateTimeFields.Count() != 2)
		{
			throw new GridifyQueryException("Invalid query, DateTime is not specified");
		}
		
		var dateTimes = GetDateTimes(tree.Root).ToArray();

		// make sure dates are less than 1 month
		if (dateTimes[0].Date > dateTimes[1].Date)
		{
			(dateTimes[0], dateTimes[1]) = (dateTimes[1], dateTimes[0]);
        }
		var start = dateTimes[0];
		var end = dateTimes[1];
		if ((end - start).TotalDays > 31)
		{
			throw new GridifyQueryException("Invalid query, DateTime range exceeds one month");
		}
		
	}
}

IEnumerable<FieldExpressionSyntax> GetFields(ISyntaxNode parentNode)
{
	foreach (var node in parentNode.GetChildren())
	{
		if (node.Kind == SyntaxKind.FieldExpression)
		{
			yield return (node as FieldExpressionSyntax)!;
		}
 		
		foreach (var subNode in GetFields(node))
		{
			yield return subNode;
		}
	}
}

IEnumerable<DateTime> GetDateTimes(ISyntaxNode parentNode)
{
	foreach (var node in parentNode.GetChildren().OfType<BinaryExpressionSyntax>())
	{
		if (node.Left is FieldExpressionSyntax fExp && fExp.FieldToken.Text.Equals(nameof(Appointment.DateTime), StringComparison.InvariantCultureIgnoreCase))
		{
			var value = node.Right as ValueExpressionSyntax;
			yield return DateTime.Parse(value!.ValueToken.Text);
		}
		
		foreach (var subNode in GetDateTimes(node))
		{
			yield return subNode;
		}
	}
}


public class Appointment
{
	public Guid PatientId { get; set; }
	public Guid PhysicianId { get; set; }
	public DateTimeOffset DateTime { get; set; }
}

[alirezanet]

Regarding the Custom Filter feature: if a valid use case arises, I'll certainly consider adding it to our roadmap. However, I'm not yet convinced of its utility. Since you have experience with this feature in Sieve, please don't hesitate to submit a feature request issue with an example to illustrate its potential benefits.

[KaJIbI4]

This is a good example, but due to the fact that the node types of the tree are internal (BinaryExpressionSyntax, Value Expression Syntax, FieldExpressionSyntax), you have to use ugly code to work with the syntax tree. Or maybe I misunderstood something?

[alirezanet]

in the next stable version, it will be public and we have already released a preview version v2.15.0-preview3, you can try to use the preview version until I release a new stable one.

[vukispasic]

Hello @alirezanet

Thank you for idea and code examples. We managed to make use of SyntaxTree and develop validation logic based on SyntaxTree to control filter combinations (properties in filter that are allowed/must be combined in same query) as well as allowed filter values like date time maximum range depending on filter