Results stream not updated after network access restored

[dhelsper]

I implemented the code in the article, Algolia getting started with Flutter Helper and have run into an issue. When putting the Android emulator into airplane mode and scrolling down, the PageController returns an error as expected. For PagedChildBuilderDelegate I have defined a newPageErrorIndicatorBuilder handler. I allow the user to refresh and call using _pagingController.retryLastFailedRequest() to retry the query. The page request listener is invoke and executes a applyState((state) => state.copyWith(page: pageKey)).

For some reason the response stream never updates, so my UI is hung displaying the circular progress indicator. Prior to executing the refresh, I turned off airplane mode so the network connection was restored. I would have expected the response stream to update with the next page of results, yet nothing happens. I also would have expected some sort of stream update even if the phone was still in airplane mode.

Am I missing something here? I have debugged this and the code execution path is always the same, minus the stream never updates after it encounters the network error for the first time.

Going into airplane mode prior to the first query does work, i.e., firstPageErrorIndicatorBuilder. When restoring the network and refreshing the query causes the response stream to update and the search results are retrieved.

Thanks.

[aallam]

Hello @dhelsper, thank you for your feedback.
This issue should be fixed with release 0.2.2, could you please give it a try?

[dhelsper]

Hello @dhelsper, thank you for your feedback. This issue should be fixed with release 0.2.2, could you please give it a try?

Thanks for the quick turn around. Greatly appreciated. I'll try and get you some feedback before the end of this weekend.

[dhelsper]

Hello @dhelsper, thank you for your feedback.

This issue should be fixed with release 0.2.2, could you please give it a try?

So I just blew through my search query limit for the month and made just a handful of search queries, but somehow made 1,000's of multi queries. The only change in my code was doing a pub get of your latest release. Prior to your update I had made somewhere around 300 queries. Not sure what is up.

I'm going to back off your latest release and see what happens, assuming I have any queries left for the month. 😀 Thx.

I'll let you know if I see the same behavior or it reverts back to what I have seen during my previous development and testing.

[dhelsper]

Hello @dhelsper, thank you for your feedback.
This issue should be fixed with release 0.2.2, could you please give it a try?

So I just blew through my search query limit for the month and made just a handful of search queries, but somehow made 1,000's of multi queries. The only change in my code was doing a pub get of your latest release. Prior to your update I had made somewhere around 300 queries. Not sure what is up.

I'm going to back off your latest release and see what happens, assuming I have any queries left for the month. 😀 Thx.

I'll let you know if I see the same behavior or it reverts back to what I have seen during my previous development and testing.

So, I'm blocked for the rest of the month, unless I switch to a paying account. Let me know if you have any insight into the issue and if I can get unblocked prior to next month. Thanks.

[aallam]

Hello @dhelsper, sorry to hear that! could you please provide us with the following:

1. Code snippets
2. Your application ID by email to [email protected]

Could you also enable logging and tell us what you see? you can do it with something like this in your main:

if (kDebugMode) {
  Logger.root.level = Level.ALL;
  Logger.root.onRecord.listen((record) =>
      print('${record.level.name}: ${record.time}: ${record.message}'));
}

[dhelsper]

Here's the class I created for accessing the search.

class SearchAPIImplementation implements SearchAPI {

// Algolia Search Engine instance
HitsSearcher? _searchEngineGroups;
// Fields to use as filters
final isGroupActive = FilterGroupID(GroupFieldNames.isGroupActive);
final isGroupLocked = FilterGroupID(GroupFieldNames.isGroupLocked);
final isClosed = FilterGroupID(GroupFieldNames.isClosed);
final groupMembers = FilterGroupID(GroupFieldNames.groupMembers);
// Filter State
FilterState? _filterState;

SearchAPIImplementation() {
Debug.log(className: runtimeType, methodName: "SearchAPIImplementation", logging: LoggingLevel.trace, message: "Entering");

try {
  _searchEngineGroups = HitsSearcher(applicationID: "XXXXXXXXXX", apiKey: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", indexName: "groupKeywords");
} catch (err) {
  Debug.log(className: runtimeType, methodName: "createSearchEngine", logging: LoggingLevel.error, message: err.toString());
  rethrow;
}

Debug.log(className: runtimeType, methodName: "SearchAPIImplementation", logging: LoggingLevel.trace, message: "Exiting");

}

@OverRide
HitsSearcher? getSearchEngineInstance() {

return _searchEngineGroups;

}

@OverRide
void dispose() {
Debug.log(className: runtimeType, methodName: "disposeSearchEngine", logging: LoggingLevel.trace, message: "Entering");

try {
  _searchEngineGroups?.dispose();
  _searchEngineGroups = null;
  _filterState?.dispose();
  _filterState = null;
} catch (err) {
  Debug.log(className: runtimeType, methodName: "disposeSearchEngine", logging: LoggingLevel.error, message: err.toString());
  rethrow;
}

Debug.log(className: runtimeType, methodName: "disposeSearchEngine", logging: LoggingLevel.trace, message: "Exiting");

}

@OverRide
void query({required String keywords, required List preferences}) {
Debug.log(className: runtimeType, methodName: "query", logging: LoggingLevel.trace, message: "Entering keywords $keywords");

try {
  // Build preferences filter array for facet filters
  List<String> facetFilterArray = [];

  for (var preference in preferences) {
    facetFilterArray.add("${GroupFieldNames.preferenceDescriptions}:$preference");
  }

  _searchEngineGroups?.applyState((state) => state.copyWith(page: 0, query: keywords, facetFilters: facetFilterArray));
} catch (err) {
  Debug.log(className: runtimeType, methodName: "query", logging: LoggingLevel.error, message: err.toString());
  rethrow;
}

Debug.log(className: runtimeType, methodName: "query", logging: LoggingLevel.trace, message: "Exiting");

}

@OverRide
Stream searchMetaDataStream() {
Debug.log(className: runtimeType, methodName: "searchMetaData", logging: LoggingLevel.trace, message: "Entering");

 Debug.log(className: runtimeType, methodName: "searchMetaData", logging: LoggingLevel.trace, message: "Exiting");

try {
  return _searchEngineGroups!.responses.map((response) => SearchMetaData.fromResponse(response));
} catch (err) {
  Debug.log(className: runtimeType, methodName: "searchMetaData", logging: LoggingLevel.error, message: err.toString());
  rethrow;
}

}

@OverRide
Stream searchDataStream() {
Debug.log(className: runtimeType, methodName: "searchDataStream", logging: LoggingLevel.trace, message: "Entering");

 Debug.log(className: runtimeType, methodName: "searchDataStream", logging: LoggingLevel.trace, message: "Exiting");

try {
  return _searchEngineGroups!.responses.map((response) => GroupSearchHitsPage.fromResponse(response));
} catch (err) {
  Debug.log(className: runtimeType, methodName: "searchDataStream", logging: LoggingLevel.error, message: err.toString());
  rethrow;
}

}

@OverRide
void setFilterCriteria() async {

try {
  // Create the search filters
  if (_filterState == null) {
    _filterState = FilterState();
    // Only want to see active groups, that are not locked, i.e., open for people to join,
    // and the limit of members has not been reached, i.e., not closed.
    _filterState!.add(isGroupActive, [Filter.facet(GroupFieldNames.isGroupActive, true)]);
    _filterState!.add(isGroupLocked, [Filter.facet(GroupFieldNames.isGroupLocked, false)]);
    _filterState!.add(isClosed, [Filter.facet(GroupFieldNames.isClosed, false)]);
    // Filter out any groups where the current user is already a member, just using membersIsCoModertor because it is the first field
    // Our filter is checking for the fact that the user entry does not exist, so the user is not currently part of the group.
    _filterState!.add(groupMembers,[Filter.facet("${GroupFieldNames.groupMembers}.${havenCurrentUser!.id}.${GroupFieldNames.membersIsCoModerator}", false, isNegated: true)]);
    // Associate with the search engine
    if (_searchEngineGroups != null) {
      _searchEngineGroups!.connectFilterState(_filterState!);
    }
  }
} catch (err) {
  Debug.log(className: runtimeType, methodName: "searchDataGroups", logging: LoggingLevel.error, message: err.toString());
}

}
}

Here's the code from my View to setup the search

_searchAPI = serviceLocator<SearchAPI>();
_searchAPI.setFilterCriteria();

_searchAPI.searchDataStream().listen(((page, {cancelOnError = false}) {
  
  if (page.pageKey == 0) {
    _pagingController.refresh();
  }

  if (page.nextPageKey == null) {
    _pagingController.appendLastPage(page.items);
  } else {
    _pagingController.appendPage(page.items, page.nextPageKey);
  }
}))
.onError((handleError) {
   Debug.log(className: runtimeType, methodName: "_searchAPI.searchDataStream().listen", logging: LoggingLevel.error, message: handleError.toString());
  _pagingController.error = handleError;
   // Pop the Circular progress indicator
   //Navigator.of(context).pop();
   // Inform the user something went wrong
    if (!mounted) return;
    SnackBar sb = const SnackBar(
      content: Text("Unrecoverable error, please contact support", textAlign: TextAlign.center, style: TextStyle(color: Colors.yellow)),
      duration: Duration(seconds: 3),
    );
    ScaffoldMessenger.of(navigatorKey.currentContext!).showSnackBar(sb);       
});

_pagingController.addPageRequestListener((pageKey) {
  try {
    _searchAPI.getSearchEngineInstance()?.applyState((state) => state.copyWith(page: pageKey));
  } catch (err) {
    Debug.log(className: runtimeType, methodName: "_pagingController.addPageRequestListener", logging: LoggingLevel.error, message: err.toString());
  }
});

Here's the code to invoke the search via a button

for (var keyword in _selectedCategoryKeywords) {
  searchKeywords.write("$keyword ");
}

// Requery the group search
try {
  _searchAPI.query(keywords: searchKeywords.toString(), preferences: _selectedPreferences);
} catch (err) {
   Debug.log(className: runtimeType, methodName: "_invokeGroupSearch", logging: LoggingLevel.error, message: err.toString());
   // Pop the Circular progress indicator
   Navigator.of(context).pop();
   // Inform the user something went wrong
    if (!mounted) return;
    SnackBar sb = const SnackBar(
      content: Text("Unrecoverable error, please contact support", textAlign: TextAlign.center, style: TextStyle(color: Colors.yellow)),
      duration: Duration(seconds: 3),
    );
    ScaffoldMessenger.of(navigatorKey.currentContext!).showSnackBar(sb);       
}

I can't really do much with logging right now as I am currently blocked from accessing my project and index. Sorry about that. If you guys can enable the project I'll be happy to enable logging and get you more feedback.

Thanks.

[aallam]

Hi again @dhelsper, could you try again now please ? thanks!

[dhelsper]

traceresults.docx

Here you go. I just ran the code and after clicking on search to invokde query it seems to go into a loop. Not sure if that's something I'm doing wrong or related to the search package. Thanks.

[dhelsper]

I just did a bit more testing and it looks like the issue arrises when there are no query results to retrieve. When I have query results the query executes and I don't see any looping. When there are no results to return based on the filter values then I just see the continuous looping of the disjunctive search. I guess I never saw it because I always had results to return, until recently. This is an issue in both versions of the package, i.e., 0.2.1 and 0.2.2.

[aallam]

Thank you @dhelsper for your precious feedback, we will take a look and get back to you!

[aallam]

Hi @dhelsper, I've taken a look at your code, and here is what I've noticed:

FilterState is best suited for dynamic filters, which is exactly your case here.
So instead of the following:

_searchEngineGroups?.applyState((state) => state.copyWith(page: 0, query: keywords, S: facetFilterArray))

Use HitsState to update your query, page number..etc, and use FilterState to update filters (facets in your case) instead of using SearchState.facetFilters.

Please give this a try, and tell us if it solved your issue.

[dhelsper]

I did originally try that approach and what I found is that I was not able to update the facet filters after creating FilterState and setting it for the first time. I set the facet filters but they seemed to be ignored on subsequent update attempts. Perhaps I was not managing them correctly. I even tried creating a new FilterState object every time I changed the filter criteria, bit I could never get it to work. What you see in SetFilterCriteria is all I could ever get to work. Any subsequent updates to the filter criteria seem to have simply been ignored. I hope that makes sense. What I coded was a hack to work around the issue. In debugging the issue, I did notice that the filter arrays in the FilterState object did say they were immutable. Not sure if that is the issue or I just misinterpreted what that meant.

If you could provide me a working snippet of code I will be happy to give that a try. Thanks.

[dhelsper]

Any update? Thanks.

[dhelsper]

I made the following update and this seems to have solved my issues with using the filter facets, i.e., not being updated correctly after the initial creation and setting of facets, and the search query looping. I do understand that I was not using the FilterState in the most effective way, but I would argue that going into a hard loop doing queries is a pretty major bug, especially based on how your billing works.

  _filterState?.modify((filters) async {
    return filters.clear([preferencesFilter]).add(preferencesFilter, [for (var preference in preferences) Filter.facet(GroupFieldNames.preferenceDescriptions, preference)]);
  },);

_searchEngineGroups?.applyState((state) => state.copyWith(page: 0, query: keywords));