[BUG] Specific search terms are throwing a SolrInternalServerErrorException

[stephangonder]

Describe the bug
If a user is typing some specific search terms in the solr query input, the page responses with an internal server error with type #1505989897 ApacheSolrForTypo3\Solr\System\Solr\SolrInternalServerErrorException.

The error gets thrown in apache-solr-for-typo3/solr/Classes/Search.php line 118

To Reproduce
Type the following search term into the search input field:

gemäß § 29 Abs. 1 EnWG i. V. m. § 6b Abs. 6 Sätze 1 und 2, Abs. 1 Satz 1 EnWG

In production mode, the common "Oops, an error occurred!" message appears.

Expected behavior
The solr service is responding with valid search results / no search results.

Used versions (please complete the following information):

• TYPO3 Version: 12.4.25
• Browser: Chrome / Safari / Firefox
• EXT:solr Version: 12.0.5
• Used Apache Solr Version: 9.6
• PHP Version: 8.2.26
• MySQL Version: 8.0.40

[derhansen]

I can reproduce this behavior. Testing the search term directly in the SOLR ui results in the following error:

{
  "responseHeader":{
    "status":500,
    "QTime":115,
    "params":{
      "q":"gemäß § 29 Abs. 1 EnWG i. V. m. § 6b Abs. 6 Sätze 1 und 2, Abs. 1 Satz 1 EnWG",
      "indent":"true",
      "q.op":"OR",
      "useParams":"",
      "_":"1737047567648"}},
  "error":{
    "msg":"Query contains too many nested clauses; maxClauseCount is set to 1024",
    "trace":"org.apache.lucene.search.IndexSearcher$TooManyNestedClauses: Query contains too many nested clauses; maxClauseCount is set to 1024\n\tat org.apache.lucene.search.IndexSearcher$3.consumeTerms(IndexSearcher.java:840)\n\tat org.apache.lucene.search.PhraseQuery.visit(PhraseQuery.java:307)\n\tat org.apache.lucene.search.BooleanQuery.visit(BooleanQuery.java:583)\n\tat org.apache.lucene.search.BoostQuery.visit(BoostQuery.java:106)\n\tat org.apache.lucene.search.BooleanQuery.visit(BooleanQuery.java:583)\n\tat org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:800)\n\tat org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:806)\n\tat org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:553)\n\tat org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:275)\n\tat org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1890)\n\tat org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1707)\n\tat org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:710)\n\tat org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1702)\n\tat org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:423)\n\tat org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:467)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:226)\n\tat org.apache.solr.core.SolrCore.execute(SolrCore.java:2884)\n\tat org.apache.solr.servlet.HttpSolrCall.executeCoreRequest(HttpSolrCall.java:875)\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:561)\n\tat org.apache.solr.servlet.SolrDispatchFilter.dispatch(SolrDispatchFilter.java:262)\n\tat org.apache.solr.servlet.SolrDispatchFilter.lambda$doFilter$0(SolrDispatchFilter.java:219)\n\tat org.apache.solr.servlet.ServletUtils.traceHttpRequestExecution2(ServletUtils.java:249)\n\tat org.apache.solr.servlet.ServletUtils.rateLimitRequest(ServletUtils.java:215)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:213)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)\n\tat org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:210)\n\tat org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:598)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1580)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1384)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1553)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1306)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:149)\n\tat org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:228)\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:141)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)\n\tat org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:301)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)\n\tat org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:822)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:563)\n\tat org.eclipse.jetty.server.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1598)\n\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:753)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:501)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:287)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)\n\tat org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)\n\tat org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421)\n\tat org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390)\n\tat org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277)\n\tat org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:199)\n\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)\n\tat java.base/java.lang.Thread.run(Unknown Source)\n",
    "code":500}}

So the reason for this problem is a Solr setting.

In solr.xml config you can increase the limit. For me, adding <int name="maxBooleanClauses">2048</int> fixed the problem at least in Solr UI. TYPO3 does however still result in a 503 error. Increasing the value to <int name="maxBooleanClauses">10240</int> then finally also fixed the issue for the TYPO3 search.

[dkd-kaehm]

Wow, @derhansen thanks for analysis and solution.
IMHO the users will try a bigger queries, which will lead to the same trouble and we can not increase the value infinitely.
So we want another solution for that, like limiting the space chars in the query and react with proper user instruction or switch some Lucene query features off on those queries...