New Features in WoltLab Suite 5.5: Search Overhaul

Photo by Hadija Saidi on Unsplash

Search Form and Search Results Combined in One Page

The previous search was strictly separated between the input using a dedicated form and the display of results on a separate page. Nowadays, this historical separation no longer offers any advantages and does not provide users with the interactivity they are used to in our software. This was also evident when a user wanted to modify the search parameters, which redirected them back to a form that was relatively extensive and difficult to keep track of.

The overhauled search now combines these two pages into a comfortable and user-friendly interface. It is now much easier to refine an existing search, and the newfound interactivity delivers results almost seamlessly, as one would expect from typical search engines. Taking advantage of clever categorization and interface elements, the variety of parameters for narrowing down results is preserved without overwhelming the user.

Search form and search results for search query "database"

Dynamic Loading of Search Results

The search results are queried from the server in the background on submit and are displayed seamlessly below the search mask. Navigation between the results pages also takes place directly on the page and allows switching between results at any time. Access to the search mask remains unrestricted.

A crucial disadvantage of the previous solution for the search results was the insufficient sharing of the results in the form of a link. This was one of the central points of criticism, which we have taken care of with this version: All parameters of a search are now part of the URL. It is thus possible to share a specific search with another user by passing on the link, the link remains valid indefinitely.

The search results are computed independently for each caller: When the link is passed on, only the values in the search mask are preserved, but not the results themselves. This ensures that the results do not contain any contents that are inaccessible to the other user.

Example search query URL

Flood Control for Search Function

The search function naturally causes a significant load on the server and thus becomes a potential point of attack to overload a website with many search queries. There was previously the option to activate the captcha function for using the search function in order to mitigate this attack vector. However, this was only available to people who were not logged in, and at the same time also made the search less convenient to use, negatively impacting the user experience by a significant degree.

We have therefore decided to remove the use of the captcha function in the search function. Instead, with WoltLab Suite 5.5, we introduce a flood control for the search function. The flood control keeps track on how many times a caller has used the search function within a certain time and automatically blocks further calls if an unusually high number of requests has been detected.

New Search Options and Contextual Display of Results

The search parameters for forum posts have been refined and more options are now available to filter the results. It is possible to search specifically for the respective starting post of a thread or for posts marked as “official”. Furthermore, there is an additional filter to limit the search to threads that contain a most helpful answer to be able to search specifically for solutions to a problem.

Filter options for search in forum posts

With WoltLab Suite 5.5, we have improved the flexibility of the search results to better display different content. True to the motto “a picture is worth a thousand words”, the search function displays the results directly as images when searching in the gallery.

Search results from the gallery

Overhauled Filebase Search Results

The filebase lends itself, comparable to the Gallery, to a customized display to better present the contents, the pure display as text for the search results has proven to be rather impractical. When searching for files in the filebase, the familiar layout from the file listing is now used to present the hits in a familiar layout.

Search results from the filebase

Modernization of the Elasticsearch Integration

The search via Elasticsearch offers many advantages over the limited possibilities of the MySQL-based search and for this reason is included in all plans for our cloud hosting without additional costs. The integration was continuously maintained by us and adapted to newer versions of Elasticsearch, but our focus was primarily on compatibility with different versions. With WoltLab Suite 5.5, we have revised the technical basis of our integration in order to better benefit from the innovations of Elasticsearch and at the same time to incorporate experience gained in years of use.

Asynchronous Management of the Search Index

New content or changes to existing content were previously sent immediately to Elasticsearch for processing. In practice, this has resulted in increased loading time when submitting content if the servers are slow. In addition, connection problems, such as with Elasticsearch servers connected via the Internet, sometimes resulted in changes to the search index getting lost.

In order to increase the reliability of maintaining the search index, with WoltLab Suite 5.5 we rely on a local cache whose content is sent collectively to Elasticsearch every few minutes. This allows both more efficient processing via Elasticsearch‘s “bulk API” and significantly higher fault tolerance against connection problems.

Better Processing of Search Inputs

Comparable to common search engines, Elasticsearch also offers a rich syntax for formulating queries to the search index. As a user, this gives you a variety of options, for example, to devalue or completely exclude certain terms in the hits. The interpretation of this search input was previously handled by our integration and passed to Elasticsearch in a standardized format. With the new version, this part is outsourced directly to Elasticsearch and has proven in all tests to yield even better results.