Written by Chris Fidao on Feb 17, 2015 in

The New Search Engine

Have you ever searched for a request, but just couldn't find it? You knew the words used in the request, but no matter what variation you tried, your search came up dry! We certainly did, and so we set out to improve search in HelpSpot. Version 4 introduced searching with the Sphinx Search engine, which vastly improves the searching engine used to find content within requests, forums and knowledge books. Searching in HelpSpot 3 used the database, and so it suffered the limitations of the supported databases. While you may guess that databases are great for searching, the opposite is often the case for text-based searching!

The Databases

MySQL, the most commonly used of the three supported databases, allowed for "FULL TEXT" searching, which required the use of a storage engine called MyISAM. This storage engine had the drawback of locking an entire table when records were being added or updated. The result of this was slowness in the system. For example, users may have had to wait for a filter to finish finding requests because tables were locked while other people were responding to requests, or the system was reading new emails. In a busy system, this locking created performance issues. The other two supported databases (PostgreSQL and SqlServer) didn't have this limitation, but were also sub-par in how they performed text-based searches. When HelpSpot was created, we had to suffer through this, as there were not viable alternatives. Luckily for us, that's no longer the case!

Sphinx Search

With the introduction of the Sphinx Search engine, HelpSpot can now quickly and efficiently search through all of HelpSpot's data. This is a two-pronged improvement:

Sphinx Search has much, much, MUCH more advanced ways of indexing and searching through text.
HelpSpot is free to use better storage engines (InnoDB in MySQL) to improve performance and reduce issues while writing new data to the database.

In addition to being just plain faster, Sphinx can also make use of numerous tools for indexing and searching text, including relevancy ranking algorithms, morphology algorithms, prefix/infix support and much more. HelpSpot can now use the InnoDB storage engine, which does not lock entire tables and fully supports database transactions, finally using the full power of the most popular database.

Day-to-Day Usage

Sphinx Search is used to search all requests, forums and knowledge books anywhere HelpSpot searches, such as:

Within HelpSpot sidebar search and advanced searched
The API
The customer Portal
Text searches within automation rules, triggers and mail rules.

This includes searching through any custom field that includes text (Predefined List, Drill Down List, Text Field, Large Text Field, Regular Expression Field and AJAX Field). For day-to-day usage, HelpSpot will work the same way as before, but the text-based search results will now be ranked by relevance, rather than date. Furthermore, there's less restriction in how far back in time search results will go.

Sidebar Search

Previously, the sidebar search only searched customer data (email, name, customer ID or request ID). In HelpSpot 4, you can now select the "Search History" checkbox, which will search all request data!

Getting Started with Sphinx Search

Windows users get Sphinx Search when they update to version 4 using our provided installer. Linux users can use our guides to installing & configuring Sphinx Search on Debian/Ubuntu servers or RedHat/CentOS servers.