Fast and accurate search feature is a very important thing. It makes it easy for users to find what they want, which leads to more loyal users and more profit. However, implementing sophisticated search is not an easy thing. You need to handle many things such as indexing, handlling typo, ordering ranking and much more. Fortunately, there are some hosted search services that can handle all those problems.
One of the most popular hosted search engine service is Algolia. If you have registered for an Algolia account, you can start to store your data on their datacenters and configure the settings to make it more accurate and powerful. They also have additonal features such as A/B testing and analytics. In this tutorial, I'm going to show you how to use Algolia Search from your Node.js application.
Algolia Concepts
Indexing
Like other search engines, it supports indexing. First you need to put your data to Algolia server. You can do it by either uploading via dashboard or programatically via API. They encourage you to insert records in batch with a recommended size of 10MB per batch. Algolia uses ObjectID field to identify unique object, you can either supply it or let Algolia set ObjectID automatically.
Searching
Not only returning the results of a query, Algolia also has the following features:
- Pagination. It supports pagination and you can set the number of items per page
- Highlighting. It can highlight part of a record that makes it match to the query
- Snippeting. For fields with long value, you can choose to display the snippet only instead of the full text
- Filtering. It’s used to limit the results returned to a specific subset of your data. Filtering is based on the attributes: For non-numeric attributes, they need to be set up as categories, referred to as facets. You need to at them as attributesForFaceting. For numeric attributes, you don't need to at them as attributesForFaceting. It also supports tags, added as _tags element. You can filter tags using tagFilters parameter
- Faceting. Facets are used to create categories on a select group of attributes. For the seleted attributes, you can get the list of all posible values and return contextual values and counts. It also enables you to compute a facet count for each value and search within the values of this attribute.
- Geo Search. It's used to filter and sort results based on geo-locations
- Query Expansion. It's about loosening query constraints when no results are found. You can add the words to be removed if no results found.
Ranking
Algolia's ranking ranking strategy is designed to handle challanging search problem by supporting the following features.
- Custom Ranking. You can specify ranking order for each attributes, either descending or ascending.
- Searchable Attributes. It's possibble to set which attributes will be used for search. It means the presence of a keyword in other attributes won't be count.
- Sorting. It allows you to use multiple sorting strategies for an index
- Distinct. You can also specify attribute for distinct.
- Personalization. This feature allows you to use different ranking strategies for different users.
Textual Reference
Textual reference is about defining attributes and text-based rules for better search results.
- Handling Natural Language. It supports text normalization adding language specificity, and language-specific configuration.
- Typo-tolerance. It can handle typo and you can set minimum number of characters for 1 typo and 2 typos.
- Plurals. It’s also possible to ignore plurals, so that the singular form and plural form will be consisted as the same word.
- Optional Words. You can also specify which words in the query are optional. It makes a record that doesn't contain the optional words can be considered as a match.
- Stop Words. You may want to remove stop words such as “the”, “and”, “at”, and “as”. By default it's not removed. But optionally you can remove it as well as specifying the stop words of which language should be removed.
- Prefix. This features enable users to get the results instantly before he's completing to type a keyword. By default it uses prefixLast, where only the last word in a query is treated as prefix. Optionally you an use prefixAll, which treats all query words as prefixes. If you don't want to use this feature, use prefixNone instead
- Synonyms. Some words have similar or same meaning with others and you want those word to be treated as the same with the synonyms. In order to do so, you have to supply your own list of synyonyms.
Query Rules
It allows you to change how Algolia would treat specific search terms. For example, you can promote results, apply segmentation, perform dynamic filtering, and much more.
Using Algolia in Node.js
Now we go to the main topic of this tutorial. First you need to register for an Algolia account. We use algoliasearch library - it's the official library by the Algolia team. I also give you some basic usages examples.
1. Register for an Algolia account
To register, open the Algolia pricing page and select the package you want. For the paid plans, you can start with a free trial. Alternatively, you can choose the community plan which is forever free, but with a lot of limitations. During registration, you'll need to enter your basic information, choose the datacenter and tell what your project is about.
After finishing the registration, you need to get the credentials for your application. Open the API Keys menu and you'll be redirected to a page. There should be the application ID along with three keys consisting of search-only API key, admin API key and monitoring API key. Copy the value of application ID and the appropriate keys to .env. In this tutorial, since we're going to add data to Algolia as well as change the configuration, the admin API key is the most appropriate. However, if you have a search features that can be used by your visitors, using the search-only API key is a more secure option.
ALGOLIA_APPLICATION_ID=XXXXXXXXXX
ALGOLIA_SEARCH_ONLY_API_KEY_ID=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
ALGOLIA_ADMIN_API_KEY_ID=yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy2. Install Dependencies
In this tutorial, we're going to use algoliasearch node module. As the credentials are stored on .env, we need to read it using dotenv. We also use bluebird and lodash.
"algoliasearch": "~3.29.0",
"bluebird": "~3.5.1",
"dotenv": "~4.0.0"
"lodash": "~4.17.10",After that, run npm install.
3. Code
Now it's time to code. We create a new helper file for Algolia Search.
helpers/algolia-search.js
require('dotenv').config();
const _ = require('lodash');
const algoliasearch = require('algoliasearch');
const Bluebird = require('bluebird');
function AlgoliaClient() {
if (!(this instanceof AlgoliaClient)) {
return new AlgoliaClient();
}
this.algoliaSearch = algoliasearch(process.env.ALGOLIA_APPLICATION_ID, process.env.ALGOLIA_ADMIN_API_KEY_ID)
}
// Later add the prototype functions here
module.exports = AlogliaClient;And we use it in other file
example.js
const AlgoliaClient = require('./helpers/algolia-search');
const client = new AlgoliaClient();Insert/Update Objects
The following code is for inserting or updating objects. In order to update an existing object, the record must have the same ObjectID.
helpers/algolia-search.js
/**
* Insert/update records to Algolia
* @param {string} indexName - The indices name where you want to insert/update data
* @param {Array.example.js
client.addObjects('contacts', yourCollections)Configure
Configuring Algolia Search can be done via dashboard. In case you need to do it from your code, here is the example.
helpers/algolia-search.js
/**
*
* @param {string} indexName - The name of indices you want to configure.
* @param {Object} settings
* @param {number} settings.minWordSizefor1Typo - The minimum number of characters to accept one typo (default = 3).
* @param {number} settings.minWordSizefor2Typos - The minimum number of characters to accept two typos (default = 7).
* @param {number} settings.hitsPerPage - The number of hits per page (default = 10)
* @param {Array.<string>} settings.attributesToRetrieve - list of attributes to retrieve in results.
* @param {Array.<string>} settings.attributesToHighlight - List of attributes to highlight in results.
* @param {Array.<string>} settings.attributesToSnippet - List of attributes you want to display the snippet in results.
* Format is attributeName:numberOfWords
* For example ['address:8']
* @param {Array.<string>} settings.attributesToIndex - List of attributes you want to index
* @param {Array.<string>} settings.attributesForFaceting - List of attributes for faceting
* @param {string} settings.attributeForDistinct - Name of attribute used for distinct
* @param {Array.<string>} settings.ranking - Set the results order
* Default order is ["typo", "geo", "proximity", "attribute", "exact", "custom"]
* @param Array.<string>} settings.customRanking - Specify ranking order
* For example `"customRanking" => ["desc(followers)", "asc(firstname)"]`
* @param {string} settings.queryType - How the query words are interpreted. Options:
* - prefixLast (default): only the last word is interpreted as a prefix.
* - prefixAll: all query words are interpreted as prefixes,
* - prefixNone: no query word is interpreted as a prefix (not recommended).
* @param {string} settings.highlightPreTag - String inserted before highlighted parts
* @param {string{ settings.highlightPostTag - String inserted after highlighted parts
* @param {Array.} settings.optionalWords - List of words considered optional when found in the query.
*/
AlgoliaClient.prototype.configure = function (indexName, settings) {
const index = this.algoliaSearch.initIndex(indexName);
return index.setSettings(settings);
};example.js
client.configure('contacts', {
attributesToIndex: [
'firstname',
'lastname',
'company'
],
minWordSizefor1Typo: 5,
minWordSizefor2Typos: 8,
hitsPerPage: 5,
attributesToRetrieve: ['firstname', 'lastname', 'county', 'address'],
attributesToHighlight: ['county'],
attributesToSnippet: ['address:8'],
attributesForFaceting: ['city', 'state'],
attributeForDistinct: 'city',
ranking: ['typo', 'geo', 'proximity', 'attribute', 'exact', 'custom'], // Ranking order. Default is
customRanking: ['desc(population)', 'asc(name)'],
queryType: 'prefixLast',
highlightPreTag: '',
highlightPostTag: '',
optionalWords: [],
})Search
The following is an example of how to perform search.
helpers/algolia-search.js
/**
* Perform the search
* @param {string} indexName - The indices name where you want to search
* @param {string|Object} queries - The search settings
* @param {number} queries.page - The page number to retrieve.
* @param {number} queries.hitsPerPage - Number of results per page.
* @param {Array.|string} queries.attributesToRetrieve - List of attributes to retrieve, either comma separated string (without space) or array of strings.
* @param {Array.|string} queries.attributesToHighlight - List of attributes to highlight in resultse, either comma separated string (without space) or array of strings.
* @param {Array.|string} queries.attributesToSnippet - List of attributes you want to display the snippet in resultse, either comma separated string (without space) or array of strings.
* @param {number} queries.minWordSizefor1Typo - The minimum number of characters to accept one typo (default = 3).
* @param {number} queries.minWordSizefor2Typos - The minimum number of characters to accept two typos (default = 7).
* @param {number} queries.getRankingInfo - If set to 1, the result hits will contain ranking
* information in _rankingInfo attribute.
* @param {string} queries.aroundLatLng - Search for entries around a given latitude and longitude defined by 2 floats separated by comma
* For example, `47.316669,5.016670`
* At indexing, you should specify geoloc of an object with the _geoloc attribute
* (in the form {"_geoloc":{"lat":48.123456, "lng":2.123456}})
* @param queries.insideBoundingBox - Search entries inside a given area defined by 4 floats separated by comma
* For example `47.3165,4.9665,47.3424,5.0201`
* At indexing, you should specify geoloc of an object with the _geoloc attribute
* (in the form {"_geoloc":{"lat":48.123456, "lng":2.123456}})
* @param {string} queries.numericFilters - List of numeric filters you want to apply.
* @param {Array.|string} queries.tagFilters - Filter the query by a set of tags.
* For example, `tags=tag1,(tag2,tag3)`
* You can also use an array `["tag1",["tag2","tag3"]]`
* Both mean tag1 AND (tag2 OR tag3)
* @param {Array.|string} queries.facetFilters - Filter the query by a list of facets.
* For example: `company:xxx,firstname:John`.
* You can also use an array `['company:xxx','firstname:John"]`.
* @param {Array.|string} queries.facets - List of object attributes that you want to use for faceting.
* @param {string} queries.queryType - How the query words are interpreted. Options:
* - prefixLast (default): only the last word is interpreted as a prefix.
* - prefixAll: all query words are interpreted as prefixes,
* - prefixNone: no query word is interpreted as a prefix (not recommended).
* @param {string} queries.optionalWords - List of words considered optional when found in the query.
* @param {number} queries.distinct - If set to 1, enable the distinct feature (disabled by default)
* @param {Array.|string} queries.restrictSearchableAttributes - List of attributes for searching either array or comma separated.
* Must be subset of attributesToIndex.
*/
AlgoliaClient.prototype.search = function (indexName, queries) {
const index = this.algoliaSearch.initIndex(indexName);
return index.search(queries);
};You can simply provide the keyword only
example.js
algoliaClient.search('contacts', 'Donald').then(console.log);Or use advanced search.
example.js
client.search('contacts', {
query: 'Donald',
page: 2,
hitsPerPage: 2,
attributesToRetrieve: 'firstname,fax,address',
attributesToHighlight: 'firstname',
attributesToSnippet: ['address:8'],
numericFilters: 'followers>1000',
})
.then(console.log);The examples above are only a few of examples using algoliasearch in Node.js. You can read the API documentation for the list of available methods.
