Typehead Spellchecker built using Typesense.
Outline: [Article Title]
Keyword: [Enter Targeted Keyword]
Keyword MSV: [Enter Targeted Keyword’s Monthly Search Volume]
Author: [Enter Author Name]
Due Date: [Enter Due Date]
Publish Date: [Enter Desired Publish Date]
User Persona: [Enter Targeted Reader and/or User Persona]
Molding a search engines from absolute scratch which are typo-tolerant, effective, and efficient is really very difficult as well as compilcated task.Even if the requested item is in the database, a typographical error could lead a search to yield absolutely nothing. Typesense might save a lot of time and effort by eliminating the need to construct a search engine.Typesense is an open-source typo-tolerant search engine for developers that aims to reduce the amount of time it takes to find information. More information on typesense can be found here.
Installing Typesense, establishing a new application from scratch, preconfiguring the client, and more are all covered in this article. This article will also demonstrate how to create a Typesense Collection. Finally, we’ll launch and initialize our app, before adding an item to our collection and beginning our search.
Creating a Spellchecker App from the Ground Up!
Setting up Typesense
Typesense can be used by installing the Typesense prebuilt docker image, which Typesense has already given, or by using the Typesense cloud hosting offering, which is the simplest way to get started. To get started, click to the Typesense cloud website and sign up using your Github account, or use the simple and straightforward docker way. For the purposes of this tutorial, we’ll use the Docker technique. To follow along, go to the Typesense Dockerhub and download the prebuilt docker image, then follow the steps below.
docker pull typesense/typesense
and
mkdir /tmp/typesense-data
docker run -p 8108:8108 -v/tmp/data:/data typesense/typesense:0.22.2 --data-dir /data --api-key=spellcheck --enable-cors --listen-port 8108
Installing Typesense
npm install typesense
Now that Typesense is installed and running, we can construct a Typesense collection, index some documents in it, and try searching for them. Finally, let’s create a search interface for our application.
Crafting Search UIs from scratch.
By making use of the open source InstantSearch.js library library or its React, Vue, and Angularjs, as well as the Typesense-InstantSearch-Adapter, you can create a plug-and-play fully-featured search interface with just a few lines of code.
Installing instantsearch-app: a step-by-step guide
Let’s begin with a basic template:
$ npx create-instantsearch-app spell-checkerapp-typesense
? The name of the application or widget spell-checkerapp-typesense
? InstantSearch template InstantSearch.js
Creating a new InstantSearch app in spell-checkerapp-typesense.
? InstantSearch.js version 4.38.1
? Application ID latency
? Search API key spellcheck
? Index name words
? Attributes to display
Used to generate the default result template
? Attributes to display
Used to filter the search interface Dynamic widgets
📦 Installing dependencies...
yarn install v1.22.0
info No lockfile found.
[1/4] 🔍 Resolving packages...
[2/4] 🚚 Fetching packages...
[3/4] 🔗 Linking dependencies...
[4/4] 🔨 Building fresh packages...
success Saved lockfile.
Done in 173.57s.
🎉 Created spell-checkerapp-typesense at spell-checkerapp-typesense.
Begin by typing:
cd spell-checkerapp-typesense
yarn start
⚡️ Start building something awesome!
First step: pick a name for your application and write it down.
Second step: Select the version of the Instantsearch application that you want to install
Third step: select the default latency as an application id.
Fourth step: Provide Search API Key to that project.
Fifth step: Specify the Index name for this application.
Sixth step: Select None as an attributes and Hit Enter.
Final step: Select Dynamic Widgets as an attributes to display and simply Hit Enter.
This is how your folder structure should look after you’ve finished installing your UI.
To use InstantSearch with a Typesense backend, we’ll also need to install the Typesense InstantSearch adapter:
$ npm install typesense-instantsearch-adapter
After you’ve installed all of the necessary dependencies, your package.json file should look like this.
{
"name": "spell-checkerapp-typesense",
"version": "1.0.0",
"private": true,
"main": "src/app.js",
"scripts": {
"start": "parcel index.html --port 3000",
"build": "parcel build index.html",
"lint": "eslint .",
"lint:fix": "npm run lint -- --fix"
},
"devDependencies": {
"babel-eslint": "10.0.3",
"eslint": "5.7.0",
"eslint-config-algolia": "13.2.3",
"eslint-config-prettier": "3.6.0",
"eslint-plugin-import": "2.19.1",
"eslint-plugin-prettier": "3.1.2",
"parcel-bundler": "1.12.5",
"prettier": "1.19.1"
},
"dependencies": {
"algoliasearch": "4",
"instantsearch.js": "4.38.1",
"typesense": "^1.1.3",
"typesense-instantsearch-adapter": "^2.3.0"
}
}
As you can see, it’s a pretty simple app with only two key files: app.js and index.html, which contains the app’s entire structure. An id can be found in the index.html file. If you wish to incorporate UI components and widgets like asearchbox,results,Hits,facets, and as well as pagination widgets, so we may make use those ids for this widget integration process.
<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta
name="viewport"
content="width=device-width, initial-scale=1, shrink-to-fit=no"
/>
<meta name="theme-color" content="#000000" />
<link rel="manifest" href="./manifest.webmanifest" />
<link rel="shortcut icon" href="./favicon.png" />
<link
rel="stylesheet"
href="https://cdn.jsdelivr.net/npm/instantsearch.css@7/themes/algolia-min.css"
/>
<link rel="stylesheet" href="./src/index.css" />
<link rel="stylesheet" href="./src/app.css" />
<title>spell-checkerapp-typesense</title>
</head>
<body>
<header class="header">
<h1 class="header-title">
<a href="/">spell-checkerapp-typesense</a>
</h1>
<p class="header-subtitle">
using
<a href="https://github.com/algolia/instantsearch.js">
InstantSearch.js
</a>
</p>
</header>
<div class="container">
<div class="search-panel">
<div class="search-panel__filters">
<div id="dynamic-widgets"></div>
</div>
<div class="search-panel__results">
<div id="searchbox"></div>
<div id="hits"></div>
</div>
</div>
<div id="pagination"></div>
</div>
<script src="https://cdn.jsdelivr.net/npm/algoliasearch@4.10.5/dist/algoliasearch-lite.umd.js"></script>
<script src="https://cdn.jsdelivr.net/npm/instantsearch.js@4.38.1"></script>
<script src="./src/app.js"></script>
</body>
</html>
The app.js file is configured for algolia, as you can see below, and we’ll need to alter it for typesense.
// app.js
const { algoliasearch, instantsearch } = window;
const searchClient = algoliasearch("latency", "spellcheck");
const search = instantsearch({
indexName: "words",
searchClient,
});
search.addWidgets([
instantsearch.widgets.searchBox({
container: "#searchbox",
}),
instantsearch.widgets.hits({
container: "#hits",
}),
instantsearch.widgets.configure({
facets: ["*"],
maxValuesPerFacet: 20,
}),
instantsearch.widgets.dynamicWidgets({
container: "#dynamic-widgets",
fallbackWidget({ container, attribute }) {
return instantsearch.widgets.refinementList({
container,
attribute,
});
},
widgets: [],
}),
instantsearch.widgets.pagination({
container: "#pagination",
}),
]);
search.start();
We’ll need to install the Typesense InstantSearch adaptor to use InstantSearch with a Typesense backend. To do so, simply open a terminal window and type the following command (you can skip this step if you’ve previously installed it into your project).
npm install --save typesense-instantsearch-adapter
Finally, the project can be tweaked to make it work with Typesense. We’ve got our typesense instance up and operating in the background. To get InstantSearch.js to use the Typesense adapter, open src/app.js and change the way InstantSearch is initialized:
const searchClient = algoliasearch("latency", "songsearch");
const search = instantsearch({
indexName: "words",
searchClient,
});
to this:
const { instantsearch } = window;
import TypesenseInstantSearchAdapter from "typesense-instantsearch-adapter";
const typesenseInstantsearchAdapter = new TypesenseInstantSearchAdapter({
server: {
apiKey: "spellcheck",
nodes: [
{
host: "localhost",
port: "8108",
protocol: "http",
},
],
},
additionalSearchParameters: {
queryBy: "word", //quering by
},
});
const searchClient = typesenseInstantsearchAdapter.searchClient;
const search = instantsearch({
searchClient,
indexName: "words",
});
Now we need to import our words data, so create a dataset folder and inside it, create your own words json file, filling it with all of the necessary words information, or download the words dataset from here. Finally, your folder structure should look something like this.
Let’s get started with the data-importing scripts. We’ll begin by creating a file named loadData.js in which we’ll export and initialize the typesense client, but first we’ll need to import a few modules such as fs,readline, and Typesense.
// loadData.js
const fs = require("fs");
const readline = require("readline");
const Typesense = require("typesense");
Now create an asynchronous indexWordsToTypesense function and copy the code exactly as it is shown below.This function will simply index all of the songs into Typesense, and if an error occurs during the process, it will simply log an error.
// loadData.js
const fs = require("fs");
const readline = require("readline");
const Typesense = require("typesense");
async function addWordsToTypesense(typesense, wordsCollection, englishWords) {
const results = await typesense
.collections(wordsCollection)
.documents()
.import(englishWords);
const failedItems = results.filter((item) => item.success === false);
if (failedItems.length > 0) {
console.log(JSON.stringify(failedItems, null, 2));
throw "Error indexing items";
}
}
Creating a collection : a step-by-step guide
In Typesense, a Collection is a group of related Documents that works similarly to a table in a relational database. When we establish a collection, we give it a name and explain the fields that will be indexed when a document is added to it .Now head over to loadData.js file and add the following changes to the code.
First step: We need to create a typesense client
const typesense = new Typesense.Client({
nodes: [
{
host: "localhost",
port: "8108",
protocol: "http",
},
],
apiKey: "spellcheck",
});
Second step: If an error occurs while loading the data(data duplication error), simply add the following snippet of code to the loadData.js file before creating schema becasue it will simply remove the existing data and populates it with the new ones.
await typesense.collections("words").delete();
Third step: Create a schema
const wordsCollection = `words`;
const schema = {
name: wordsCollection,
fields: [
{ name: "word", type: "string" },
{ name: "popularity", type: "int32" },
],
default_sorting_field: "popularity",
};
console.log(`Populating data in Typesense "${wordsCollection}" collection`);
console.log("Creating schema: ");
await typesense.collections().create(schema);
console.log("Adding records: ");
Fourth step: Now that we have the schema, we must add the documents. We have our documents saved as words.jsonl in our dataset folder. All you have to do now is import and load the document into our loadData.js file.Also, we can specify the batch file size, and for now, let’s insert 5000 elements.
const fileStream = fs.createReadStream("./dataset/words.jsonl");
const rl = readline.createInterface({
input: fileStream,
crlfDelay: Infinity,
});
let englishWords = [];
let currentLine = 0;
for await (const line of rl) {
currentLine += 1;
const parsedRecord = JSON.parse(line);
try {
englishWords.push({
word: String(parsedRecord["word"]),
popularity: parseInt(parsedRecord["count"]),
});
} catch (e) {
console.error(e);
console.error(parsedRecord);
throw e;
}
if (currentLine % 5000 === 0) {
await addWordsToTypesense(typesense, wordsCollection, englishWords);
console.log(` Words upto ${currentLine} ✅`);
englishWords = [];
}
if (currentLine >= Infinity) {
break;
}
}
if (englishWords.length > 0) {
await addWordsToTypesense(typesense, wordsCollection, englishWords);
console.log("✅");
}
Fifth step: Now, make a self-invoking asynchronous function then add and export everything.
module.exports = (async () => {
const typesense = new Typesense.Client({
nodes: [
{
host: "localhost",
port: "8108",
protocol: "http",
},
],
apiKey: "spellcheck",
});
await typesense.collections("words").delete();
const wordsCollection = `words`;
const schema = {
name: wordsCollection,
fields: [
{ name: "word", type: "string" },
{ name: "popularity", type: "int32" },
],
default_sorting_field: "popularity",
};
console.log(`Populating data in Typesense "${wordsCollection}" collection`);
console.log("Creating schema: ");
await typesense.collections().create(schema);
console.log("Adding records: ");
const fileStream = fs.createReadStream("./dataset/words.jsonl");
const rl = readline.createInterface({
input: fileStream,
crlfDelay: Infinity,
});
let englishWords = [];
let currentLine = 0;
for await (const line of rl) {
currentLine += 1;
const parsedRecord = JSON.parse(line);
try {
englishWords.push({
word: String(parsedRecord["word"]),
popularity: parseInt(parsedRecord["count"]),
});
} catch (e) {
console.error(e);
console.error(parsedRecord);
throw e;
}
if (currentLine % 5000 === 0) {
await addWordsToTypesense(typesense, wordsCollection, englishWords);
console.log(` Words upto ${currentLine} ✅`);
englishWords = [];
}
if (currentLine >= Infinity) {
break;
}
}
if (englishWords.length > 0) {
await addWordsToTypesense(typesense, wordsCollection, englishWords);
console.log("✅");
}
})();
Your final code inside loadData.js file should look like this.
// loadData.js
const fs = require("fs");
const readline = require("readline");
const Typesense = require("typesense");
async function addWordsToTypesense(typesense, wordsCollection, englishWords) {
const results = await typesense
.collections(wordsCollection)
.documents()
.import(englishWords);
const failedItems = results.filter((item) => item.success === false);
if (failedItems.length > 0) {
console.log(JSON.stringify(failedItems, null, 2));
throw "Error indexing items";
}
}
module.exports = (async () => {
const typesense = new Typesense.Client({
nodes: [
{
host: "localhost",
port: "8108",
protocol: "http",
},
],
apiKey: "spellcheck",
});
await typesense.collections("words").delete();
const wordsCollection = `words`;
const schema = {
name: wordsCollection,
fields: [
{ name: "word", type: "string" },
{ name: "popularity", type: "int32" },
],
default_sorting_field: "popularity",
};
console.log(`Populating data in Typesense "${wordsCollection}" collection`);
console.log("Creating schema: ");
await typesense.collections().create(schema);
console.log("Adding records: ");
const fileStream = fs.createReadStream("./dataset/words.jsonl");
const rl = readline.createInterface({
input: fileStream,
crlfDelay: Infinity,
});
let englishWords = [];
let currentLine = 0;
for await (const line of rl) {
currentLine += 1;
const parsedRecord = JSON.parse(line);
try {
englishWords.push({
word: String(parsedRecord["word"]),
popularity: parseInt(parsedRecord["count"]),
});
} catch (e) {
console.error(e);
console.error(parsedRecord);
throw e;
}
if (currentLine % 5000 === 0) {
await addWordsToTypesense(typesense, wordsCollection, englishWords);
console.log(` Words upto ${currentLine} ✅`);
englishWords = [];
}
if (currentLine >= Infinity) {
break;
}
}
if (englishWords.length > 0) {
await addWordsToTypesense(typesense, wordsCollection, englishWords);
console.log("DONE ✅!!");
}
})();
Finally, type node .\loadData.js into the terminal of that same project directory, and you have successfully imported One thousand documents into our search index.
Now,you can start your application by typing npm start in the terminal of that project directory.
We need to fix the fact that the application works flawlessly but that nothing is displayed.So, to fix it we need to incorporate lot of changes,but first go to our app.js file and remove a few of the widgets that we don’t need
const { instantsearch } = window;
import TypesenseInstantSearchAdapter from "typesense-instantsearch-adapter";
const typesenseInstantsearchAdapter = new TypesenseInstantSearchAdapter({
server: {
apiKey: "spellcheck",
nodes: [
{
host: "localhost",
port: "8108",
protocol: "http",
},
],
},
additionalSearchParameters: {
queryBy: "word", //quering by
},
});
const searchClient = typesenseInstantsearchAdapter.searchClient;
const search = instantsearch({
searchClient,
indexName: "words",
});
search.addWidgets([
instantsearch.widgets.searchBox({
container: "#searchbox",
}),
instantsearch.widgets.hits({
container: "#hits",
}),
instantsearch.widgets.pagination({
container: "#pagination",
}),
]);
search.start();
Simply re-run the application after you’ve fixed it, and your application should now look like this.
You can now build a search interface using any of the InstantSearch widgets, and we’ll be adding a template to present the data in a pleasing manner.Simplycopy the template code below and add it inside the #hits widget present inside app.js file.
templates: {
item: ` <article class="hit">
<div>
<h1>
{ "attribute": "word" }
</h1>
</div>
<div
style="
display: grid;
grid-auto-flow: column;
justify-content: start;
align-items: center;
gap: 8px;
"
>
<div
style="
display: grid;
grid-auto-flow: column;
justify-content: start;
align-items: center;
gap: 4px;
"
>
<svg
width="16"
height="16"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
stroke-width="2"
stroke-linecap="round"
stroke-linejoin="round"
style="
position: relative;
top: 1px;
"
>
<path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"></path>
</svg>
</div>
</div>
</article>`,
},
app.js file should resemble something like this.
const { instantsearch } = window;
import TypesenseInstantSearchAdapter from "typesense-instantsearch-adapter";
const typesenseInstantsearchAdapter = new TypesenseInstantSearchAdapter({
server: {
apiKey: "spellcheck",
nodes: [
{
host: "localhost",
port: "8108",
protocol: "http",
},
],
},
additionalSearchParameters: {
queryBy: "word", //quering by
},
});
const searchClient = typesenseInstantsearchAdapter.searchClient;
const search = instantsearch({
indexName: "words",
searchClient,
});
search.addWidgets([
instantsearch.widgets.configure({
hitsPerPage: 8,
}),
instantsearch.widgets.searchBox({
container: "#searchbox",
}),
instantsearch.widgets.index({ indexName: "words" }).addWidgets([
instantsearch.widgets.configure({
hitsPerPage: 8,
}),
instantsearch.widgets.hits({
container: "#hits",
templates: {
item: ` <article class="hit">
<div>
<h1>
{ "attribute": "word" }
</h1>
</div>
<div
style="
display: grid;
grid-auto-flow: column;
justify-content: start;
align-items: center;
gap: 8px;
"
>
<div
style="
display: grid;
grid-auto-flow: column;
justify-content: start;
align-items: center;
gap: 4px;
"
>
<svg
width="16"
height="16"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
stroke-width="2"
stroke-linecap="round"
stroke-linejoin="round"
style="
position: relative;
top: 1px;
"
>
<path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"></path>
</svg>
</div>
</div>
</article>`,
},
}),
]),
]);
search.start();
Again, re-run the application after you’ve fixed it, and your application should now look like this.
To configure the 8 hits per page or you can configure to your own requirements, simply add the following code above the #hits widgets inside the app.js file.
instantsearch.widgets.configure({
hitsPerPage: 8,
}),
Finally, lets fix our index.html file
<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta
name="viewport"
content="width=device-width, initial-scale=1, shrink-to-fit=no"
/>
<meta name="theme-color" content="#000000" />
<link rel="manifest" href="./manifest.webmanifest" />
<link rel="shortcut icon" href="./favicon.png" />
<link
rel="stylesheet"
href="https://cdn.jsdelivr.net/npm/instantsearch.css@7/themes/algolia-min.css"
/>
<link rel="stylesheet" href="./src/index.css" />
<link rel="stylesheet" href="./src/app.css" />
<title>spell-checkerapp-typesense</title>
</head>
<body>
<header class="header">
<h1 class="header-title">
<a href="/">Spell checker application Typesense</a>
</h1>
</header>
<div class="container">
<div class="search-panel">
<div class="search-panel__results">
<div
id="searchbox"
style="min-width:500px; max-width:100%;min-height:20px;width:100%;font-size: 30px; padding:10px; border: 1px solid #8661d1; border-radius: 15px; box-sizing: border-box; resize: none;"
></div>
<div id="hits"></div>
</div>
</div>
</div>
<script src="https://cdn.jsdelivr.net/npm/algoliasearch@4.10.5/dist/algoliasearch-lite.umd.js"></script>
<script src="https://cdn.jsdelivr.net/npm/instantsearch.js@4.38.1"></script>
<script src="./src/app.js"></script>
</body>
</html>
Ultimately, as a final step, let’s update the styles and template for the information that we want to display in our app, so here’s how your code should look.
/* app.css */
@import url("https://fonts.googleapis.com/css2?family=Inter:wght@100;200;300;400;500;600&display=swap");
body {
font-family: "Inter", sans-serif;
background-color: #e8e8e8;
}
.header {
display: flex;
align-items: center;
min-height: 70px;
padding: 0.5rem 1rem;
background-color: #8661d1;
color: #fff;
margin-bottom: 2rem;
}
.header a {
color: #fff;
text-decoration: none;
}
.header-title {
font-size: 1.2rem;
font-weight: normal;
}
.header-title::after {
content: " ▸ ";
padding: 0 0.5rem;
}
.header-subtitle {
font-size: 1.2rem;
}
.container {
max-width: 1200px;
margin: 0 auto;
padding: 1rem;
}
.search-panel {
display: flex;
}
.search-panel__filters {
flex: 1;
}
.search-panel__results {
flex: 3;
}
.ais-Highlight-highlighted {
color: inherit;
font-size: inherit;
}
#searchbox {
margin-bottom: 2rem;
}
#pagination {
margin: 2rem auto;
text-align: center;
}
Let’s take a look at the final version of our typesense-integrated spellcheckr application.
Closing
Typesense was built with several distinctive features primarily aimed at making the developer’s job easier while also giving customer as well as user the ability to provide a better search experience as possible.This article may have been entertaining as well as instructive in terms of how to install typesense from the ground up on a variety of platforms. Join Aviyel’s community to learn more about the open source project, get tips on how to contribute, and join active dev groups.
Call-to-Action
Aviyel is a collaborative platform that assists open source project communities in monetizing and long-term sustainability. To know more visit Aviyel.com and find great blogs and events, just like this one! Sign up now for early access, and don’t forget to follow us on our socials!