Powerful, configurable, and extensible text search for your content.

• Create a copy of your content optimised for full-text search

• Store this in Elasticsearch, and automatically keep it up to date

• Generate a query with out-of-the-box support for:

  • Partial word matching
  • Localised natural language queries
  • Result highlighting

Contentful's search is good, but not optimised for text content. You might want to consider Elasticsearch over the built-in search when:

• You have a lot of content and it is important for users to find the right piece (increasing precision of search results)
• You want to customise the relevance scoring of search results (e.g. ranking popular content higher)
• Your users often search using natural language or partial words, for example a user searching for 'force' would get back results for 'Salesforce' using this package)

npm install --save contentful-text-search

const ContentfulTextSearch = require('contentful-text-search')
const search = new ContentfulTextSearch({ space: 'space_id', token: 'access_token' })
search.indexer.fullReindex()

// later
search.query('searchTerm', 'en-US')

Initialise the module using the new operator, passing in the mandatory values for:

• Contentful space ID
• Contentful access token

Optionally, also pass in:

• Elasticsearch host - Default: http://localhost:9200
• Contentful API host - Default: cdn.contentful.com
• Redis URL - Default: redis://localhost:6379
• Elasticsearch username - Default: elastic
• Elasticsearch password - Default: none
• Elasticsearch log level - Default: info

const ContentfulTextSearch = require('contentful-text-search')
const search = new ContentfulTextSearch({
  space: 'string',
  token: 'string',
  elasticHost: 'optionalString',
  contentfulHost: 'optionalString',
  redisHost: 'optionalString',
  elasticUser: 'optionalString',
  elasticPassword: 'optionalString',
  elasticLogLevel: 'optionalString'
})

Although most of the indexing functions return a promise, you should be aware that Elasticsearch is 'near real-time', so you might have to deal with a short delay before an indexed document is available in search results.

Delete and recreate an index for each locale in the space, and index the content into these indices. You need to call this the first time you use the module, but after that only when your content model changes.

search.indexer.fullReindex() // returns a promise

Clear the indices and reindex all the content from Contentful. You can use this to update the indices if they are out of date, assuming the content model hasn't changed.

search.indexer.reindexContent() // returns a promise

Deletes all indices related to this space. Could be used to clean up your Elasticsearch cluster after deleting a Contentful space.

search.indexer.deleteAllIndices() // returns a promise

Queries the Elasticsearch index and get back search results as JSON ordered by relevance, with highlights showing where your search term appeared in the result. Both parameters are mandatory.

search.query('searchTerm', 'localeCode') // returns a promise containing the results and highlights

Uses the contentful-webhook-listener package to listen for webhooks. You can start the server like this:

const server = search.update.createServer()
server.listen(3000)

The createServer method returns an extended instance of Node's http.server, so you have all those methods available, and could for example pass in an Express instance as the requestListener. The server object is also always available at search.update.server.

You can use basic auth with the webhooks like this:

const server = search.update.createServer({ auth: username:password })

You should set up webhooks in Contentful pointing to the URL of this server, with all events, or at least all events for Entries.

When developing locally, or behind a proxy, this package uses contentful-webhook-tunnel instead, which automatically sets up the webhook in Contentful, and creates a tunnel through ngrok. See the documentation of that package for more details on how to enable this.

See the debug module. Use the package name (contentful-text-search) as the string in the environment variable.

Use the Contentful Sync API to keep a local copy of our content in Redis - because we need all/most of our content for indexing, Redis should be faster than the Content Delivery API.

Here we remap Contentful fields (e.g. dereferencing, de-localising, and stripping out extraneous info), and reformat some data, for example converting markdown to plain text.

• Contentful entry title is always mapped to a field called 'title' (unless there is a field named 'title'). Don't use long text fields / fields with markdown as the title.
• Long text fields have their formatting stripped in case they are markdown. (Using marked-plaintext)

At this step the transformed data is passed through our analysis chain.

The content for each locale from Contentful is uploaded to a separate index.

• Long and short text fields are indexed as-is, and after going through a partial analyser
• Long text fields are also put through a local language analyser, i.e. english analyser for English content or german analyser for German content.

Send a string and get back search results as JSON ordered by relevance

• best_fields multi-match on all fields

Also get back highlighted text snippets with your search results, showing where your query appears in results.

• FVH for speed on long text fields with support for localisation and partial word matches
• Default highlighting on short text fields with support for partial word matches

We keep the index up to date via Contentful webhooks. You could also perform a complete reindex of the content every so often using a scheduler like node-cron

• Remove an entry from the index when it is unpublished, archived, or deleted
• Update an entry in the index or add an entry to the index when it is published

• Remove an entry when it is archived or deleted
• Update or add an entry when it is created, saved, or unarchived

• Exclude content types and fields from being indexed

• Specify the transformation and analyser for each field

• Exclude fields from query

• Boost fields in query

• Create a new transformation e.g. markdownToPlainText()
• Create a new analyser e.g. edge-ngram for instant search

• Localised indexing and querying generated from the content model

• Update index automatically

• Add configuration options

• Add extensibility options / plugin system
• Add autocomplete feature
• Add popularity feature

All contributions welcome! Please feel free to open an issue/PR 😄