Guide 11 · Developer docs

Search, filtering and faceted navigation

Site search on Core dna is an Elasticsearch index that the platform fills and queries, and a page you build. What reaches the index, search views, the search page, product facets, filtered listings that need no index, and when to go beyond the built-in index.

L2
Unit 03 · Faceted navigation for product finders

Facet counts at scale

Faceted navigation for product finders Page 13 of 20 Updated

Facet counts are the part of a product finder that gets slow. What each count costs, the two ways to keep it fast, counting only the facets a page shows and caching the counts, with the trade-off each carries, and why counts go stale whatever you do.

Who this is for. The developer whose product finder is fast on a test catalogue and slow on the real one, and the one sizing a finder before it is built. Afterwards you can say what every count on the page costs, and choose between counting less and caching more.

What a count costs

Every facet marked Renderable on a Faceted Search view is one aggregation: Elasticsearch groups every product that matches the current query and filters by that facet's values, counts each group, sorts the groups by the facet's Order and returns as many as its Limit. It does this for every renderable facet, on every request, over the whole matching set, not only the page of results shown. So the cost grows with three things: the number of renderable facets, the number of matching products, and the number of distinct values in each facet. A browse page with no query and no filters matches the whole catalogue, which makes it the most expensive page of the finder, and often the most visited.

Two habits multiply the cost: true counts within a facet, which add a full search for each facet with a selection (Facets with a text query), and a facet with many distinct values, such as a product code, made renderable when it is only ever filtered on. Measure before mitigating: compare the search page's response time with and without facets on a copy of the real catalogue (Measure first).

Mitigation 1: count only the facets the page shows

Only renderable filter fields are aggregated, so the count you do not ask for costs nothing:

  • Mark Renderable only the facets this page displays, and give a page that shows fewer facets its own view: a category landing page with three facets does not need the twelve of the full finder.
  • Do not make a Hidden facet renderable. A facet marked Hidden but still renderable on the view is counted and then thrown away, never displayed. Filter on it without Renderable.
  • Keep each facet's Limit to what the interface lists. A facet that shows ten options does not need fifty buckets.

The trade-off: a facet that is not counted cannot be shown with counts on that view, and every extra view is one more record to keep in step with the templates that call it.

Mitigation 2: cache the counts

A search page whose template declares every parameter that changes it can be stored by the page cache, one copy per combination, for signed-out visitors (Caching: declare every parameter the page reads). Declare q, page, orderBy and every facet parameter the finder uses, with PHP's underscore for the dot: fields.brand is declared as fields_brand. Declare the same list on every render, by name rather than from the facets returned: the cache keeps one list of parameters for each path, and a render that declares a different list replaces it. To bound how stale a stored copy can be, give the page a shorter life than the site's default of a day:

Template code search/search.html 
{* the same list on every render, then a five-minute life *}
{$_response->setCacheVaryParameters(['q', 'page', 'orderBy', 'fields_brand', 'fields_colour', 'fields_price-band'])}
{$_response->setCacheTTL(300)}
{# the same list on every render, then a five-minute life #}
{% do _response.setCacheVaryParameters(['q', 'page', 'orderBy', 'fields_brand', 'fields_colour', 'fields_price-band']) %}
{% do _response.setCacheTTL(300) %}

The trade-offs are real. Every combination of values is a separate stored copy, so the cache helps only the popular ones, and a long tail of combinations still reaches Elasticsearch. Counts on a stored copy are as old as the copy. And a parameter that is not declared makes the cache serve one combination's page for another, with nothing on the page to show it, so a facet added to the finder has to be added to the list too. Unless traffic demands it, prevent caching and count on every request, which is the safe default.

Counts that are stale whatever you do

Caching is not the only reason a count can be wrong. Facet values are computed when a product's description is indexed, and a change to the product itself, such as a new price or a stock level, does not reindex the description. A price band, or an in-stock facet computed in the mapping template, keeps its old value until the description is saved again or the facets index is reindexed (What leaves the index stale). Schedule a Reindex of the Faceted Search index after every price or stock import, and do not compute a facet from data that changes many times a day.

Go deeper
GuideFacets with a text query GuideFacets and the mapping template GuideThe caching tiers GuideMeasure first

Was this page helpful?

Your answer goes to the docs team.