Querying DecentCMS, part2: querying the index
In the previous post, we looked at the basic concepts DecentCMS querying are built upon, and at how an index is built. In this post, we’ll examine how to query such an index in order to produce a result set.
The getIndex function returns an object that you can use to query the index. Until the index has had time to be built, querying a given index will return empty results. This is because indexing can take a very long time, depending on the number of items in the system, so even an asynchronous operation would delay the response to the user by too much.
As soon as the index has been built, the querying API will return actual result sets.
There are two ways an index can be queried: using filter, or using reduce.
Filter can be used to obtain a paginated list of index entries. It can take an optional where function that can specify what index entries should be in the result set. Remember that this condition will need to be run on potentially all index entries. For this reason, if you can, you should integrate such conditions into the mapping function, moving the burden of filtering to be run only once during the indexing phase rather than running it every time when querying.
Reduce applies an aggregating function on the index entries. Reduce can take an optional where function, with the same caveats that were mentioned in the previous paragraph.
This map/reduce model of querying may seem strange if you are used to querying databases using SQL. It is however not harder to use than SQL, and brings some unique advantages to a CMS:
- It can query heterogeneously stored items, such as database-bound items, API documentation extracted from JavaScript source files, or JSON files on disk, in one operation.
- It can then present the results in a unique, homogeneous view. It can also be easily scaled out: data stores and indexes can be sharded and spread across multiple servers, without changing the API.
An implementation of an index, as well as the standard API that all index implementations must expose, can be found under this topic:
http://decentcms.org/docs/api/decent-core-search/file-index
In the next post, I’ll show an example that uses the search content part to create and query an index to build a paginated list of API documentation topics.