Azure Service Bus Subscriptions with Correlation Filters
Azure Service Bus pub/sub is implemented using topics and subscriptions. Messages are published to topics and copied over to subscription queues with matching criteria. Criteria are declared using Rules. Each rule has a Filter. Filters help the broker decide if a message sent to a topic will be copied over to a subscription or not. Let’s dive into the world of filters to understand how they work. There are three types of filters supported by the broker:
- Boolean filters
- SQL filters
- Correlation filters
Boolean filters
These filters (TrueFilter and
FalseFilter) are not the most sophisticated.
They are literally "catch-all" or "catch nothing" options.
The TrueFilter is the default when nothing else
is defined. It’s handy when implementing a wiretap to
analyze all messages flowing through a topic.
SQL filters
Just as the name indicates, SQL filters allow SQL
language-based expressions to define criteria used to filter
to identify messages that will be copied over to
subscription. If you’re interested in the syntax, see
documentation. One thing I’ll mention is the idea of scope, which
denotes the type of property; user-defined properties
prefixed with user and system defined
properties prefixed with sys.
With SQL filters it’s possible to create very complex rules for filtering messages out. Keep in mind that the more complex these rules are, the higher performance tall will be on the broker will have to apply these rules to every message. An example of a SQL rule:
sys.Label LIKE '%bus%'` OR `user.tag IN ('queue', 'topic', 'subscription')
Correlation filters
Unlike Boolean and SQL filters, this group is used to perform matching against one or more user and system properties in a very efficient way. Paraphrasing official documentation:
The CorrelationFilter provides an efficient filter that deal with equality only. As such, the cost of evaluating filter expression is minimal and almost immediate w/o extra compute required
The only challenge with this filter, it’s not quite clear how to use it. There are two constructors
- Default (empty) constructor
- Constructor taking a single argument
The second constructor with a single
string argument initiates a correlation filter
to use the passed in value to be the criteria for
CorrelationId. The first constructor is a
mystery. Well, not quite. Empty correlation filter can be
used to assign other system and user properties that can be
used for filtering. For system properties
ContentType, MessageId,
ReplyTo, ReplyToSessionId,
SessionId, To, and
CorrelationId can be assigned values to filter
on. What about user-defined properties? That’s nwhere it’s a
little unclear if you follow the documentation, which
hopefully will get updated soon.
To specify user-defined properties for correlation filter, a
property Properties of type
IDictionary <string, object> is exposed.
Keys for this dictionary are the user-defined properties to
look up on messages. Values associated with keys are the
values to correlate on. Here’s an example.
var filter = new CorrelationFilter();
filter.Label = "blah";
filter.ReplyTo = "x";
filter.Properties["prop1"] = "abc";
filter.Properties["prop2"] = "xyz";
Created filter will have the following criteria:
sys.ReplyTo = 'x' AND sys.Label = 'blah' AND prop1 = 'abc' AND prop2 = 'xyz'
And you’ve guessed it right. When correlating on multiple properties, logical AND will be used so that all properties have to have the expected values for the filter to be evaluated as truthy.
In case you wondered how user-defined properties are populated, here's an example:
message.Properties["prop1"] = "abc";
Conclusions
Filtering messages can be done in several ways. Evaluate
what filter to create and don’t default to SQL filter just
because it’s easier to create. If filters can be simple
enough to be expressed with correlation, prefer
CorrelationFilter over SqlFilter.
And remember, no matter what filter is used, filters cannot
evaluate message body, but you can always promote from
message body to properties/headers.