Introducing attribute-based feature tracking

It’s common to track feature usage with attributes. For example, at Bucket, we’re tracking if customers use our weekly reporting feature. We do this by passing an attribute called "weekly_reporting" via Segment’s "group" call.

This is great for ad-hoc analysis. It’s straightforward to query all companies that have "weekly_reporting" set to "true" or "false".

Unfortunately, though, attribute values change over time and its history is lost, which means we can’t know whether customers that aren’t using the weekly reporting feature ("weekly_reporting=false") have never used it, or once were using it, but turned it off.

Additionally, we can’t know when companies started using the reporting feature, and for how long they kept it on.

These things matter a lot. Is there a feature awareness problem or a feature retention problem?

…🥁…

Great news - now you can know! Today, we’re releasing support for attribute-based features. We think this is a really powerful addition to Bucket.

If you’re already tracking attributes in your application, this will just work™ as is. Starting today, Bucket automatically keeps track of all company-level attributes and will automatically track any attribute value changes over time.

(If you’re not familiar with attributes, check out our post from last week: When to track features with events or attributes)

Getting started

Let’s say we work at Slack and that we want to track usage of the third-party Slack apps.

Whenever a user logs into Slack, we’re tracking which company they belong to and send the latest attribute values for the respective company. In code, this looks like this:

Followed by

Whenever a user logs into Slack, we check which Slack apps the user’s workspace is using and report that via the "apps" attribute. As you can see in the snippet above, “Acme Inc” is only using the Datadog app at the moment.

As a developer, simply grabbing the current state and passing it on as attributes is a really convenient workflow. Let Bucket handle the historic changes.

Feature reporting

With the attribute data flowing in, we can start tracking an attribute-based feature on Bucket.

Let’s say we want to track usage of the Stripe Slack app. To do so, we simply create a new feature and define active usage as: "apps" contains "stripe".

Here’s the feature report:

We now know that 6% of our customers use the Stripe app. We can also see that 2% have then stopped using the Stripe app. We can also see when customers installed and uninstalled the app, and who they are. Neat!

How it works

Let’s look at what’s going on under the hood. When the "group" call comes in, per company, we check if the "apps" values match our "apps" contains "stripe" filter. For "Acme Inc" there’s no match, so we place "Acme Inc" in the "Never used it"-state on the feature report.

Later, someone at "Acme Inc" installs the Stripe app. Bucket notices the difference in "apps" value, and runs the filter again. This time, there’s a match! "Acme Inc" is now actively using the Stripe app.

On the Bucket chart, we can even see which week the shift happened.

Later, "Acme Inc" decides to uninstall the Stripe app. Bucket again sees a difference in the "apps" value and now automatically places "Acme Inc" in the "Stopped using it"-state.

Notice that Bucket didn’t just put "Acme Inc" in "Never" when the "stripe" value disappeared. That’s because Bucket keeps track of the history and is aware that "Acme Inc" once used the Stripe app.

On the same chart, Bucket charts when the customer churned from this feature.

We love this chart and use it all the time. For most features, we eagerly check the amount of purple bars after shipping a feature. New users, yay! In the following weeks, though, we also keep track of the amount of red bars, which is just as "exciting" and crucially important.

More updates next week.