Changelog

If your feature has an onboarding funnel, you can now filter on event attributes when creating funnel steps. This enables you to track the funnel sliced by a certain attribute value - like browser, feature version or any other custom attribute - or to use just one event for all the funnel steps.

In the example below, the funnel onboarding steps use an event called `messaging_setup` which then has an attribute called `funnelStep: [1,2,3…]` for each step of the funnel.

More updates soon!

You can now sort companies by when they first or last used a feature. When shipping a new feature, this view enables you to easily track its new adopters. Who are they and how often do they use the new feature after the first try.

On the “Stopped” tab, you can track which adopters that ended up churning away. Those are likely the companies you want to reach out to for feedback in order to learn how to improve the feature in its next iteration.

We use this feature ourselves whenever we ship a new feature, like this one ;)

More updates soon!

We’ve spent the past weeks making significant improvements to load times in Bucket. The application was getting pretty slow for accounts with a large number of end-users (~20K+) that also tracked a large number of features. In particular, the list of features could take a loong time to load. Sorry!

To address this, we’ve enabled caching on most queries. Once you’ve visited a page, Bucket will cache it and make sure the data is fresh and loading fast going forward.

Here’s a visualization of the load time improvement for a very large account - from 10+ seconds to 1-2 seconds:

Speaking of larger accounts, we’ve also shipped better formatting of large numbers.. Instead of seeing e.g. “19542 companies” you’ll now see “19K companies”in the UI.

More updates soon 👋

‍

When creating a feature in Bucket it's now possible to specify that only events with attributes which fulfill certain criteria be included in the feature report. This makes it even easier to start tracking advanced features in Bucket!

Imagine having built a chat application and introducing the possibility of sending emojis in messages. You might be tracking whenever anyone sends a message by generating a “Sent message” event and when the message contains emojis you set the “hasEmoji” attribute on those events. Or you want to track how many companies have made a real group chat that wasn’t just a 1 on 1 chat by selecting the “Started Group Chat” event, but only selecting those events where there were more than two members in the group.

Happy shipping!

‍

Last week we shipped the ability to create a feature onboarding funnel which tracks the steps people go through before moving into the “Tried it” state.

In Bucket you can track features based on events that your users trigger but you can also track feature usage through attributes set on companies. Imagine having built a Slack integration for your application. It’s convenient to track who has set up the integration using an attribute "has_slack_enabled=true" which you set on the account.

However, enabling the slack integration is a multi-step process. Users must authenticate with Slack, decide which kind of updates they want to receive in Slack, pick a Slack channel to receive the updates, etc.. Finally when the user has gone through the setup process, your application will update the attribute on their account to indicate the Slack integration is now enabled for this account. 

This week we extended the functionality we shipped last week to work for features that track feature-usage using attributes - in addition to features that use events to track usage.

The way it works is that to be counted in the last step of the funnel, a company must have performed all the previous steps in order and then the company attributes must be updated to fulfill the Adoption criteria as it’s defined for the given feature. In the case of our example, the company must have first sent the “Authenticated” event, then the “Channel selected” event and then finally the company attributes must have been updated such that "has_slack_enabled=true".

Happy shipping!

The Bucket feature report has four feature adoption states. All of your companies are placed in one of these states based on their usage of the respective feature:

Companies enter the “Tried it”-state as soon as they’ve interacted with the feature at least once.

For some features, though, interacting with that feature might require some prior actions to be completed first. We refer to this as the feature onboarding funnel.

For example, let’s say your product has a Slack integration and you want to track how many companies share reports to Slack from your product. The event to track this might be called “Send to Slack”.

However, before being able to send reports to Slack, your companies must first set up the Slack integration. They need to authenticate with Slack and then choose a channel. In this case, the feature’s onboarding funnel looks like this:

For companies to reach “Tried it”-state, they have to get through the funnel at least once.

From Never to Tried it

For features with onboarding steps, you specifically want to answer this question:

Are the companies in Never not interested in this feature or are they getting stuck trying to use it?

The feature onboarding funnel is designed to help you answer this question.

Let’s combine the adoption states with the onboarding funnel to get a good overview:

Looking at this, it’s quite clear that a lot of companies are trying to use the “Send to Slack” feature, but the vast majority fails to progress from the “Choose channel” step. Of the 455 companies that enter the funnel, only 135 manage to get through it!  

(Fun fact: We discovered this very problem ourselves at Bucket as we had forgotten to support pagination when fetching the Slack channels. Oops!)

Available today

The onboarding funnel is available today and is now a default part of the feature report. Here’s what it looks like in action:

For features with onboarding steps, simply click “+” to add those steps to the funnel and learn what the drop-off rates are between steps. Each step has to be a distinct event.

The onboarding funnel doesn’t affect how the Tried it or Using it is calculated – it’s simply a secondary insight into what happens between Never and Tried it. To learn more about how the funnel works exactly, check out this help article.

Happy shipping!

‍

When you track an event-based feature on Bucket, you want to know how often your users interact with the feature. The Bucket frequency score tells you exactly that.

Score, not count

Many tools let you see how many times users have interacted with a certain event (feature). This provides you with some insight - none, few or many times - but what does that actually mean?

Are they still using the feature? If so, how often? When did they stop?

To illustrate the above, users of these two companies have interacted with the same feature 100 times, but they have very different underlying usage patterns:

‍
Company A used the feature consecutively for a period and then churned. Company B on the other hand has recently started using the feature a lot.

Daily, Weekly, Monthly or Quarterly

To  transform count into something more useful and actionable, Bucket calculates a frequency score out of the box. We look at recent usage data, per company, and transform the result into an understandable and informative frequency score. The score results can be one of the following:

• (Never)
• (Tried it)
• Daily usage
• Weekly usage
• Monthly usage
• Quarterly usage
• (Stopped)

Here’s what that looks like when looking at feature usage on aggregate for all companies (average):

Each company has a frequency score, so you can sort them by frequency score:

You can also dive into the raw data, per company, to better understand the frequency score or to keep track of key feature usage over time:

How is the frequency score calculated?

The frequency score is primarily based on how many feature interactions a company has performed in a recent time period. Each score option has a dedicated time period. For “Daily” it’s past 2 weeks, for “Weekly” it’s past 4 weeks and for “Monthly” it’s past 12 weeks.

When calculating the score, we look each time period sequentially. The first period where the company has interacted with the feature more than 60% of the time, is the score the company gets.

For example, if a company has used the feature more than 60% of the time in the past 2 weeks - meaning equal or greater than 8 days out of 14 - the company gets scored as “Daily”.

Using individual time periods per score option means that companies will quite quickly jump from e.g. “Monthly” to “Daily” - or vice versa - when usage spikes or tanks, which is great for getting notified quickly about rapid usage changes in key accounts.

The frequency score is calculated for all companies in the “Using it” state, meaning all companies that are actively using the feature.

To learn more about how this all works in detail, with concrete examples, please read our help article.

More updates soon.

‍

We’ve given our UI a facelift and improved the layout. Most changes are subtle, but they add up and give a much more polished look and feel, and a more scalable interface.

Here’s three highlights:

App layout

Bucket is quickly getting increasingly powerful and configurable, which means we need to up the information density in the UI. We’ve therefore swapped the card-based layout for a flexible full-width layout that gives us more UI real estate. Furthermore, Bucket has a lot of tabular views with large datasets. When exploring multiple rows and columns, like in spreadsheets, you want as much space as possible, especially in the width. We liked the cards, but this change will enable us to grow and match our roadmap much better.

Adoption criteria

The sidebar holds a bunch of necessary feature information, but it’s all information that you only need to read once. For example, the new “Adoption criteria” section. The purpose of this section is to make it clear what data the feature report is based on. For both event- and attribute-based features, you’ll now see either event name or attribute expression in the top right, so you can instantly understand where the data is coming from and trust the feature report metrics.

Navigation polish

While the three main sections (Features, Companies, and Tracking) haven't changed, the nav links now have a better feel to it as the icon changes subtly on active state. The Bucket logo has also gotten a makeover (now rocking our purple brand color) and we've moved the app switcher down to the bottom of the screen and combined it with the user profile card.

More updates soon!

‍

It’s common to track feature usage with attributes. For example, at Bucket, we’re tracking if customers use our weekly reporting feature. We do this by passing an attribute called "weekly_reporting" via Segment’s "group" call.

This is great for ad-hoc analysis. It’s straightforward to query all companies that have "weekly_reporting" set to "true" or "false".

Unfortunately, though, attribute values change over time and its history is lost, which means we can’t know whether customers that aren’t using the weekly reporting feature ("weekly_reporting=false") have never used it, or once were using it, but turned it off.

Additionally, we can’t know when companies started using the reporting feature, and for how long they kept it on.

These things matter a lot. Is there a feature awareness problem or a feature retention problem?

…🥁…

Great news - now you can know! Today, we’re releasing support for attribute-based features. We think this is a really powerful addition to Bucket.

If you’re already tracking attributes in your application, this will just work™ as is. Starting today, Bucket automatically keeps track of all company-level attributes and will automatically track any attribute value changes over time.

(If you’re not familiar with attributes, check out our post from last week: When to track features with events or attributes)

Getting started

Let’s say we work at Slack and that we want to track usage of the third-party Slack apps.

Whenever a user logs into Slack, we’re tracking which company they belong to and send the latest attribute values for the respective company. In code, this looks like this:

Followed by

Whenever a user logs into Slack, we check which Slack apps the user’s workspace is using and report that via the "apps" attribute. As you can see in the snippet above, “Acme Inc” is only using the Datadog app at the moment.

As a developer, simply grabbing the current state and passing it on as attributes is a really convenient workflow. Let Bucket handle the historic changes.

Feature reporting

With the attribute data flowing in, we can start tracking an attribute-based feature on Bucket.

Let’s say we want to track usage of the Stripe Slack app. To do so, we simply create a new feature and define active usage as: "apps" contains "stripe".

Here’s the feature report:

We now know that 6% of our customers use the Stripe app. We can also see that 2% have then stopped using the Stripe app. We can also see when customers installed and uninstalled the app, and who they are. Neat!

How it works

Let’s look at what’s going on under the hood. When the "group" call comes in, per company, we check if the "apps" values match our "apps" contains "stripe" filter. For "Acme Inc" there’s no match, so we place "Acme Inc" in the "Never used it"-state on the feature report.

Later, someone at "Acme Inc" installs the Stripe app. Bucket notices the difference in "apps" value, and runs the filter again. This time, there’s a match! "Acme Inc" is now actively using the Stripe app.

On the Bucket chart, we can even see which week the shift happened.

Later, "Acme Inc" decides to uninstall the Stripe app. Bucket again sees a difference in the "apps" value and now automatically places "Acme Inc" in the "Stopped using it"-state.

Notice that Bucket didn’t just put "Acme Inc" in "Never" when the "stripe" value disappeared. That’s because Bucket keeps track of the history and is aware that "Acme Inc" once used the Stripe app.

On the same chart, Bucket charts when the customer churned from this feature.

We love this chart and use it all the time. For most features, we eagerly check the amount of purple bars after shipping a feature. New users, yay! In the following weeks, though, we also keep track of the amount of red bars, which is just as "exciting" and crucially important.

More updates next week.

We’ve added typeahead search to the various dropdowns in the Bucket UI. It’s now much faster to find the event you’re looking for when tracking a new feature.

Similarly, it’s now much easier to pick the Slack channel in which you want the daily and/or weekly Bucket feature reports.

More updates next week!