Implementing Feature Flags

Last week, we learned about the basics of feature flags. This week, we are going to create our own, simple and custom implementation. Our main application will call our feature flags service, asking for the state of particular feature flags based on their environment – for example, if we were implementing a new menu and wanted to use the feature flag in dev, the dev website will ask if the dev version of the new menu feature flag is enabled.

• Our feature flags application will be it’s own application, in it’s own separate data source, web service, and web site, as it needs to communicate with all of our environments.
• We will use Azure Storage tables for the data source, storing JSON documents to describe the details of the feature flag
• We will create an ASP.NET Core API to act as our service and expose the feature flag status to our SamLearnsAzure application
• We will create an ASP.NET Core website, to act as the administration application
• We will also track the number of hits each feature flag will receive and the last date the feature flag was used. This will help us to track feature flags that become stale and can be removed, assisting with tracking technical debt
• Limitations:
  • In the short term we are assuming there is just one application and the three environments of Dev, QA and Prod. In the future we could add functionality to make this more dynamic, but for this demonstration, hardcoding it works.
  • While the feature flags service api is rest based and could accept client side requests, we will only setup server side feature flags. If we need client side calls in the short term, we can pass the server side result to the client side.
• The diagram below shows the current architecture:

Administration

We build out a simple UI, where we can create new feature flags, and set their on and off state for each dev, qa, and production environments.

We can also see the number of times each feature flag was used, and when. The time is important, as after we have finished with a feature flag, we need to be confident that the flag isn’t still being used. In the screenshot above, on the “NewMenu” feature flag, we can see that it was last used 162 days ago – giving us some confidence that we have removed the feature flag from our code some time ago, and it can be deleted from our feature flag manager.

Web service

The web service is relatively simple. It has a few functions in the controller to get all feature flags, get individual feature flag details, update flags, and delete flags, and check the status of a feature flag for a particular environment, all stored as JSON documents in our Azure Storage table account. Here is an example of the JSON document for the “NewMenu” feature:

Demo

Finally, we update our website to call the feature flag service. As ab example, supposed we write this code below to divide a variable by zero, and wrap this in a “DivideByZero” feature flag. If the flag is enabled, then we divide by zero, creating a critical error. If the flag is off, we don’t run this code. Fun fact: The compiler is surprisingly good at not letting you divide by zero, you need to assign the numbers to variables.

In our administration tool, we enable the feature flag in our QA environment.

When we browse to that page on the QA site, we receive a critical error. When we go back to the administration tool and disable the same flag, the error goes away.

Ask yourself, even if you know the exact issue, and can change an offending line of code in seconds, how quickly does your current process allow you to address a production issue? Even our environment, from code to production, takes about 75 minutes to completely and comprehensively build, test, and deploy our changes (through successive dev, qa, and production) environments. In our old monolithic release world, in this situation, we would have to scramble to hot fix, (usually without a complete regression test), or rollback our release, both of which could often take 2-6 hours, if not more.

Based on this, you can being to see the huge advantages feature flags offer in these situations. It separates functionality releases from deployments, and puts more control into the business, who can change the flag quickly when everyone in the organization, (marketing, support, etc), is in their best position and ready to launch a feature.

Alternative Feature Flag Systems

After our post last week, we received a question about why we didn’t use an existing feature flag system. It was a great question, as there are a number of existing alternatives out there worth having a look, before you build your own custom system, such as:

• LaunchDarkly: really the best on the market today. Being a comprehensive SaaS solution, it does unfortunately have a cost that rules it out for this project. In our professional experience and opinion, LaunchDarkly is the best feature flag management tool out there, and should be used for enterprise applications
• Microsoft.FeatureManagement: While still in preview, this new product has a lot of potential. Andrew Lock wrote a fantastic blog series about this namespace, and we believe it has a lot of potential. We elected to not use it for this example.

There are many other open source solutions our there, make sure you carefully look at your requirements and understand what features you need.

Wrap up

We’ve proven that a custom feature flag system can be created relatively quickly, cheaply, and easily, (we built this system in roughly 7-8 hours). There are a lot of features we could add, (and may add later), to extend our system, but for now, we’ve added a useful service that can help us deploy features more confidently in the future.

References

• Getting started with Azure Storage Table access in C#: https://docs.microsoft.com/en-us/azure/visual-studio/vs-storage-aspnet5-getting-started-tables
• How to use “Partition Key” and “Row Key” in Azure Storage Tables: https://blog.maartenballiauw.be/post/2012/10/08/what-partitionkey-and-rowkey-are-for-in-windows-azure-table-storage.html
• LaunchDarkly: https://launchdarkly.com/
• Microsoft.FeatureManagement: https://github.com/microsoft/FeatureManagement-Dotnet
• Andrew Locks amazing Microsoft.FeatureManagement walkthrough: https://andrewlock.net/alternatives-to-microsoft-featuremanagement/
• Esquio, another open source ASP.NET Core feature flag system currently targeting ASP.NET Core 3.0: https://github.com/xabaril/esquio
• Featured Image: https://s0.geograph.org.uk/geophotos/03/94/57/3945709_72d534b6.jpg