Troubleshooting ASP.NET Core 500.30/502.5 errors

Startup errors in your ASP.NET Core middle-ware can cause real headaches. The 500/502 errors often don’t provide enough useful information to debug, often leaving us stumped for hours as we try to troubleshoot the root cause problem. While upgrading another piece of our site, we hit it again, but this time around, we let ourselves be sidetracked somewhat, and vowed to get to the bottom of the best ways to troubleshoot and prevent these errors going forward.

Troubleshooting

Let’s look at the most recent example, “HTTP Error 500.30 – ANCM In-Process Start Failure”. As you can see from the error below, there are no details, apart from statements that your application could not start – but we already know that, it’s why we are looking at an error message instead of the front page of our application.

Note that the error “HTTP Error 502.5 – Process Failure”, is very similar.

What these errors really mean, is that there is an unhandled error in your startup.cs or program.cs file, and it’s usually caused by a piece of middle ware. For us, it’s almost always related to Azure Key Vault access policies. When we run the app in Visual Studio on our laptop, our web page loads correctly, but as soon we deployed to our Azure web app, we receive the error. We verified this by removing our Visual Studio account from the Key Vault, and instantly received the same error locally. How can we troubleshoot an application in Azure when it works locally? Remember that due to the nature of these startup errors, they usually don’t show up in Application Insights either.

The best solution we’ve found, is to open the web app, and open “Advanced tools” to open Kudu.

Once in Kudu, we can open a “CMD” browser using the “Debug console” menu item.

Now we browse to our website wwwroot, “D:\home\site\wwwroot”, and run our application with “dotnet [your assembly name]”. This returns much more useful information. In the example below, we can see Key Vault did indeed return a forbidden permissions exception. Now we know enough to troubleshoot, and in this situation, we adding a missing access policy to allow our site to start working again.

Preventing

So how do we report these errors better in the future? As it turns out, startup error handling is purposely disabled. To show these errors, we need to add “CaptureStartUpErrors(true)” to our program.cs.

Now when we run our application, we see a much more useful error. Note that the ASPNETCORE_ENVIRONMENT variable must be set to “Development” to see this content, otherwise the regular 500.30 error will be shown. This is set to “Development” by default in Visual Studio, but “Release” by default in Azure.

Given that, we edit our startup code to show the useful error when deployed to our development Azure resource group, and the regular 500.30 error when in QA or production. We do this by adding an appsettings variable we will set to true in dev, but false in QA or Production. We also modify our deployment code to specifically set the ASPNETCORE_ENVIRONMENT variable to “Development” in Dev – leaving QA and Production in “Release”. Finally, we update our startup code, so that we can read in the the “CaptureStartErrors” variable from our appsettings, and use that to control whether or not we will see the errors.

Wrap-up

Today, we’ve learned about how to debug these difficult errors, and a better way to report on them in development.

References:

• 500.30/ 502.5 errors: https://odetocode.com/blogs/scott/archive/2018/07/16/7-tips-for-troubleshooting-asp-net-core-startup-errors.aspx
• Troublshooting Key Vault errors: https://stackoverflow.com/questions/51124843/keyvaulterrorexception-operation-returned-an-invalid-status-code-forbidden
• Application startup info: https://docs.microsoft.com/en-us/aspnet/core/fundamentals/error-handling?view=aspnetcore-3.1
• How to capture startup errors: https://docs.microsoft.com/en-us/aspnet/core/fundamentals/host/web-host?view=aspnetcore-3.1#capture-startup-errors
• Featured image credit: https://stackify.com/wp-content/uploads/2017/04/Best_Practices_for_Error_Handling_in_ASP-793×397.jpg