FileLoadException: Could not load file or assembly ‘Microsoft.AspNetCore.Mvc.Core, Version=2.1.3.0’

Today I got an alert that somehow my Azure Traffic Manager endpoint stopped working with returning a 500. That means that both regions I set up behind the traffic manager were down. So I checked both endpoints and both came back with the same issue, the web app couldn’t start up at all. So that was interesting since there were no deployment or any other changes I did.

I enabled the developer error page to get the exact error and I didn’t expect to get this error:

FileLoadException: Could not load file or assembly 'Microsoft.AspNetCore.Mvc.Core, Version=2.1.3.0'

For some reason it could not find that assembly???

So I went to check the project and references, tried it locally and it ran totally fine. Googled a bit and found this:

https://github.com/Azure/app-service-announcements-discussions/issues/65

https://github.com/aspnet/AspNetCore/issues/3503

Apparently many have the same issue and it appears that somehow it was related with updates internally to Azure and the AspNet.Core version they are using. Its a bit annoying since it happened over night and were not changes I did, but here is the fix:

If you have this in your web app project file:

<PackageReference Include="Microsoft.AspNetCore.App" />

Change it to:

<PackageReference Include="Microsoft.AspNetCore.App" Version="2.1.3" />

Annoying but works….

Prometheus/Grafana in AKS – No data points for pods

Here is a quick one I ran into today after installing Prometheus/Grafana again to my AKS.

I was looking into the graphs for my Pods but it basically didn’t show any data, all other graphs like Nodes or Deployments were showing everything fine.

All I saw was “No data points” …

So I was trying to figure out why it was missing data and had a look into Prometheus metrics:

kubectl --namespace monitoring port-forward $(kubectl get pod --namespace monitoring -l prometheus=kube-prometheus -l app=prometheus -o template --template "{{(index .items 0).metadata.name}}") 9090:9090

What I saw was all kubelet targets not working with an 401 Unauthorized error, so I had a quick google and found that it is a known issue.

https://github.com/coreos/prometheus-operator/issues/926

From here its pretty easy to fix:

Get the kube-prometheus-exporter-kubelets yaml:

kubectl -n monitoring get servicemonitor kube-prometheus-exporter-kubelets -o yaml

Replace all https with http and apply the yaml again:

kubectl replace -f youryamlfile.yaml

It should show this then:

“servicemonitor.monitoring.coreos.com “kube-prometheus-exporter-kubelets” replaced”

After you have done this, give it a few minutes and check your Grafana again, it should show all the data points now.