To start with I will give you a little context about the ELK setup I was working on and how it was being used to better understand the issue I will describe I ran into.
I’m currently using the ELK stack for logging purposes, so I dump all log messages from my code onto a RabbitMQ and from there it will get consumed by multiple Logstash server dumping the messages into Elasticsearch. Those Logstash server are setup to push to different indexes in Elasticsearch, mainly to differentiate the log messages.
So imagine there is an error log queue and an audit log queue in RabbitMQ, you now have a Logstash server to consume from both queues and depending on the message/queue it should push to either the error index or the audit index.
So each Logstash server had two config files, one for every queue to consume from. Every config file had the RabbitMQ input plugin, a custom transformation for some fields and the Elasticsearch output plugin.
Today I got an alert that somehow my Azure Traffic Manager endpoint stopped working with returning a 500. That means that both regions I set up behind the traffic manager were down. So I checked both endpoints and both came back with the same issue, the web app couldn’t start up at all. So that was interesting since there were no deployment or any other changes I did.
I enabled the developer error page to get the exact error and I didn’t expect to get this error:
FileLoadException: Could not load file or assembly 'Microsoft.AspNetCore.Mvc.Core, Version=22.214.171.124'
For some reason it could not find that assembly???
So I went to check the project and references, tried it locally and it ran totally fine. Googled a bit and found this:
Apparently many have the same issue and it appears that somehow it was related with updates internally to Azure and the AspNet.Core version they are using. Its a bit annoying since it happened over night and were not changes I did, but here is the fix: