Imaging you are running RabbitMQ in production, everything is fine…until you introduce a new product or feature which increases the amount of queues and connections you have. Suddenly your system is experiencing a drop in performance at peak time and you have no idea whats going. You log into RabbitMQ management website and under “Overview” the File Descriptors are showing a very low number of available descriptors, it is possible you started with the default number. If you are lucky you started with 4096 if not you might have started with 1024, which is incredibly low unless you are only running on a few queues and connections but in my case I was running with over 10k queues and over 500 connections. So is that number affecting the performance of your RabbitMQ? It certainly does, that number shows you how many file handles and network sockets RabbitMQ has available and you can imagine what happens if its running out of those, like mine was on Saturday afternoon. But the good thing is, you can increase them!
First lets check the limits.
Under “open files” you might see 4096, lets change that.
First we are increasing the maximum number of files in sysctl.
Add and Save:
fs.file-max = 100000
Now we load sysctl again.
Continue reading “RabbitMQ – Performance and File Descriptors”
I recently had to upgrade RabbitMQ to the latest version (3.7.8) on Centos VM’s. There is that lovely upgrade page on the RabbitMQ website itself https://www.rabbitmq.com/install-rpm.html which is kind of helpful but gave me a hard time because of those Yum repos and version locking. It didn’t want to work at all especially with locking Erlang to a specific version since RabbitMQ 3.7.8 only supports 21.0.x and there are newer versions online. (Which Erlang?)
So I decided to just download the packages I want to install myself and do that instead. Was easier than expected. Happy days!
Finding the packages
You want to go for the stripped down version by RabbitMQ.
Since I needed 21.0.x its: erlang-21.0.9-1.el7.centos.x86_64.rpm
#Shutdown RabbitMQ… just in case
#Import Public Signing Key
rpm --import https://github.com/rabbitmq/signing-keys/releases/download/2.0/rabbitmq-release-signing-key.asc
#Download Erlang package
rpm -Uvh erlang-21.0.9-1.el7.centos.x86_64.rpm(The "-Uvh" installs the package if it does not exist, otherwise it will just upgrade it and remove the old one. You can use "-U" if you just want to upgrade and remove the old package)
#Download RabbitMQ package
rpm -Uvh rabbitmq-server-3.7.8-1.el7.noarch.rpm
service rabbitmq-server start