Python messaging: ActiveMQ and RabbitMQ

I found myself recently with a task of testing various Python messaging clients. In this article I’ll present my findings regarding performance and scalability on two Python clients connecting to ActiveMQ and RabbitMQ message brokers.

For ActiveMQ Python client, I used pyactivemq library version 0.1.0. It’s basically a Python wrapper for ActiveMQ-CPP library, allowing Python clients to communicate with the broker using both OpenWire (ActiveMQ specific binary protocol) and Stomp (Simple text-oriented protocol) protocols. Since it uses ActiveMQ-CPP, installation requires special attention and some of those details you can find in a related blog post.

As a RabbitMQ client I used py-amqplib version 0.6, since that’s currently the most natural solution if you don’t want to use frameworks like Twisted.

I used the latest broker versions, which means ActiveMQ 5.3.0 and RabbitMQ 1.7. Also, all numbers shown below are for tests executed on an “average linux desktop box” running Ubuntu 9.0.4.

Now let’s go testing.

Performance

#

The performance test I used is really simple: One producer, one consumer and one queue. Both producer and consumer tries to do their best in terms of performances and we’re not going to use any transactions. I used small text messages, with text Example message __num__. The full source code of tests can be found at http://github.com/dejanb/pymsg/. For every client (and broker) there are two Python scripts:

• test_receive_async.py - which starts an asynchronous consumer in a thread and samples consuming rates every 10 seconds
• test_send.py - which starts a producer in a thread and samples producing rates every 10 seconds

Both of these scripts try to get 100 rate samples, which means tests last for about 15 minutes.

Persistent messages

#

ActiveMQ 5.3.0 comes with the number of example configuration files that helps you configure it for different usage scenarios. One of those configuration files is called conf/activemq-throughput.xml and it is the one that should be used for high performance scenarios.

To start a broker with this configuration file, just execute:

bin/activemq xbean:conf/activemq-throughput.xml

Now you can run the consumer and producer with

python test_receive_async.py

and

python test_send.py

in separate console windows.

When ran like this, the client will use OpenWire protocol. The following diagram shows the result (the full output can be found here).

As you can see, after the initial spike, both producer and consumer have consistent rates of around 1300 msg/sec.

To repeat the same test, now using Stomp wire protocol, we’ll execute the scripts with one additional parameter, stomp (of course, a clean broker start is needed)

python test_receive_async.py stomp

and

python test_send.py stomp

The results for pyactivemq client using Stomp protocols can be found here and the diagram visualizing those data is shown below.

Again, after the initial glitches, producer and consumer settles on a bit more than 1150 msg/sec. This is an interesting result that shows that with persistent messages the limit is not in the network protocol, but the ability of the broker to successfully persist messages.

Now let’s try do the same with RabbitMQ and py-amqplib client. The output with the result samples can be found here and they are visualized on the following diagram.

On the first look you can notice the high message producing rates, starting with more than 3000 msg/sec and slowly decreasing toward 2000 messages per second. On the other hand the consumer is considerably slower at rates around 1000 msg/sec. This situation causes more and more messages being stored in the broker, when it finally crashes after 12-13 minutes. I tried setting Memory-based flow control, but without much success.

The high value on a producer side have one more important drawback regarding reliability. The AMQP protocol send operation is strictly asynchronous, meaning that producer does not have any confirmation from the broker that the message was actually queued. So what I observed is that the number of messages sent shown by the producer and messages stored in the broker are not in sync. You can check this by starting the producer that sends a large number of messages for some time and check the number of messages in the broker. You’ll notice that messages arrive to the broker, “long” after the producer has finished its job. So if broker crashes in the meantime, all messages producer thought it sent are lost. I didn’t test this with transactions, so I’m not sure if the behavior is somewhat different in this case. My guess it that it is and that is one of the things that I’d like to test further.

Also, RabbitMQ allows you to set the “returned listener” on the channel which can be used by the broker to return messages that were not routed (if the mandatory parameter is used during publishing) or could be immediately delivered (if the immediate parameter is used during publishing). py-amqplib library is currently missing this functionality, which can be another reliability issue for concern.

So generally, it seems like RabbitMQ have a very high producing rate (sacrificing reliability), while consuming rates are pretty standard and a bit lower than those seen by ActiveMQ consumers.

Non-persistent messages

#

Now let’s try to do all that again with non-persistent messages. To modify pyactivemq producer to send non-persistent messages, we have to uncomment the following line

#self.producer.deliveryMode = DeliveryMode.NON_PERSISTENT

in perftest.py library.

Now we can rerun our tests. For OpenWire protocol results now quite different

showing that the whole throughput of non-persistent messages through the broker is around 3000 msg/sec.

Similarly, the results for Stomp are visualized in the diagram below.

This shows that at around 2000 msg/sec the limitations of Stomp text-oriented protocol kicks in and limit the throughput.

To modify RabbitMQ client to send non-persistent messages, we have to comment the following line

msg.properties["delivery_mode"] = 2

in the PerfProducer class.

Results now look like this

which is pretty much the same as in the persistent case. This is probably due to the nature of queuing and persisting messages in the broker discussed above. The only difference is that now the broker keep the rates at steady numbers of around 3200 msg/sec for producer and 950 for consumer.

Scalability

#

I also tried a simple scalability test against two brokers: try to send a message to as many queues as you can. The test source can be found in the appropriate test_scale.py file for both py-amqplib and pyactivemq libraries.

When you run appropriate

python test_scale.py

against RabbitMQ you’ll get that the message was sent to the around 32000 queues before it crashes, which is very good result.

ActiveMQ, again, comes with the predefined configuration file which should be used if you want to scale your broker (conf/activemq-scalability.xml). As it’s stated in the configuration file, there a couple of minor changes needed to be made to the startup script in order to achieve maximum scalability. We’ll use these

ACTIVEMQ_OPTS="-Xmx1024M -Dorg.apache.activemq.UseDedicatedTaskRunner=false"

properties in order to give broker a bit more memory and turn of dedicated task runner.

When started with the configuration like this, the

python test_scale.py

shows that message was sent to impressive 64000 queues before it crashed.

We can try the scalability of Stomp protocol as well. In order to do that, we should add

<transportConnector name="stomp+nio" uri="stomp+nio://0.0.0.0:61613"/>

to the list of available connectors.

Now we can start the test again

python test_scale.py stomp

and we’ll get the result very similar to the one of the default OpenWire connector.

Conclusion

#

To conclude, both ActiveMQ and RabbitMQ are decent brokers that will serve their purpose well in normal conditions, but put to their extremes in terms of throughput, scalability and reliablilty, ActiveMQ currently outperforms RabbitMQ for messaging usage in Python. All tests performed here are done with honest intentions of providing realistic results. If you spot any problems or have any ideas what more can be tested (transactions, etc.), please let me know.