Currently, OpenStack heavily relies on central broker for message distribution, like RabbitMQ. Theoretically it cannot scale-out very well due to restrictions of central messaging cluster. As a result, nova-cell has been proposed to improve its scalability, but other projects cannot benefit from this feature and moreover, this nova-only feature is still not production-ready.
According to our observation and experience on large-scale deployments, we prefer the only broker-less RPC driver supported by oslo_messaging library, 0MQ. Although the driver in the upstream still needs polish, the general architecture is reliable, resilient, efficient and more important, it's fully distributed (No SPOF).
This driver has been used in private & public clouds which we are in charge of. I would like to talk about the idea behind 0MQ driver, the lessons learned along the way, the current development status in the upstream and why brokerless messaging is the future.