MQTT Pub/Sub

MQTT is an over-the-wire protocol for interprocess-communication. Developed by IBM, it is used in Microsoft Messenger, and supported by AWS and Microsoft Azure. There is a rich ecosystem of 26 MQTT brokers, and innumerable client applications. It is particularly popular for iOT applications.

MQTT is an over-the-wire protocol for interprocess-communication. Developed by IBM, it is used in Microsoft Messenger, and supported by AWS and Microsoft Azure.It is an Oasis standard. There is a rich ecosystem of 26 MQTT brokers, and innumerable client applications. It is particularly popular for iOT applications. It is particularly popular for iOT applications, but has much broader applicability. I am using it as a hierarchical chat servers.

MQTT is a light-weight protocol.
It is a very simple, and clean but powerful abstraction of how to build a distributed application. Users can subscribe to, or publish to a tree of topics (channels). Applications can send regular messages, guaranteed-delivery messages, and deliver-exactly-once messages. Security determines who can publish and who can subscribe to a channel. History tells you about past messages on any topic. Presence tells you who is subscribed to a channel. Connections support a “Last Will and Testament”. When your connection disconnects your last will and testament is published on the specified channel.

MQTT is fun to use. I am using it to run numerous chat rooms, one for each talk at conferences which I attend. And shortly one for each library listed on Awesome Python and other curated lists.

Compared to developing an application in AsyncIO, MQTT is trivial to use. Indeed, the Python MQTT clients use AsyncIO. But rather than write an entire real-time server oneself, it is much faster to just write an MQTT client, reusing one of those wonderful, mature, reliable MQTT brokers. And reusing the mature, tested and reliable MQTT client libraries.

Slides. You can see my rapidly improving collection of MQTT slides at PythonLinks.info/presentations/MQTT.

1. Title

2. IBM, FaceBook, Microsoft Azure and AWS use it.

2.Limitations of Point to Point communitcatinos. N * (N-1) IP addresses to remember.

3, A single Chat Server.

1. A tree of Pub/Sub topics. Sales can have their channels, support can have their channels, Finance can have their channels. Everyone only sees the content on the channels where they have security clearance. No one else can see finance channels.

2. Quality of Service. Regular Delivery, Guaranteed Delivery. Exactly Once Delivery.

3. Security. Talk about security on a branch of the Tree. Permission to publish, permission to read, permission to access presence and history.

4. Filtering. It is possible to use topics and wildcards to only subscribe to certain channels. I will give the typical domestic thermometer example to illustrate this.

5. History. Obviously we need chat room history by topic.

6. Presence Just show the demo for my chat application.

7. 26 different brokers. https://github.com/mqtt/mqtt.github.io/wiki/server-support

8. Innumerable Clients.
   https://github.com/hobbyquaker/awesome-mqtt

9. Show the Python Code for a simple app. https://www.eclipse.org/paho/clients/python/#

10. A web page, quickly becomes stale, keep it fresh with MQTT updates.

11. MQTT works particularly well with the ZODB. Both support Hierarchy.

12. Demo of my MQTT application. It is a tree of Python Topics. Of particular interest is the tree of talks at PyCon USA. You can see the best talks, sorted by vote count. There is a chat page for every talk. As of May 20th, the (unreliable) dev site is at m.pythonlinks.info. Within a week the production site will be at PythonLinks.info

13. Scalability. Python only runs one thread at a time. Multiple MQTT brokers are multithreaded. Some are even cluster able. MQTT scales your distributed apps, much better than Python can.

14. Conclusion MQTT makes real-time apps easy to build, and scale. It is fun to use. Give it a try. There are multiple free MQTT brokers online.

ADDITIONAL NOTES

MQTT is generally targeted at the IoT market. Which is why I submitted it to this category. Frankly I think it makes sense for any real-time application. Since I do not have a lot of sensors to connect with, I am using it for a chat application, but the ideas all apply to IoT as well. So I included a thermometer example to make it a bit more relevant to this group.

I am proposing two related talks, one on the ZODB, one on MQTT. The ZODB talk is mature. It has been given at multiple PyCons. The important audience questions have been answered. The rooms are always full. The MQTT talk is much newer, but it is a much hotter technology. While the two talks stand on their own, it is really best to see both talks. There is a huge synergy in using the ZODB with MQTT. Both support hierarchy. Both do something that the other is weak at. They work best together.