MQTTS : How to use MQTT with TLS?

In a previous article we presented how the MQTT protocol works. Here we use its secure variant: MQTTS. It is a good practice to use it, especially for embedded systems.

The goal is to establish an encrypted MQTTS connection between a broker and MQTTS clients present on the same machine. The options used for OpenSSL are a suggestion, so it is up to you to determine which ones are right for your needs exactly when you have successfully established the communication.

First, make sure you have installed mosquitto (broker and clients):

# For Debian/Ubuntu
$ apt-get install mosquitto mosquitto-clients

# For Fedora
$ dnf install mosquitto

Of course OpenSSL is also already installed on your Linux machine.

Example key and certificate generation with OpenSSL

Certificate Authority

We create a working folder and then generate the key and certificate from our own certificate authority :

Caution: The Common Name (CN) must not be the same as for customers.

$ mkdir certs
$ cd certs
$ mkdir ca
$ cd ca/
$ openssl req -new -x509 -days 365 -extensions v3_ca -keyout ca.key -out ca.crt
Generating a RSA private key
.....+++++
................................+++++
writing new private key to 'ca.key'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:FR
State or Province Name (full name) []:France
Locality Name (eg, city) [Default City]:Strasbourg
Organization Name (eg, company) [Default Company Ltd]:opeNest
Organizational Unit Name (eg, section) []:  
Common Name (eg, your name or your server's hostname) []:openest.io
Email Address []:contact@openest.io                                   
$ ls
ca.crt  ca.key
$ cd ..

Certificates for the MQTTS broker

Now we have a certificate. We can create the keys and certificates of the broker:

A private key is generated (without a password):

$ mkdir broker
$ cd broker
$ openssl genrsa -out broker.key 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
.................................................................................................................................+++++
.......................................................................................+++++
e is 65537 (0x010001)
$ ls
broker.key

Now, we create a signing request file from this key :

$ openssl req -out broker.csr -key broker.key -new
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:FR
State or Province Name (full name) []:France
Locality Name (eg, city) [Default City]:Strasbourg
Organization Name (eg, company) [Default Company Ltd]:opeNest
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:localhost
Email Address []:contact@openest.io

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
$ ls
broker.csr  broker.key

Now we can pass the Certificate Signing Request (csr) file to our validation authority:

$ openssl x509 -req -in broker.csr -CA ../ca/ca.crt -CAkey ../ca/ca.key -CAcreateserial -out broker.crt -days 100
Signature ok
subject=C = FR, ST = France, L = Strasbourg, O = opeNest, CN = localhost, emailAddress = contact@openest.io
Getting CA Private Key
Enter pass phrase for ../ca/ca.key:
$ ls
broker.crt  broker.csr  broker.key
$ rm broker.csr 
$ cd ..

MQTTS clients certificates

We’re practically doing the same thing again, this time to certify a client:

$ mkdir client
$ cd client
$ openssl genrsa -out client.key 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
..................................................+++++
..........+++++
e is 65537 (0x010001)
$ openssl req -out client.csr -key client.key -new
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:FR
State or Province Name (full name) []:France 
Locality Name (eg, city) [Default City]:Strasbourg
Organization Name (eg, company) [Default Company Ltd]:opeNest
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:localhost
Email Address []:contact@openest.io

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
$ ls
$ openssl x509 -req -in client.csr -CA ../ca/ca.crt -CAkey ../ca/ca.key -CAcreateserial -out client.crt -days 100
Signature ok
subject=C = FR, ST = France, L = Strasbourg, O = opeNest, CN = localhost, emailAddress = contact@openest.io
Getting CA Private Key
Enter pass phrase for ../ca/ca.key:
$ ls
client.crt  client.csr  client.key
$ cd ..

MQTTS broker configuration

By now you should have all these files:

$ tree .
.
├── broker
│   ├── broker.crt
│   └── broker.key
├── ca
│   ├── ca.crt
│   ├── ca.key
│   └── ca.srl
└── client
    ├── client.crt
    └── client.key

Now we can configure mosquitto: the broker. I present here a simplified but functional version of its configuration file :

$ sudo vim /etc/mosquitto/mosquitto.conf 

port 8883

cafile /home/openest/certs/ca/ca.crt
#capath /home/openest/certs/ca

# Path to the PEM encoded server certificate.
certfile /home/openest/certs/broker/broker.crt

# Path to the PEM encoded keyfile.
keyfile /home/openest/certs/broker/broker.key
require_certificate true

Port 8883 is the standard port for encrypted MQTTS connections (for unencrypted MQTT connections it is 1883). But you can use any port as long as you use the same port for clients.

Make sure it is not already running and then start your broker to use this configuration file:

$ sudo mosquitto -v -c /etc/mosquitto/mosquitto.conf 
[sudo] password for openest: 
1548955097: mosquitto version 1.5.4 starting
1548955097: Config loaded from /etc/mosquitto/mosquitto.conf.
1548955097: Opening ipv4 listen socket on port 8883.
1548955097: Opening ipv6 listen socket on port 8883.

Configuration d’un client MQTTS

In a second terminal you can now publish using mosquitto_pub :

$ cd client
$ mosquitto_pub -p 8883 --cafile ../ca/ca.crt --cert client.crt --key client.key -h localhost -m hello -t /world

Dans le terminal où vous avez lance le broker vous verrez que le message est bien arrivé:

1549402034: New connection from ::1 on port 8883.
1549402034: New client connected from ::1 as mosqpub|32006-alderan (c1, k60).
1549402034: No will message specified.
1549402034: Sending CONNACK to mosqpub|32006-alderan (0, 0)
1549402034: Received PUBLISH from mosqpub|32006-alderan (d0, q0, r0, m0, '/world', ... (5 bytes))
1549402034: Received DISCONNECT from mosqpub|32006-alderan
1549402034: Client mosqpub|32006-alderan disconnected.

Now you can create an additional key set for a second client and use mosquitto_sub to subscribe, using the same options (-p 8883 -cafile ../ca/ca.crt -cert client.crt -key client.key -h localhost).

Common mistakes:

Here are some problems you may encounter along the way with some suggestions to help you understand and correct them.

Error: A TLS error occurred.

$ mosquitto_pub --cafile ../ca_authority/ca.crt --cert client.crt --key client.key -h localhost  -m toto -t /toto
Error: A TLS error occurred.


# On the broker side:
1548955428: New connection from ::1 on port 8883.
1548955428: OpenSSL Error: error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
1548955428: Socket error on client <unknown>, disconnecting.

The probable cause is that the hostname of your broker does not match the Common Name (CN) you defined during the generation step of the .csr “request” file.

This error is also encountered when the Common Name of the Certification Authority is the same as that of the client. It is necessary that the CN of the certification authority is different from the clients.

peer did not return a certificate

# On the broker's side you may have this error:
OpenSSL Error: error:1417C0C7:SSL routines:tls_process_client_certificate:peer did not return a certificate

This means that the require_certificate option in mosquitto.conf is enabled, but the client does not send its certificate. Make sure that mosquitto_pub or mosquitto_sub uses these options: -cert client.crt -key client.key. Otherwise you can try to temporarily disable the broker’s require_certificate true option.

Check the contents of a certificate:

To debug your communication it is often useful to check the information contained in your certificates. Especially the Common Names. If you encounter problems you can easily check them with OpenSSL:

$ openssl x509 -in broker.crt -text -noout
Certificate:
    Data:
        Version: 1 (0x0)
        Serial Number:
            ab:2e:86:94:c4:18:1d:a0
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = FR, ST = France, L = Strasbourg, O = openest, CN = rootca
        Validity
            Not Before: Feb  1 10:23:19 2019 GMT
            Not After : May 12 10:23:19 2019 GMT
        Subject: C = FR, ST = Some-State, O = Internet Widgits Pty Ltd, CN = localhost
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:be:3a:7a:45:ad:f4:d5:33:c8:94:da:9e:38:5b:
                    d1:2b:13:26:c6:77:45:bc:a0:c3:f0:7a:56:8a:2d:
                    5b:7c:6c:75:54:a4:2b:1f:e3:ab:2a:c8:4f:74:5f:
                    47:7b:90:d2:39:1e:20:54:2c:55:61:41:81:42:72:
                    9d:ee:d1:ab:3e:3f:0f:fc:73:01:df:e0:3d:de:ae:
                    9c:d0:51:8c:dc:34:92:ff:53:97:c8:3c:86:c6:4d:
                    ae:7a:17:5e:65:38:21:f3:dc:1d:cc:28:a9:f9:ac:
                    3c:24:9a:01:f3:f8:75:41:81:81:23:7b:17:e0:55:
                    f4:1f:0f:b1:d0:f2:5e:de:d4:7e:72:bf:a9:8d:8f:
                    ab:4f:5f:38:3b:8c:49:cf:cf:2c:22:ba:0c:7e:7a:
                    01:65:08:1a:e9:1a:b6:d7:9c:93:bb:66:b4:83:65:
                    91:ae:89:e1:e3:11:21:33:f6:b2:4c:f3:c6:3d:b1:
                    7b:3a:28:6d:e7:36:b1:df:17:41:e1:e0:f3:d1:0c:
                    62:3a:e1:af:c2:62:47:30:6f:9b:28:4e:e4:aa:cb:
                    35:f0:f1:fd:17:f5:b5:cc:0d:99:b6:25:38:57:17:
                    1d:a6:f5:47:9c:76:8c:6b:32:aa:75:2e:f5:ff:b1:
                    25:77:31:fa:b9:5a:33:d4:18:8d:86:bf:9a:b0:c8:
                    1d:69
                Exponent: 65537 (0x10001)
    Signature Algorithm: sha256WithRSAEncryption
         22:08:e1:5a:5f:d9:33:7e:ca:f3:98:88:ba:ab:5d:34:b7:0e:
         58:b9:15:f8:57:e5:eb:17:ce:ed:80:0e:f6:90:7c:45:fc:1a:
         c3:1b:7a:29:8d:6b:e2:79:4a:23:ba:6b:74:6f:c5:1e:93:bd:
         ce:b2:7a:60:74:ba:c5:49:9a:48:67:92:c9:80:02:a0:43:c4:
         3a:90:b0:ae:aa:7d:49:ab:80:b8:64:e7:c7:0a:b2:84:90:01:
         78:79:cb:95:d3:88:4f:ae:57:6c:bd:d3:88:40:e1:3e:f7:b4:
         79:7f:a4:d8:01:37:9a:03:78:f5:ec:81:5e:1a:cb:bc:8f:da:
         eb:1d:2e:07:72:9d:d9:0e:9f:28:00:6e:4d:07:d8:5a:5c:ce:
         e7:bc:7d:30:44:77:fd:c0:c3:a8:4f:b8:e1:6f:62:32:ac:a8:
         0c:06:50:19:5a:02:1a:10:a0:55:bb:00:86:0f:d4:22:49:0c:
         14:58:65:96:00:e2:01:05:d2:fe:d3:49:c2:1f:e1:21:25:5b:
         82:c6:bd:01:ac:5a:e4:65:b8:4c:5c:e1:a8:ff:41:e8:74:ad:
         80:8e:da:37:de:8f:30:0b:b7:e0:b2:d2:e2:4b:db:f5:5b:37:
         10:57:ac:f7:fe:db:bd:44:01:5c:40:f1:80:a1:6e:9e:05:08:
         fc:0e:91:85

And with paho-c-mqtt?

As we explained in a previous article, we chose to use paho.mqtt.c for a m2m communication project involving embedded Linux development. Its API is well documented, here is an example of use that uses the files we generated at the beginning of this article:

MQTTAsync_create( &paho_handler, "ssl://localhost:8883", "paho_test_client", MQTTCLIENT_PERSISTENCE_NONE, NULL);

And the more specific part that allows you to configure the certificates :

         MQTTAsync_connectOptions conn_opts = MQTTAsync_connectOptions_initializer;
          MQTTAsync_SSLOptions ssl_opts = MQTTAsync_SSLOptions_initializer;       
                                                                                  
          conn_opts.keepAliveInterval = 10;                                        
          conn_opts.cleansession = 1;                                             
          conn_opts.onSuccess = _on_connect;                                      
          conn_opts.onFailure = _on_connect_failure;                              
          conn_opts.context = g_mqtt_client.paho_handler;                         
                                                                                  
          if(g_mqtt_client.certs_path != NULL ) {                                 

                  conn_opts.ssl = &ssl_opts;                                      
                  conn_opts.ssl->trustStore = "/home/openest/certs/ca/ca.crt";   
                  conn_opts.ssl->privateKey = "/home/openest/certs/client/client.key";
                  conn_opts.ssl->keyStore =   "/home/openest/certs/clientclient.crt";
                  conn_opts.ssl->ssl_error_cb = ssl_error_cb;                     
                  conn_opts.ssl->enableServerCertAuth = 1;                        
                  conn_opts.ssl->verify = 1;                                      
          }                                                                       
                                                                                  
         ret = MQTTAsync_connect( &paho_handler, &conn_opts);  

References

Here are some documents that were useful to us to encrypt the MQTT communication of our devices:

• The mosquitto man page
• The mosquitto-tls man page
• The documentation of the MQTTClient_SSLOptions structure

For the writing of this article we used version 1.5.4 of mosquitto (broker and clients) and version 1.3.0 of paho.c.mFor the writing of this article we used version 1.5.4 of mosquitto (broker and clients) and version 1.3.0 of paho.c.mqtt.qtt.

Let us know in the comments if you need any help !

MQTTS is the TLS secured version of the MQTT protocol. MQTTS means Message Queuing Telemetry Transport Secured. It is commonly used for machine to machine communication between embedded systems or IoT devices. As for MQTT, MQTTS can run on top of the TCP/IP, it allows users to configure the desired Quality of Service (QoS) to ensure data is delivered from one client to another.