ESP8266 Workship - Wednesday 4th February - MQTT + Low Power

Following on in our forthnightly ESP8266 Workshop theme, I guess we should have an MQTT workshop.

As explanation, MQTT is one of the more important Internet-of-Things protocols. It can run on all sorts of devices ranging from mobile-phones, desktops, servers and now the ESP8266.

Everybody is welcome! We’re getting more and more people along and that is making the workshops better every time. Of course, sometimes they’re a bit random because people have different needs and we’re happy to bring newcomers up to speed. It’s all fun though.

I can give a brief introduction to MQTT and show how it works and why it’s so cool.

1 Like

Now in the Calendar - I’ll be there to open at 6.30

I think it’s worth pointing out in the thread that the reason why MQTT is interesting is because it solves the problem of how you communicate updates between your micro boards and your computers and back again or your micro boards and other micro boards.

Will be a good night I think.

A couple of things we should get done before this night to make it useful:

  • Sort out a wireless router that works properly with the ESPs.
  • Setup a MQTT Broker within the network at MHV.

I don’t know what we’ve got at MHV currently in terms of “server” (i.e. box that’s on all the time)?

There’s a lot of scope for some really cool stuff with these device and MQTT at the space. Individuals could work on publishing various pieces of information from the space (e.g. temperature, PIR sensors, power consumption, etc, etc). Whilst others could be dreaming up ways to consume this info. With the ESPs being very easy to configure as both publisher and/or consumer of messages.

Agreed!

aspirin (10.0.0.11) now has an MQTT broker running on all interfaces on the default port (1883). It also has the standard tools and python bindings installed so if you have shell on that you can use those tools, but you can also us it by connecting to it from outside.

Do you know what is needed in terms of wireless access point that the network at MHV doesn’t provide?

Cool, thanks for sorting that out.

Problem is we have no idea yet. David and myself were both connecting to the MHV wireless and it was causing the devices to reset as soon as they connected. No idea why, haven’t observed this behavior on any other network I’ve tried on yet.

Could just be a firmware issue with the nodemcu version we were using.

At home I also found I couldn’t connect to a router with a ‘-’ in the ssid. So I might try the same SSID and password on a router at home and see if that can replicate the problem.

Problem is we have no idea yet. David and myself were both connecting to the MHV wireless and it was causing the devices to reset as soon as they connected. No idea why

I’ve seen it where there’s a duplication in soft MAC address. However I actually checked for this when I bought 10 modules and they all came back with different MAC addresses.

Something like an rpi might be good for a mqtt, but if something already is there we could use that. I will look into it more.

On a related matter: I want to save power, wakig up regularly and publishing some data.
I am looking at

node.dsleep(us)

where us is sleep time in micro seconds. The module drops from 0.12A to 0.00A for the nominated time, but then only half wakes up (to 0.05A) and does not talk to the console anymore.

I see this node in the doco:

This function can only be used in the condition that esp8266 PIN32(RST) and PIN8(XPD_DCDC) are connected together.

I do not have an XPD_DCDC pin on the esp-07. I do not see it on the esp-12 (which is on the dev board) either.

I am waiting for delivery of this module
http://www.aliexpress.com/item/-/32254810086.html
which seems to be included in this dev board
http://www.aliexpress.com/store/product/-/733580_32256680637.html
and I still do not see the required pin here…

I also tried wifi.sleeptype(type_need) which did not change the power usage that I see.

The espressif doco makes bit statements about the low power capabilities

ESP8266_Specifications_English.pdf

from:
https://drive.google.com/folderview?id=0BwK3EhAfht8uWTdBdG55NEFCakE&usp=sharing

Does anyone have any idea how this works? More known doco?

I started with the schematic here which is apparently for the ESP-1 which I have:

Using a DMM with fine probes I went around the board and pretty quickly came to the conclusion that it’s not the same. However, the reset pin is connected to the pin labeled GPIO16 on the ESP-1.

The XPD_DCDC pin was unconnected.

I repeated what you did and got the same results more or less (current measurements a bit different, but the same pattern.)

I hooked up the XPD_DCDC pin to the EXT_RSTB pin as per the following.

Now all works as expected, call node.dsleep(us) and the current consumption drops to 360uA. At the end of the time period, the ESP is reset.

Next time we meet I can bring the soldering gear in and perform the mod to boards if anyone wants.

  1. My current measurements were with a basic USB V/A dongle. Connecting my DMM shows the esp using 71mA with wireless enabled, dropping to 430uA in dsleep, then waking up to 14.3mA.

  2. The esp-07 has metal casing, so no access to the chip pins :-(

  3. The other module I mentioned above (AX #32254810086) seems to have a IO16 listed, and is elsewhere marked as XPD, so should be good.

I consider this feature absolutely critical for use of the esp on battery.

Eyal

By zooming in and looking in closely at the art work on this page:
http://www.alibaba.com/product-detail/Wireless-WIFI-Module-ESP8266-ESP-07_60091055084/showimage.html

The Reset pin is obviously brought out, and the XPD_DCDC is brought out and labeled GPIO16. I reckon you should just be able to join those two pins together and you’ll be in business.

But of course, I should have looked closer. Thanks.
[later] It works.

For those wondering how the above discussion about sleep mode applies to the NodeMCU Devkit, here’s a pinout diagram:

[So what, now it is a "Workship? More like a “Worship” I say…]

My current thoughts on using MQTT:

  1. I note that node.dsleep() does not put the module into sleep immediately. I put some printouts following it and these were still executed. It was inside init.lua in this case.

[later] I now confirmed that the dsleep waits for the current program to exit to the prompt.
I uploaded this program as 'timer.lua':

print ("Low Power test");

wifi.setmode(wifi.SOFTAP);
wifi.ap.config({ssid="eyal-esp8266",pwd="12345678"});
print ("Node MAC: " .. wifi.ap.getmac());
print ("Node  IP: " .. wifi.ap.getip());
-- print ("Running "..tmr.time().." seconds");

for i = 1,5 do
        print (i.." Waiting...");
        tmr.delay (1000000);
end

print ("Sleeping 5 seconds");
node.dsleep(5000000);

for i = 1,10 do
        print (i.." Still here?...");
        tmr.delay (1000000);
end

and then ran it:

> dofile ("timer.lua");
Low Power test
Node MAC: 1A-FE-34-9C-DA-B6
Node  IP: 192.168.4.1
1 Waiting...
2 Waiting...
3 Waiting...
4 Waiting...
5 Waiting...
Sleeping 5 seconds
1 Still here?...
2 Still here?...
3 Still here?...
4 Still here?...
5 Still here?...
6 Still here?...
7 Still here?...
8 Still here?...
9 Still here?...
10 Still here?...
¦¦,?<p¦¦¦¦-z¦¦</¦r02[¦

NodeMCU 0.9.5 build 20150127  powered by Lua 5.1.4
>

As is clear, after the sleep was requested the program continued to run for 10 more seconds. I then timed it to be 5 seconds before the restart.

  1. The wakeup looks like a cold start, so how do I keep values (e.g. message sequence)across wakeups? I could keep stuff in a file, but is there an easier way to store it in flash using some variable attribute (to make it persist)? I guess it does not matter much as settings and files all end up in the same flash device.

  2. I also want to time-stamp my (published) readings. Does the broker keep accurate published date/time? If not then I will need to add a clock to the mix (no big deal really). I am still reading on MQTT…

It’s not generally a broker function to log data, albeit some have plug-ins to do that. From what I’ve read, using a separate client (can be a another service running on the same broker server) that subscribes to what you want to log and inserts it into an SQL database seems popular. That would be a very simple program to code up, I’ve already got a client working in C# in only half a dozen lines of code, but could be done in something more platform agnostic just as easily. There are already some open source ones out there as well.

There’s a discussion on the topic linked below, in short hivemq don’t recommend doing this, instead recommend the plugin method. But that depends on whether the broker supports it, hivemq isn’t free. For small scale stuff, I don’t really see a major disadvantage to the separate logger client.

I’ve setup a Raspberry Pi with Mosquito MQTT broker. Installed SQLite3, and the Paho MQTT python library.

Created a database with a Temps table, then patched a couple of examples of the Paho library and SQLite3 to make a logger. It’s a complete hack of a program, but still works well.

import paho.mqtt.client as mqtt
import sqlite3


def add_temp_reading(zonestr, temp):
    # I used triple quotes so that I could break this string into
    # two lines for formatting purposes
    curs.execute("""INSERT INTO temps values(date('now'),
        time('now'), (?), (?))""", (zonestr,temp))

    # commit the changes
    conn.commit()

# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))

    # Subscribing in on_connect() means that if we lose the connection and
    # reconnect then subscriptions will be renewed.
    client.subscribe("Home/+/Temp")

	
# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
	add_temp_reading(msg.topic, float(msg.payload))
    print(msg.topic+" "+str(msg.payload))
	
conn=sqlite3.connect('mydatabase.db')
curs=conn.cursor()
	
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message

client.connect("192.168.0.3", 1883, 60)

# Blocking call that processes network traffic, dispatches callbacks and
# handles reconnecting.
# Other loop*() functions are available that give a threaded interface and a
# manual interface.
client.loop_forever()

# Will never actually get here...
conn.close()
2 Likes

I am playing with mqtt and have some problems. I assume that I do not properly understand the mqtt module or how lua runs from a file.

I uploaded a file ‘t’:

m = mqtt.Client("esp-07", 600);
m:connect("192.168.2.7", 1883, 0, function(conn) print("connected") end);
m:publish("stats/temperature/esp-07","30C",0,1,function(conn) print("sent 30C") end);
m:close();

I first ran the above commands individually from a terminal:

> m = mqtt.Client("esp-07", 600);
> m:connect("192.168.2.7", 1883, 0, function(conn) print("connected") end);
> connected
m:publish("stats/temperature/esp-07","30C",0,1,function(conn) print("sent 30C") end);
> sent 30C
m:close();
> m=nil

As one can see, it worked and the correct messages were logged.
In another terminal I was running a ‘sub’ and got the expected value:

$ mosquitto_sub -i e4 -h 192.168.2.7 -v -t 'stats/temperature/esp-07'
stats/temperature/esp-07 30C

I now ran the same commands from the file:

> dofile("t")
>

No messages here and none showing on the ‘sub’ terminal.

I now manually did:

> m:connect("192.168.2.7", 1883, 0, function(conn) print("connected") end);
> sent 30C
connected

Interestingly, not only did I get the expected ‘connected’, but the older missing 'sent’
message now showed up. The mqtt server did not show a new published value.

I proceeded and it still works OK from the command line:

m:publish("stats/temperature/esp-07","30C",0,1,function(conn) print("sent 30C") end);
> sent 30C
m:close();
> m=nil
>

This time a new published value showed up as expected in the ‘sub’ terminal.

Inserting delays between the commands in the file made no difference.
Inserting 'print ("…")'s in the file does show, so I conclude the statements are executed.

It was a good meeting for me, I got things mostly working. But not fully.

My current problem is as simple as establishing the network connection at init time.
My init.lua (see below) simply checks for wifi.sta.getip() to be set. If it is nil then it retries 10 times with one second delay.

The problem is that after a reset it never shows an IP. Here is a log of a test:

=== toggle the reset pin

$J¦C¦¦:6¦n¦:#¦¦@b¦¦$¦¦¦¦

NodeMCU 0.9.5 build 20150127  powered by Lua 5.1.4
no IP yet...
no IP yet...
no IP yet...
no IP yet...
no IP yet...
no IP yet...
no IP yet...
no IP yet...
no IP yet...
no IP yet...
no IP
done

=== no IP seen. Run the init script again:

> dofile("init.lua")
no IP yet...
no IP yet...
no IP yet...
no IP yet...
no IP yet...
no IP yet...
no IP yet...
no IP yet...
no IP yet...
no IP yet...
no IP
done

=== still no IP. I can repeat this many times with no IP. Now directly check for IP:

> = wifi.sta.getip();
192.168.2.30    255.255.255.0   192.168.2.7

=== OK, so there is an IP at the command line. Now run the init script again:

> dofile("init.lua")
Node MAC: 18-FE-34-9C-DA-B6
Node  IP: 192.168.2.30
done

=== now there is an IP where there was none earlier.

It seems that the environment when init runs is not the same as the command line. I do not yet know how to get a connection at init time, something that I expected to be a trivial requirement.

Clearly I do not yet understand the unusual execution model of this lua…

Here is my simple init script:

user, pass = "eyal", "xxxxxxxxxxxxxx";
 
ip = wifi.sta.getip();
if nil == ip then
        wifi.setmode(wifi.STATION);
        wifi.sta.config(user, pass);
        wifi.sta.connect();

        for n = 1, 10 do
                ip = wifi.sta.getip();
                if nil ~= ip then
                        break;
                end
                print ("no IP yet...");
                tmr.delay (1000000);
        end
end

if nil == ip then
        print ("no IP");
else
        print ("Node MAC: " .. wifi.sta.getmac());
        print ("Node  IP: " .. ip);
end

print ("done");

I think it comes down to the event model again, this is how I do it.

The important bit begin, between each call to wifi.sta.getip(), I’m returning control to the framework to do it’s background work. I think when you just put a delay in, you’re stalling everything, not just your program but the wifi stuff as well.

function testConnection() 
     if wifi.sta.getip()== nil then 
          print("IP unavaiable, Waiting...") 
     else 
          tmr.stop(1)
          print("Config done, IP is "..wifi.sta.getip())
          collectgarbage()
		  StartProgram() -- StartProgram() is the start of the real program
     end 
end

print("Setting up WIFI...")
wifi.setmode(wifi.STATION)
wifi.sta.config("xxx", "xxx")
wifi.sta.connect()
tmr.alarm(1, 2500, 1, testConnection)

Pardon the dumb/slightly OT question, where is the best place to obtain one
of these? (I have a little bit of a project in mind)

I see that seeedstudios has them but I noticed on the wiki for them that
there is like, 5 or 6 different “”“mainstream”"" layouts and countless more
other spins

Ta,
Max