Agree, it was just the basis of my initial decision some time ago WRT MSP430.
Have a look around, see if any devices take your fancy.
The Energy Micro devices are quite nice, with low power modes having similar current draw to the MSP (i.e. sub uA with clock running.) But with plenty of peripherals and processing power. The CM3 variants are easily available in low qty.
We should list the required features and then pick the best controller. Without some sort of such cost function it is difficult (for me) to make an intelligent decision.
very low power (speed probably not important)
hware uart (or two)
good RTC
enough gpio
enough ram
enough flash
open source development tools
If we expect it to take care of all the devices while the esp is ājustā a WiFi bridge then the demand on swr (ram, flash) and hwr (i2c, spi, adc etc.) is much higher than if it is only a controller keeping time and turning things on/off.
I think that planning for it to be a minimal controller is the way to go.
very low power (speed probably not important)
How low is very low, and in what state?
To determine this, we need a basic use case in which we know what the consumption of the ESP is going to be. Otherwise we can be optimising where there is no real gain to be had. Have you got a good handle on how much power the ESP takes to boot up, send a UDP packet, then shutdown?
hware uart (or two)
Are there any other peripherals that are important? ADC, other serial interfaces, etc?
good RTC
The RTC itself is an easy requirement, pretty much every small modern uC will have the capability to take a 32k crystal and do clock/calendar functions in software. Some do those functions in hardware, but itās no big deal either way.
Possibly more troubling is variable storage across sleep. Some (e.g. MSP430) keep the RAM alive in low power states, where as some (e.g. STM32) need the RAM to be powered down to get the sleep current low enough.
enough gpio
enough ram
enough flash
How much is enough for each of these things?
open source development tools
How open is open enough? A lot of vendors provide tool chains based on Eclipse and the relevant variant of GCC. In practically all cases you can build your development tools from scratch. There is a GCC for ARM, MSP430, AVR, etc. For small uCs itās rare you see binary blobs, only where thereās some specific licenced functionality (e.g. AVRs touch library, or at least was, havenāt used it in years.)
Yes. The fast modules (esp-01, -07) use 100mAsec doing the whole dance in 1.5s, the slow ones use 400mAsec in 4-5s.
Not is we do the minimal timekeeper role. The sky is the limit if we want to be in charge of sensors etc.
This is not critical for all ram. A timekeeper needs to maintain an event list which can be stored in flash/fram. My temp app on the esp keeps a few words in RTC memory which provides 512 (4-byte) words for user storage.
I even have a version that sends state info to a (WiFi) server and then reads it after wakeup. I do not think that keeping program ram alive is very important but I expect it can make programming a bit simpler.
Depends on the role. For a minimal controller the bar is rather low, but I hate to program where ram is so tight that a 2 or 3 level call stack breaks the app. With some very small uCs that have only 128/256/512 bytes even constructing a message can be painful.
The more the better but it is not a deal breaker for me. Others feel more strongly about this so it will make this project attractive to a wider audience.
So lets work on a use case to let us define our power requirements. Please correct any of this that doesnāt make sense or isnāt along the lines of what you were thinking.
We want to measure temperature at an interval of 1 min, and log the results to an SQL database. Weāll use a DS18B20 sensor for temp measurement and an ESP as the network connection. The ESP will use UDP, and confirmation of send will not be required.
Some options:
Use an ESP to do everything.
Constant current draw = ~30uA + 50uA (regulator) = 80uA (this is worse case for the LDO I linked in the other thread)
Therefore = 80E-6 * 60sec = 4.8mAsec/min
Periodic draw = 100mAsec/min
So if we use one of the regulators Iāve linked earlier that has only a 50uA quiescent current, then the power used to send the reading every min is 20x more than the standby current. In this use case thereās little point using another microcontroller.
Even if we were to use the LDOs that I got the other day which werenāt close to matching the ones linked in the previous thread.
Constant current draw = ~30uA + 120uA (regulator) = 150uA
150E-6 * 60sec = 9.0mAsec
Based on the 120uA regulator, we have 109mAsec per min. Dividing that out I get 1.32mAh/hour of run time. Using your aldi 2.4Ah battery, that equates to 55 Days of run time.
So⦠Have I missed something critical?
If not, we would need to reduce the update time (that is increase the interval that the data is reported to server) for standby current to start having any real impact.
No, I will not use SQL, I have a trivial python server fielding the packets. But it makes no difference to the basic project.
This is about it. The use of a separate controller meant to reduce the quiescent current to a few 1uA, and ensure that any other devices attached to the esp are also turned off (not the case now). The esp does not have a good clock, again not critical.
I will assume we will use a fast esp module (the slow one used 4 times as much power). I have an esp-12e on order, and it may be better than the esp-12 I have now which are slow. I also suspect that there is a software issue that makes it go slow rather than a hardware one (the 4MB modules go slow, but one of them was fast with an earlier setup).
A quick time calculation: a (reference) 1000mAh battery will run this setup (100-110mAsec) for 23-25 days. Letās call it almost two months from a common 2300-2500mAh 3.6-3.7 battery (1x18650 or 3xAA).
To improve it significantly we need to either find a lower power WiFi device (BLE is suitable for some scenarios) or one that connects much faster. The other leakages will not noticeably change the picture.
My timing shows that the esp used 13mA without wireless, around 80 with it on but idle, and a few 100s when transmitting. It takes under 300ms to wakeup, about 1s to have an IP, and bugger all to send a UDP packet. My current app attaches these stats to every reading it sends.
btw, I wrote a Web based python logger that will log to CSV:
@app.route('/logsensor/<projectname>')
def log_sensor(projectname):
""" Allows a user to log a sensor
"""
projects = clixxIOListProjects()
num_format = re.compile("^[1-9][0-9]*\.?[0-9]*")
if projectname in projects:
# Obtain a timestamp
ds = datetime
ds = ds.now()
# Place the logfile in the project directory
logpath = os.path.join(clixxIOProjectDir(projectname),projectname + ".csv")
header_row = None
if not os.path.exists(logpath):
header_row = ['Timestamp']
for k in request.args.keys():
header_row.append(k)
ofile = open(logpath, "ab")
writer = csv.writer(ofile, quoting=csv.QUOTE_NONNUMERIC)
# Add the header row with field names
if header_row:
writer.writerow(header_row)
# Build the row and write it
r = [ds.isoformat(' '),]
for k in request.args.keys():
isnumber = re.match(num_format,request.args[k])
if isnumber:
r.append(float(request.args[k]))
else:
r.append(request.args[k])
writer.writerow(r)
return 'Thank you. Those values have now been logged to %s.csv.' % projectname
else:
return 'Project %s is not a valid project.' % projectname
Just need to start the webserver and do http commands with sockets. The python logging system will do the rest.
I did not upload my app for a while, I will do so now. My server (server.{sh,py}) is simpler (it logs any message to a file) and richer (it has other features). It listens for both TCP and UDP.
One option could be to build a parasitic coil and energy harvesting circuitry to extract power from āthe etherā. Most homes have a 50Hz field that is strong enough to leech from.
If you want to deploy these things outside, even a tiny solar panel will provide enough charge to prolong the life of these devices. A 0.5W 6V panel will be in the order of 8x5cm. Expose that to the sun for a few minutes a day and thereās your eternal power supply. You can get even smaller panels with even lower output, depending on how small you want this device to be.
You can even use the solar panel as a mounting system, with sufficient application of glue
Instead of updating the temp every minute, if we were to do it every 10min in theory this would give us a runtime of about 11 months. The reality is nmhi batteries that arenāt the low self discharge type wonāt last that long due to internal leakage.
Or as a compromise, updating every 4 min (15 per hour) should get close to 6 months in theory.
This is all still based on using the regular with 120uA quiescent current.
At an update rate of every 10 min, the standby current consumption starts to come close to that of the powered up current.
So, what Iāll do is mount some of those LDOs I bought on some PCB so we can start playing with them and the ESP. Sound good?
Iām looking at using several sensors for my home brewing setup, probably some ESP, probably some connected directly to a linux board (probably BBB).
I intend to use Flask to provide a web-app to visualise the data and provide some basic controls for heating temp etc.
Now the question. Has anyone looked into the best way to store and retrieve this sensor data that weāre collecting? Weāve talked briefly about flat files above, which is fine for limited situations, however it becomes cumbersome to deal with many sensors over a long time and quickly serve up graphs of an arbitrary time slice or search for events.
Thereās a number of questions along similar lines on stack overflow, with reference to solutions using Redis or NoSQL.
Collecting a few hundred readings a day from a few sensors around your house isnāt really ābig dataā. Use an SQL database until you find that itās not up to the task of rapidly recovering records based on time periods. Once you have identified a problem (e.g.: āit takes longer than 0.05s to recover three days of data from six months agoā) you can then find a solution.
Ok, Iāve never really tried so Iām only going off the problems others have posted about.
So a simple setup of say 30 sensors around the house, will give about 16 million records a year. You donāt think that will be an issue in a single table of MQSQL with something like a RPi acting as the server?
Realistically if it takes a second to pull all the data for rendering a page itās not going to be an issue, thereās going to be about 1 user most of the time of the web interface.
My approach is that IoT data collection is a āwrite often read rarelyā kind so a flat file is suitable (lowest cost of write).
Reading, when used for management (by online automatic control) should probably feed directly from the incoming readings and not from the data store. Weāll talkā¦
[later] I actually do not consider SQL a suitable format for storing data unless complex retrieval is required.
16 million is about where you start to have proper database
The main issues with any engine, SQL or NoSQL or whatever, will be how you set the software up in the environment it is operating. With more RAM available you can use in-memory indexes, otherwise you have to resort to indexes on disk. With all data on a SD card, you can be more liberal with indexes because reads are so fast.
It might even be worth experimenting with different options to see which youāre most comfortable with.
āI have lots of recordsā is not a deciding factor in which data storage option to choose. Factors that do matter include availability, scalability, read vs write, frequency of access, required response times, etc.
This could be a useful topic to discuss (and experiment with) on a separate thread, with dedicated sessions at the space!
Iāve done the Flask code for this now, just havenāt deployed it.
CSV is a pretty good starting point for keeping data.
Donāt forget that you can store CSV files by the megabyte easily and performance reading them doesnāt degrade.
If somebody started with CSV, then wanted to move the data into SQL, thatās just a simple database import as all the databases will load a csv file now.
Your Flask web server can decide where it wants to put the data, so that decision is really a back end one. It should be a configuration option almost.
Oh boy! Sorry, but cloud with IoT is an invitation for a disaster. Unlock your front (and any other) door and open it wide⦠I know that everyone is pushing this but I am convinced that it is just wrong. IoT is an internal thing, and any external access should be done from a different network with strong protection and no direct access to the internal network.
It might provide an easy enabler for somebody to turn on a temperature sender, and they get to see a graph on their phone during a ādemo periodā.
They either pay money after 30-days of seeing it or set up their own server somewhere in the home. Which just may be another ESP-8266 or a rPi or whatever.
I realise about security, but lotās of people want to āplayā and ālearnā, thatās how I think of it anyway.
If some cloud sensor site ever got 500,000 hits by ESP8266 devices, well itās better than a few thousand hits on a forum page etc etc. Even though the current forum listings seemed to have got good interest.