esp8266 iot sdk

I’ve spent a couple of nights now programming the ESP using the both the non-OS SDK (the most widely used one which nodeMCU is built on) and the FreeRTOS version of the Espressif SDK.

Whilst they’re certainly not perfect, I’d encourage anyone playing with the ESP to have a go. Open-esp-sdk project has made building the toolchain a piece of cake on linux.

For me, the C SDK is significantly more complete than dealing with the nodeMCU LUA derivative. Even the non-os version of the SDK has parts of it’s cooperative multi-tasking API exposed that make writing user applications much neater than the LUA version, not to mention the memory savings.

There’s some real gems of info in the SDK programmers guides that’s worth looking at (albeit they’re complete rubbish compared to what you would expect from any western chip manufacturer.)

The FreeRTOS SDK holds most promise for me. I’ve used FreeRTOS in projects before on the STM32 chip and I’m a big fan. From the small amount I’ve done with this port, all seems to be working fine in the core SDK. The peripheral libraries however have not been ported across correctly, and most are missing. I had to modify the uart library just to get it to build. However, the peripherals (uart, spi, timer, i2c) are pretty basic things. Porting Espressif’s non-os version or even writing your own libraries isn’t hard. Espressif have released the documents that “detail” these peripherals enough to know what all the registers do.

So I think I’m going to stick with the FreeRTOS SDK unless I find any real show stoppers. For the universal remote project, this means there’ll be no requirement for an additional processor which is nice.

The upcoming ESP32 (now in beta) will arrive with a FreeRTOS SDK so this effort will have further use.
I use the lua firmware but find it clashes with my simple (C?) brain and should probably make the move to FreeRTOS (which I never used).

You’d find FreeRTOS trivial to learn, it’s about as simple as a preemptive scheduler gets. It’s also very widely used in embedded systems, so it’s a handy one to know. People only tend to find it intimidating if they don’t properly understand C function pointers, void pointers, and the syntax.

Using C and the register maps that Espressif have published is the only really functional way to use the SPI, UART and I2C peripherals. There’s just too many of the register functions that haven’t been exposed in the LUA framework (and that’s not having a go at NodeMCU, it’s just a simple fact that trying to abstract these types of peripherals only works if what you’re externally talking to happens to want its data in exactly that format.)

It would be interesting to see more information on this. I’ve never used C on the ESP8266 so I can’t really comment further.

I’ll come along Wednesday night, so we can chat about it then if you like.

The information is out there on the net for this. But it takes quite a bit of sifting to find what’s relevant to the current SDK, and current toolchain make process. It’s a bit of a sea of miss information that’s still out there from the build process evolving.

It’s actually quite simple to setup on a *nix system now. I can give you my VM with Ubuntu and the toolchain setup if you don’t want to download it all.

Good idea, maybe it is time for me to try RTOS too.

So is this SDK something that already comes from Expressif ? that you can just download and run?

The SDK (both non-os and FreeRTOS) are from Espressif. They’re actually on Github.

Can you download and run? Not quite. The SDK is nothing more than a bunch of binary blobs and header files.

However, head over to the open-esp-sdk project (https://github.com/pfalcon/esp-open-sdk) and they have a set of make files that will get all the dependencies (including SDK) and make the complete toolchain for you.

That’ll get you the non-os version of the SDK up and running. For the FreeRTOS version you need to do the above, then clone the FreeRTOS Espressif SDK git project, then you’re ready to build using that.

It was interesting seeing you guys have so much fun.

I was watching with interest and would like to set it up sometime soon myself.

Following up from discussion during the last meeting I started recoding my app using the RTOS SDK (I used lua before).

While technically it looks simple, I found the lack of proper doco frustrating. I coded most of what I need (still need to approach one-wire and i2c) but so far I stumbled on a timing issue that I was unable to resolve. I am trying the RTOS fw forum on github (which is rather quiet) first.

Without having all the code of the port (i.e. code for all the binary blobs) it’s hard to say what the underlying TCP/IP stack is doing to cause that sort of delay. The problem is the TCP/IP stack is a high priority task (I think it was 12), and it looks like it’s starving all other tasks of execution time at that point of connection. Who knows why?

On another note, this port of FreeRTOS has the tick time set at 10ms (where as “normally” I’ve always seen it at 1ms for microcontroller ports.) So doing a delay for 10ms isn’t really going to work out how you want. It’s going to yield to other tasks, then get execution again on the next scheduler tick event. So there’s no granularity to that.

I.e. 10/portTICK_RATE_MS = 1

While I really like FreeRTOS, I don’t believe it’s the right tool for your temp sensor application. It’s a bit like hitting a nail with a sledge hammer. You’d probably be better off with the non-os SDK.

1 I already tried longer delays which did not change the result. I also tried no delay, just a yield in the loop, same result.

2 I now tried to play with the priority. Raising it to 4, 8, 12 did not change. Raising it higher caused some trouble as I probably starved the wifi engine. I added some retry logic and then got the same 1s delay happening.

I now think that it may be a case of interrupts abuse or maybe scheduling held back for a while. This 1s is so very reliable.

3 non-os will not be much help (for me) because it will require the same trickery as lua (which does the job well now). I may do this to see how much faster it goes (removing the argument marshalling and GC overhead of lua).

Maybe the next rtos release will have this issue resolved. If the non-os can connect in about 100ms then why not rtos?

Finally, since I want to try using p2p beacon payload (to UDP and no IP) maybe rtos will still do well then?

See my code here

WIth 1 & 2, that’s totally what I would expect. I would imagine that the TCP/IP stack (at priority 12) is just holding execution and not returning it to the scheduler. From my interpretation they just haven’t really implemented an RTOS version of their stack, rather they’ve just tacked it on.

Is this compiled for the 512k or larger? Just wondering if that 1 sec delay is the same delay I see in the Lua code when I compile for larger flash sizes?

Initially I used SPI_SIZE_MAP=4 but later changed to SPI_SIZE_MAP=0. Did not seem to have an effect.

When I compile lua with auto flash size things go really slow but I did not see a specific 1s. I did not try other sizes.

It is probably not related to networking, simply doing

    file.format()

shows a dramatic slowdown from a fraction of a second (512K build) to about 30s (auto size). This is on the same 4MB module.

Edit: in my earlier link to my code I should have mentioned that the ,w1.092, item in the sent message is the measured time of the waiting for wifi. For example:

12:12:08.433760222 show esp-witty times=s0.078,w1.092,c8,t1.370 adc=5.196 vdd=3.339 0.0000
12:12:12.025680351 show esp-witty times=s0.078,w1.092,c8,t1.370 adc=5.316 vdd=3.340 0.0000
12:12:15.611628221 show esp-witty times=s0.078,w1.092,c8,t1.370 adc=5.172 vdd=3.340 0.0000
12:12:19.196470119 show esp-witty times=s0.078,w1.092,c8,t1.370 adc=4.992 vdd=3.336 0.0000

Here are some timings with lua fw. esp-witty seems to be slow generally so I also added my esp-201 (a 512K module) to show the speed of a more common module. The witty was tested with four different builds. The format is done twice to get past the work done once after a flash.

As you can see, the format time went up from 1s (512K) to 6s (1MB), more that just a 1s delay.

esp-witty (4M)

512K
        > t=tmr.now() ; file.format() print(tmr.now()-t)
        format done.
        1172575
        > t=tmr.now() ; file.format() print(tmr.now()-t)
        format done.
        1176613
1M
        > t=tmr.now() ; file.format() print(tmr.now()-t)
        format done.
        6164786
        > t=tmr.now() ; file.format() print(tmr.now()-t)
        format done.
        6140405
2M
        > t=tmr.now() ; file.format() print(tmr.now()-t)
        format done.
        15810891
        > t=tmr.now() ; file.format() print(tmr.now()-t)
        format done.
        15831934
4M
        > t=tmr.now() ; file.format() print(tmr.now()-t)
        format done.
        35266714
        > t=tmr.now() ; file.format() print(tmr.now()-t)
        format done.
        35319145

esp-201 (512K)

512K
        > t=tmr.now() ; file.format() print(tmr.now()-t)
        format done.
        765416
        > t=tmr.now() ; file.format() print(tmr.now()-t)
        format done.
        530692

Later thinking: If SPIFF writes the full filesystem then the above difference may be simply related to the fs size. On 512K the fs is small (say 100k) but on the 1m it will be 512k+100k so 6 times larger, and so on. Does SPIFF do this?

Are we doing anything this week?

I’d like a hand with some DS18B20 temp sensors.

FWIW @projectgus has put up some slides from his ESP8266 linux.conf.au talk today :smile:

http://projectgus.com/talks/lca_esp8266