Linked In
Linked In

User Tools

Site Tools

esp32_ota

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision 		Previous revision
esp32_ota [2024/03/12 12:57]
sausage created 		esp32_ota [2024/03/20 01:50] (current)
sausage [ESP32 OTA and Rollbacks - What happens under the hood?]
Line 1: Line 1:
-===== ESP32 OTA and Rollbacks - What happens under the hood? =====+====== ESP32 OTA and Rollbacks - What happens under the hood? ======
  
 {{ ::​esp32:​2blinks.jpg?​direct&​400|}} {{ ::​esp32:​2blinks.jpg?​direct&​400|}}
Line 5: Line 5:
 Over The Air (OTA) provides a way to send firmware to your ESP32 board in the field without the need for a cable and physical access to the device. Over The Air (OTA) provides a way to send firmware to your ESP32 board in the field without the need for a cable and physical access to the device.
  
-The purpose of this article is manually step through the various stages of the OTA process and illustrate the state of the ESP32 at each stage. ​+The purpose of this article is to manually step through the various stages of the OTA process and illustrate the state of the ESP32 at each stage. ​
  
 Together we will set up a number of firmwares suitable for testing on different partitions in flash memory. Together we will set up a number of firmwares suitable for testing on different partitions in flash memory.
Line 11: Line 11:
 OTA updates are often done via WIFI or Bluetooth, but we will be manually implementing each step (by hand) to fully appreciate what is going on under the hood. OTA updates are often done via WIFI or Bluetooth, but we will be manually implementing each step (by hand) to fully appreciate what is going on under the hood.
  
-==== Pre-requisites ====+===== Pre-requisites ​=====
  
 It will be assumed that you have ''​esp-idf''​ installed and working, and that you know what your COM port number currently is. I'll use COM7 as an example throughout the article. It will be assumed that you have ''​esp-idf''​ installed and working, and that you know what your COM port number currently is. I'll use COM7 as an example throughout the article.
Line 22: Line 22:
 https://​docs.espressif.com/​projects/​esp-idf/​en/​latest/​esp32/​api-reference/​system/​ota.html#​rollback-process https://​docs.espressif.com/​projects/​esp-idf/​en/​latest/​esp32/​api-reference/​system/​ota.html#​rollback-process
  
-This article is based on esp-idf-v5.1.1-2.+This article is based on esp-idf-v5.2.
  
-==== The stock firmware ====+===== The stock firmware ​=====
  
 Let's assume that your ESP32 board is fresh out of the packet. It might have some demo firmware and partition information on it or, it might be completely blank. Let's take a look at what could be stored in the flash memory on the ESP32. Let's assume that your ESP32 board is fresh out of the packet. It might have some demo firmware and partition information on it or, it might be completely blank. Let's take a look at what could be stored in the flash memory on the ESP32.
Line 35: Line 35:
  
   gen_esp32part.py .\partitions.log > .\partitions.csv   gen_esp32part.py .\partitions.log > .\partitions.csv
 +  ​
 +<WRAP center round important 90%>
 +If you don't have gen_esp32part.py,​ please check the troubleshooter below.
 +</​WRAP>​
  
 Open the CSV, and it will probably look something like this (if anything at all): Open the CSV, and it will probably look something like this (if anything at all):
Line 46: Line 50:
 </​code>​ </​code>​
  
-Ignoring the ''​nvs''​ and ''​phy_init''​ partitions, we have the default ''​factory''​ partition that starts at 0x10000. Ironically, the list of partitions doesn'​t list the bootloader partition at 0x1000 or the partition that the partition table itself is stored ​on (at 0x8000). ​+Ignoring the ''​nvs''​ and ''​phy_init''​ partitions, we have the default ''​factory''​ partition that starts at 0x10000. Ironically, the list of partitions doesn'​t list the bootloader partition at 0x1000 or the partition that the partition table itself is stored (at 0x8000). ​
  
-==== OTA partitions ====+===== OTA partitions ​=====
  
 The ''​factory''​ partition is currently the only one here and this is where your firmware gets flashed to. We are going to add two extra partitions for OTA which can contain firmware as well. The ''​factory''​ partition is currently the only one here and this is where your firmware gets flashed to. We are going to add two extra partitions for OTA which can contain firmware as well.
Line 60: Line 64:
 It's all about safely upgrading your device with your new software, and being able to keep going if there is a failure. This ensures you can try again another day. It's all about safely upgrading your device with your new software, and being able to keep going if there is a failure. This ensures you can try again another day.
  
-==== Setting up an ESP32 project ====+===== Setting up an ESP32 project ​=====
  
 In order that we can start doing some stuff together, let's make a new project. In order that we can start doing some stuff together, let's make a new project.
Line 76: Line 80:
   idf.py build   idf.py build
  
-==== Setting up OTA partitions and ESP32 configuration ====+===== Setting up OTA partitions and ESP32 configuration ​=====
  
 We will now set up the ESP32 to have OTA partitions. This can be done with the ''​menuconfig''​. We will now set up the ESP32 to have OTA partitions. This can be done with the ''​menuconfig''​.
Line 109: Line 113:
   idf.py -p COM7 flash   idf.py -p COM7 flash
  
-==== Inspecting the partitions in flash ====+===== Inspecting the partitions in flash =====
  
 Now let's download the partition setup again like we did before. This can be done with: Now let's download the partition setup again like we did before. This can be done with:
Line 135: Line 139:
  
  
-==== Preparing some test Firmware ====+===== Preparing some test Firmware ​=====
  
 Let's make a simple firmware that blinks the onboard LED twice, then one that blinks three times and finally one that blinks four times. ​ Let's make a simple firmware that blinks the onboard LED twice, then one that blinks three times and finally one that blinks four times. ​
Line 212: Line 216:
   - 4blinks.bin   - 4blinks.bin
  
-Now the ''​idf.py flash''​ command is only able to flash the default project firmware to the ''​factory''​ partition. So we'll need to switch to a different command to flash a named firmware to a particular partition.+Note that the ''​idf.py flash''​ command is only able to flash the default project firmware to the ''​factory''​ partition. So we'll need to switch to a different command to flash a named firmware to a particular partition.
  
 Take another look at our partitions table. Take another look at our partitions table.
Line 235: Line 239:
   esptool.py -p COM7 write_flash 0x10000 .\build\3blinks.bin   esptool.py -p COM7 write_flash 0x10000 .\build\3blinks.bin
  
-{{ :eso32:​3blinks.jpg?​direct&​600 |}}+{{ :esp32:​3blinks.jpg?​direct&​600 |}}
  
   esptool.py -p COM7 write_flash 0x10000 .\build\4blinks.bin   esptool.py -p COM7 write_flash 0x10000 .\build\4blinks.bin
  
-{{ :4esp32:blinks.jpg?​direct&​600 |}}+{{ :esp32:4blinks.jpg?​direct&​600 |}}
  
 And put the 2blinks version back onto ''​factory''​ with: And put the 2blinks version back onto ''​factory''​ with:
Line 255: Line 259:
 Again, ''​2blinks''​ firmware still operates from the ''​factory''​ partition. Again, ''​2blinks''​ firmware still operates from the ''​factory''​ partition.
  
-==== Switching firmware, and the otadata partition ====+===== Switching firmware, and the otadata partition ​=====
  
 Check the partition table again. ​ Check the partition table again. ​
Line 280: Line 284:
 This reads out the ''​otadata''​ partition, which starts at 0xd000 and is 0x2000 in length out to a file. This reads out the ''​otadata''​ partition, which starts at 0xd000 and is 0x2000 in length out to a file.
  
-If you open the ''​otadata-partition.bin''​ file in a hex reader (I recommend ​HxD - https://​mh-nexus.de/​en/​hxd/​),​ you'll see that the entire file is filled with 0xFF bytes.+If you open the ''​otadata-partition.bin''​ file in a hex reader (I recommend ​[[https://​mh-nexus.de/​en/​hxd/​|HxD]]), you'll see that the entire file is filled with 0xFF bytes.
  
 We can use a tool to set ''​ota_0''​s firmware as the one we want to boot from: We can use a tool to set ''​ota_0''​s firmware as the one we want to boot from:
Line 286: Line 290:
   otatool.py -p COM7 switch_ota_partition --name=ota_0   otatool.py -p COM7 switch_ota_partition --name=ota_0
  
-{{ :eso32:​3blinks.jpg?​direct&​600 |}}+{{ :esp32:​3blinks.jpg?​direct&​600 |}}
  
 The device will reset and notice that there are three LED blinks. This is the ''​3blinks''​ firmware running on ''​ota_0''​. The device will reset and notice that there are three LED blinks. This is the ''​3blinks''​ firmware running on ''​ota_0''​.
Line 322: Line 326:
 The ESP32 will reboot and the LED will begin blinking 4 times. This is the ''​4blinks''​ firmware booted on the ''​ota_1''​ partition. The ESP32 will reboot and the LED will begin blinking 4 times. This is the ''​4blinks''​ firmware booted on the ''​ota_1''​ partition.
  
-{{ :4esp32:blinks.jpg?​direct&​600 |}}+{{ :esp32:4blinks.jpg?​direct&​600 |}}
  
 Take a look at the ''​otadata''​ information again: Take a look at the ''​otadata''​ information again:
Line 354: Line 358:
  
 In our case, the second sector has a new entry, the ''​02''​ sequence. This is for the firmware on ''​ota_1''​. ''​02''​ is greater than ''​01''​ so this is the firmware selected for boot by the bootloader. In our case, the second sector has a new entry, the ''​02''​ sequence. This is for the firmware on ''​ota_1''​. ''​02''​ is greater than ''​01''​ so this is the firmware selected for boot by the bootloader.
 +
 +How an OTA partition is determined from sequence numbers in the otadata partition is not explained in the documentation,​ but is explained in the code. You can see it here: https://​github.com/​pycom/​esp-idf-2.0/​blob/​master/​components/​app_update/​esp_ota_ops.c#​L297
 +
 +Essentially,​ for two partitions all odd sequence numbers are ''​ota_0''​ and even numbers are ''​ota_1''​. The formula is:
 +
 +  (ota_seq - 1) % number_of_ota_partitions
  
 Revise on the ''​otadata''​ partition here:  Revise on the ''​otadata''​ partition here: 
Line 359: Line 369:
  
  
-==== Simulating Rollback process ====+===== Simulating Rollback process ​=====
  
 During a real OTA update where a firmware is sent to the ESP32 over Wifi or Bluetooth, the firmware running on the ESP32 is responsible for setting the ''​otadata''​ flash sector with the status of ''​ESP_OTA_IMG_NEW''​ for that OTA partition. During a real OTA update where a firmware is sent to the ESP32 over Wifi or Bluetooth, the firmware running on the ESP32 is responsible for setting the ''​otadata''​ flash sector with the status of ''​ESP_OTA_IMG_NEW''​ for that OTA partition.
Line 387: Line 397:
   esptool.py -p COM7 write_flash 0xd000 .\otadata-partition.bin   esptool.py -p COM7 write_flash 0xd000 .\otadata-partition.bin
  
-{{ :4esp32:blinks.jpg?​direct&​600 |}}+{{ :esp32:4blinks.jpg?​direct&​600 |}}
  
 The device will reset and the ''​4blinks''​ firmware is still running on the ESP32 from the ''​otadata''​ partition. The device will reset and the ''​4blinks''​ firmware is still running on the ESP32 from the ''​otadata''​ partition.
Line 410: Line 420:
 Let's see. All we need to do is to reset the ESP32. Press the reset button. Let's see. All we need to do is to reset the ESP32. Press the reset button.
  
-{{ :eso32:​3blinks.jpg?​direct&​600 |}}+{{ :esp32:​3blinks.jpg?​direct&​600 |}}
  
 Wow, notice something? The ''​3blinks''​ firmware from ''​ota_0''​ has been booted. Let's extract the otadata again and take a look what has occurred: Wow, notice something? The ''​3blinks''​ firmware from ''​ota_0''​ has been booted. Let's extract the otadata again and take a look what has occurred:
Line 439: Line 449:
   esptool.py -p COM7 write_flash 0xd000 .\otadata-partition.bin   esptool.py -p COM7 write_flash 0xd000 .\otadata-partition.bin
  
-{{ :eso32:​3blinks.jpg?​direct&​600 |}}+{{ :esp32:​3blinks.jpg?​direct&​600 |}}
  
 The ESP32 will reset and ''​3blinks''​ firmware continues to execute. Let's take a look at the raw ''​otadata''​ again: The ESP32 will reset and ''​3blinks''​ firmware continues to execute. Let's take a look at the raw ''​otadata''​ again:
Line 460: Line 470:
 All our firmwares failed to run the ''​esp_ota_mark_app_valid_cancel_rollback()''​ to mark the ''​otadata''​ as valid. So in each case, they have become aborted and rolled back to the previously known good firmware. All our firmwares failed to run the ''​esp_ota_mark_app_valid_cancel_rollback()''​ to mark the ''​otadata''​ as valid. So in each case, they have become aborted and rolled back to the previously known good firmware.
  
-The ''​factory''​ firmware isn'​t ​controlled ​by the ''​otadata'',​ the rollback process or OTA at all.+The ''​factory''​ firmware isn'​t ​affected ​by ''​otadata'', ​or the rollback process, or OTA at all.
  
 To finish up, let's take a last look at the raw otadata: To finish up, let's take a last look at the raw otadata:
Line 487: Line 497:
  
  
-==== More reading ==== +===== Troubleshooting ===== 
-https://​blog.espressif.com/​ota-updates-framework-ab5438e30c12 +//The gen_esp32part.py command isn't found.// 
-https://​esp32.com/​viewtopic.php?​t=14939+ 
 +For some reason, gen_esp32part.py is not added to your path by default when starting your ESP-IDF console. For Powershell add the following to line 59 of your ''​frameworks\esp-idf-v5.2\export.ps1''​ file: 
 + 
 +  function gen_esp32part.py { &python "​$IDF_PATH\components\partition_table\gen_esp32part.py"​ $args } 
 + 
 +For Windows Command Prompt, in ''​frameworks\esp-idf-v5.2\export.bat''​ add to line 66: 
 + 
 +  DOSKEY gen_esp32part.py=python.exe "​%IDF_PATH%\components\partition_table\gen_esp32part.py"​ $* 
 + 
 +For shell and fish terminals, it's the same deal. 
 + 
 +===== More reading ​===== 
 +  ​- ​https://​blog.espressif.com/​ota-updates-framework-ab5438e30c12 
 +  ​- ​https://​esp32.com/​viewtopic.php?​t=14939 
 + 
 +===== Thank yous ===== 
 +Many thanks to boarchuz and craig from the [[https://​discord.gg/​wwK29JJV|Espressif MCUs]] discord server for their assistance with this article.
  
esp32_ota.1710248235.txt.gz · Last modified: 2024/03/12 12:57 by sausage

Page Tools

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki