Add persistent script storage + hot-reload, verified live

lua_runtime now loads the running script from a new LittleFS-backed
drivers::storage (joltwallet/littlefs, the project's fifth external
managed component -- mounts the "storage" partition already reserved
in partitions.csv), falling back to the compiled-in default script
only on first boot. transport exposes GET/POST /api/script -- raw Lua
text, not JSON, since json_helpers.h can't round-trip scripts safely
-- relayed to lua_runtime via the same callback-registration pattern
already used for LedColorHandler, keeping transport free of a
dependency on lua_runtime. lua_runtime_reload() persists and hot-swaps
to a fresh Lua state immediately, no reboot.

Verified on real ESP32-C6 hardware: script loads via kvida-sdk, an
edited script hot-reloads within one tick, and the edit survives a
power cycle (proving LittleFS persistence, not just an in-RAM change).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Ronny Eia
2026-07-12 10:27:00 +02:00
parent eae6d3af5a
commit 967fecbf69
12 changed files with 329 additions and 29 deletions

View File

@@ -6,20 +6,21 @@ Built on `espressif/lua` (v5.5.0, pinned in `idf_component.yml`) -- Espressif's
## What's here
- `lua_runtime_init()` creates the Lua state, opens only `base`/`table`/`string`/`math` (deliberately *not* `luaL_openlibs()`, which would also expose `io`, `os`, `package`/`require`, `debug` -- see AGENTS.md's "Lua should never access ESP-IDF directly"), registers the `kvida` API table, then loads the default script (`src/default_script.h`, a compiled-in string) which just *defines* `on_tick()`.
- `lua_runtime_init()` mounts storage (`drivers::storage_init()`) and tries to read `/storage/script.lua`; if none is stored yet (first boot, or the read fails), it falls back to the compiled-in default script (`src/default_script.h`). Either way it then builds the Lua state: opens only `base`/`table`/`string`/`math` (deliberately *not* `luaL_openlibs()`, which would also expose `io`, `os`, `package`/`require`, `debug` -- see AGENTS.md's "Lua should never access ESP-IDF directly"), registers the `kvida` API table, then runs the script, which just *defines* `on_tick()`.
- `lua_runtime_tick()` calls `on_tick()` -- one "wake, execute Lua, publish changes, sleep" cycle. A Lua runtime error is logged, not fatal.
- `lua_runtime_reload(script, len)` persists a new script to storage and hot-reloads it into a fresh Lua state immediately -- no reboot. Wired to `transport::set_script_change_handler()` (POST /api/script) by `main.cpp`. `lua_runtime_current_script(buf, cap)` is the read side, wired to `transport::set_script_provider()` (GET /api/script).
- The `kvida` table exposed to scripts:
- `kvida.chip_temperature()` -- reads the ESP32-C6's internal die temperature via `profiles::ChipTemperatureSensor` (the same `Sensor` implementation `main.cpp` used directly before this component existed).
- `kvida.set_led(r, g, b)` -- sets the onboard WS2812 via `drivers::rgb_led_set_color()`.
- `kvida.publish(name, value)` -- calls `transport::publish_value()`, AGENTS.md's semantic publish API, built for the first time here.
- The default script reads chip temperature, maps it to a blue(cool)-to-red(hot) LED color (linear interpolation, 20-60C, clamped), and publishes the reading to Home Assistant. Verified end-to-end on real hardware.
## Known limitation, accepted for now
## Known limitations, accepted for now
The onboard LED is also controllable manually from Home Assistant (an MQTT light entity, see `components/transport/src/mqtt.cpp`). The two aren't reconciled: `on_tick()` overwrites a manually-chosen HA color every 30s. Accepted as a conflict between two "quick proof" demos rather than solved now -- a real answer (e.g. a Lua-settable "mode" toggle, or the MQTT light entity deferring to Lua) needs product input, not a guess.
- The onboard LED is also controllable manually from Home Assistant (an MQTT light entity, see `components/transport/src/mqtt.cpp`). The two aren't reconciled: `on_tick()` overwrites a manually-chosen HA color every 30s. Accepted as a conflict between two "quick proof" demos rather than solved now -- a real answer (e.g. a Lua-settable "mode" toggle, or the MQTT light entity deferring to Lua) needs product input, not a guess.
- `lua_runtime_reload()` doesn't roll back to the last-good script if the new one fails to compile/run -- it just leaves `on_tick` undefined until the next successful reload (same behavior `lua_runtime_init()` already has for a bad default script). Storage and `lua_runtime_current_script()` always reflect the most recently POSTed script either way, even a broken one, so a user editing from `kvida-sdk` can see (and fix) exactly what they submitted.
## Not yet done
- The script is compiled in, not loaded from storage -- there's no upload mechanism yet. Once `kvida-sdk` can push a script (and `drivers` mounts the `storage` LittleFS partition -- both still TODO), scripts should be loaded from there instead.
- No Blockly-to-Lua compiler exists yet (that's `kvida-sdk`'s job per AGENTS.md's "Level 2").
- No memory/CPU/runtime limits on the Lua state beyond the restricted library set -- a script with an infinite loop would hang `lua_runtime_tick()` (and, since it currently runs on the same timer callback, block other `esp_timer` callbacks too). Not a concern for a compiled-in, developer-authored script; becomes one once arbitrary user scripts can be uploaded.
- No Blockly-to-Lua compiler exists yet (that's `kvida-sdk`'s job per AGENTS.md's "Level 2") -- for now scripts are hand-written Lua, edited as raw text in `kvida-sdk`.
- No memory/CPU/runtime limits on the Lua state beyond the restricted library set -- a script with an infinite loop would hang `lua_runtime_tick()` (and, since it currently runs on the same timer callback, block other `esp_timer` callbacks too). Was a non-issue for a compiled-in, developer-authored script; now that scripts are user-editable and persisted, this is a real gap, just not one this pass solves.

View File

@@ -1,11 +1,14 @@
#pragma once
#include <cstddef>
namespace kvida {
// Creates the sandboxed Lua state and loads the default script (see
// src/default_script.h). The script only *defines* the on_tick()
// function -- call lua_runtime_tick() to actually run it. Call once at
// boot.
// Mounts storage and loads the script from there (see
// components/drivers/include/storage.h), falling back to the compiled-in
// default (src/default_script.h) if none is stored yet. The script only
// *defines* the on_tick() function -- call lua_runtime_tick() to actually
// run it. Call once at boot.
void lua_runtime_init();
// Calls the script's on_tick() global function -- one "wake, execute
@@ -14,4 +17,14 @@ void lua_runtime_init();
// down the device.
void lua_runtime_tick();
// Persists `script` to storage and hot-reloads it into a fresh Lua
// state, replacing whatever's currently running -- no reboot needed.
// Wired to transport::set_script_change_handler() by main.cpp.
void lua_runtime_reload(const char *script, size_t len);
// Copies the currently-running script into buf (capacity cap), returns
// the number of bytes written. Wired to transport::set_script_provider()
// by main.cpp.
size_t lua_runtime_current_script(char *buf, size_t cap);
} // namespace kvida

View File

@@ -1,10 +1,14 @@
#include "lua_runtime.h"
#include <algorithm>
#include <cstring>
#include "default_script.h"
#include "chip_temperature_sensor.h"
#include "mqtt.h"
#include "rgb_led.h"
#include "storage.h"
#include "esp_log.h"
@@ -19,10 +23,18 @@ namespace kvida {
namespace {
constexpr const char *TAG = "lua_runtime";
constexpr const char *kScriptPath = "/storage/script.lua";
lua_State *g_L = nullptr;
ChipTemperatureSensor g_chip_temp_sensor;
// Holds whatever script is currently loaded into g_L -- either read from
// storage at boot, the compiled-in default (no stored script yet, or the
// read failed), or the most recent successful lua_runtime_reload() body.
// This is what GET /api/script (via lua_runtime_current_script()) serves.
char g_script[8192];
size_t g_script_len = 0;
// Deliberately not luaL_openlibs(), which would also expose `io`, `os`,
// `package`/`require`, and `debug` -- AGENTS.md: "Lua should never
// access ESP-IDF directly. Lua interacts only through the Kvida API."
@@ -85,6 +97,50 @@ void register_kvida_api(lua_State *L)
lua_setglobal(L, "kvida");
}
// Copies into g_script/g_script_len, truncating (with a logged warning)
// if it doesn't fit -- the same fixed buffer backs both what gets
// persisted to storage and what GET /api/script serves back.
void copy_script(const char *script, size_t len)
{
size_t n = std::min(len, sizeof(g_script) - 1);
if (n < len) {
ESP_LOGW(TAG, "script truncated from %u to %u bytes to fit the buffer", static_cast<unsigned>(len),
static_cast<unsigned>(n));
}
memcpy(g_script, script, n);
g_script[n] = '\0';
g_script_len = n;
}
// (Re)creates the Lua state and runs `script` in it -- shared by both
// lua_runtime_init() and lua_runtime_reload(). Uses luaL_loadbuffer()
// (explicit length) rather than luaL_dostring() (relies on strlen())
// since callers -- e.g. the POST /api/script body -- don't guarantee
// NUL-termination.
bool load_script(const char *script, size_t len)
{
if (g_L) {
lua_close(g_L);
g_L = nullptr;
}
g_L = luaL_newstate();
if (!g_L) {
ESP_LOGE(TAG, "luaL_newstate failed");
return false;
}
open_safe_libs(g_L);
register_kvida_api(g_L);
if (luaL_loadbuffer(g_L, script, len, "script") != LUA_OK || lua_pcall(g_L, 0, 0, 0) != LUA_OK) {
ESP_LOGE(TAG, "Failed to load script: %s", lua_tostring(g_L, -1));
lua_pop(g_L, 1);
return false;
}
return true;
}
} // namespace
void lua_runtime_init()
@@ -93,20 +149,36 @@ void lua_runtime_init()
return;
}
g_L = luaL_newstate();
if (!g_L) {
ESP_LOGE(TAG, "luaL_newstate failed");
return;
}
open_safe_libs(g_L);
drivers::storage_init();
g_chip_temp_sensor.begin();
register_kvida_api(g_L);
if (luaL_dostring(g_L, default_script()) != LUA_OK) {
ESP_LOGE(TAG, "Failed to load default script: %s", lua_tostring(g_L, -1));
lua_pop(g_L, 1);
size_t stored_len = 0;
if (!drivers::storage_read_file(kScriptPath, g_script, sizeof(g_script), &stored_len)) {
copy_script(default_script(), strlen(default_script()));
} else {
g_script_len = stored_len;
}
load_script(g_script, g_script_len);
}
// No rollback to the last-good script if the new one fails to
// compile/run -- accepted simplification for now (same as
// lua_runtime_init()'s handling of a bad default script), see the
// README. g_script/storage always reflect the most recent POST either
// way, so the user can see (and fix) what they submitted.
void lua_runtime_reload(const char *script, size_t len)
{
copy_script(script, len);
drivers::storage_write_file(kScriptPath, g_script, g_script_len);
load_script(g_script, g_script_len);
}
size_t lua_runtime_current_script(char *buf, size_t cap)
{
size_t n = std::min(g_script_len, cap);
memcpy(buf, g_script, n);
return n;
}
void lua_runtime_tick()