--- title: "Bridging Singapore Power Smart Meters to Home Assistant with Rust and MQTT" date: 2026-10-23 description: "How I built a lightweight Rust daemon that interoperates with SP Digital's OAuth2 + SMS MFA flow and brings half-hourly AMI electricity, water, and gas telemetry into Home Assistant's Energy Dashboard." author: "Daan van Gorkum" url: "https://dj.vg/posts/bridging-singapore-power-smart-meters-to-home-assistant-rust-mqtt/" markdown_url: "https://dj.vg/posts/bridging-singapore-power-smart-meters-to-home-assistant-rust-mqtt/index.md" tags: ["Rust", "Home Assistant", "MQTT", "Singapore", "Kubernetes", "IoT", "Home Automation"] reading_time: "11 min read" --- # Bridging Singapore Power Smart Meters to Home Assistant with Rust and MQTT One of the best built-in features of [Home Assistant](https://www.home-assistant.io/) is its **Energy Dashboard**. Feed it cumulative electricity, water, and gas counters, and it gives you long-term hourly and daily consumption curves, cost breakdowns, and baselines against individual smart plugs and air-conditioning circuits. In Singapore, **SP Group (SP Services / SP Digital)** has rolled out **Advanced Metering Infrastructure (AMI)** smart electricity meters across residential households. Instead of waiting for a manual meter reader once every two months, AMI meters record half-hourly electricity consumption (`48` intervals per day) alongside monthly water (`m³`) and piped town gas (`kWh`) readings. In a typical Singapore condominium or HDB flat, however, your utility meters sit inside a common riser closet in the lift lobby—meaning clamping a local CT sensor or ESPHome optical pulse reader onto the meter isn't practical. While the official **SP app** displays this data nicely on your phone, there is no built-in way to feed your household's own utility readings into Home Assistant for local energy automation and historical dashboards. Fresh off building [`iplusliving2mqtt`](/posts/bridging-condo-intercom-to-home-assistant-rust-sip-mqtt/) for my condo intercom, I wanted the exact same setup for my household utilities: a tiny, polite, self-contained **Rust** daemon running in Kubernetes that authenticates with my SP account and exposes my home's meter readings to Home Assistant via **MQTT Auto-Discovery**. That project is **`sp2mqtt`**. --- ## Understanding the Client Protocol: Three Design Challenges Inspecting how the mobile client communicates with SP Digital's backend (`identity.spdigital.sg` and `b2c.api.spdigital.sg`) highlighted three interesting engineering problems to solve for a clean, unattended Kubernetes integration. ### 1. Standard Auth0 Android SDK Telemetry & Request Metadata SP Digital authenticates users against `https://identity.spdigital.sg` using Auth0's Resource Owner Password Realm flow (`http://auth0.com/oauth/grant-type/password-realm`) with realm `Username-Password-Authentication` and audience `https://profile.up.spdigital.sg/`. Like most modern Auth0 deployments, the identity provider expects requests to include standard Auth0 mobile SDK telemetry and client headers rather than bare HTTP requests with a generic library User-Agent. When a request omits the expected SDK context, Auth0 flags it for additional verification: ```json { "error": "requires_verification", "error_description": "Suspicious request requires verification" } ``` Looking at how the client's HTTP layer is structured, outgoing requests include standard metadata headers across three areas: - **Application User-Agent**: Identifies the client version (`Infinity/ (Android)`). - **Auth0 SDK Telemetry**: Includes the standard `Auth0-Client` header (Base64URL-encoded JSON describing the `Auth0.Android` SDK version and target API level) along with `Accept-Language: en-US`. - **Installation & Distributed Tracing Headers**: Attaches a persistent per-install UUIDv4 (`X-Installation-ID`) and standard Zipkin B3 tracing headers (`X-B3-TraceId` and `X-B3-SpanId`) for request correlation. Including the standard client headers and OAuth2 scopes (`openid email profile offline_access enroll read:authenticators user_metadata me me:uportal me:eva me:rbac`) allows the OAuth2 handshake to proceed smoothly into the standard Multi-Factor Authentication flow. ### 2. Handling SMS OTP Multi-Factor Authentication in Kubernetes Security on utility accounts matters, and SP Digital enforces Multi-Factor Authentication (MFA). Once password authentication succeeds, Auth0 responds with `HTTP 403` and `{"error": "mfa_required", "mfa_token": "..."}`. Completing the login follows the standard Auth0 Out-of-Band (OOB) sequence: 1. Querying `GET /mfa/authenticators` with `Bearer ` to list the account's active authenticators (such as `sms|dev_...`). 2. Calling `POST /mfa/challenge` (`challenge_type: "oob"`) to request a 6-digit SMS OTP and receive an `oob_code`. 3. Submitting the 6-digit `binding_code` to `POST /oauth/token` using grant type `http://auth0.com/oauth/grant-type/mfa-oob`. Because the initial scope includes `offline_access`, completing MFA issues a **long-lived `refresh_token`**. Subsequent token refreshes (`grant_type: "refresh_token"`) mint fresh 1-hour `access_token` and `id_token` JWTs automatically without sending repeat SMS messages. Still, I wanted the initial onboarding—and any future re-authentication if a session ever expires—to feel native in a headless Kubernetes deployment **without** needing to `kubectl exec` into a running pod. To achieve that, `sp2mqtt` exposes the MFA workflow directly inside Home Assistant over MQTT: - When `sp2mqtt` starts with `SP_EMAIL` and `SP_PASSWORD` (and no cached `refresh_token` on its persistent volume), it publishes an **SP Services Account** device via MQTT Auto-Discovery with `sensor.sp_services_auth_status` set to `MFA Required`. - Pressing the **`1. Send SP SMS OTP`** button in Home Assistant triggers the Auth0 OOB challenge and updates the status sensor to `OTP Sent (XXXXXXX8632)`. - Typing the 6-digit code into the **`2. Submit SP 6-Digit SMS OTP`** text box in Home Assistant completes the `mfa-oob` token exchange, atomically saves the session to `/data/state.json`, and starts the telemetry poller. For declarative GitOps setups, `sp2mqtt` also supports two non-interactive authentication paths: - **Pre-provisioned `SP_REFRESH_TOKEN`**: Seeding an initial `refresh_token` and `installation_id` via a Kubernetes `Secret` so the daemon can mint access tokens immediately on first boot. - **RFC 6238 TOTP (`SP_TOTP_SECRET`)**: If an OTP authenticator is enrolled on the account, `sp2mqtt` can compute the 6-digit TOTP code in memory during login without waiting for an SMS challenge. ### 3. Reconciling Monthly Meter Reads with Half-Hourly AMI Intervals Authenticated requests to SP Digital's B2C API (`https://b2c.api.spdigital.sg`) include both the OAuth2 `Authorization: Bearer ` header and the `X-id-token: ` header. One helpful detail when working with the API is how accounts and premises are indexed: `GET /jarvis/v3/me` returns both a 12-digit billing `account_number` (`008938562736`) and a 10-digit `premise_id` (`2100102993`). All `/jarvis/*` utility endpoints are keyed by the 10-digit **`premise_id`** rather than the billing account number. Telemetry for each premise is served across three endpoints: - **`GET /jarvis/v3/smrd-uportal/{premise_id}`**: Returns the official cumulative meter readings (`last_actual_value` and `last_actual_at`) for `electric` (`kWh`), `water` (`m³`), and `gas` (`kWh`). - **`GET /jarvis/v4/charts/{premise_id}`**: Returns the monthly billed consumption history, reading status (`Actual` vs. `Estimated`), and neighborhood averages. - **`POST /jarvis/v3/ami/charts`**: Returns Smart Meter (AMI) interval data—either `48` half-hourly slots for a single day (`grouped_by: "day"`) or daily totals across a calendar month (`grouped_by: "month"`). There is a subtle nuance when feeding `/jarvis/v3/smrd-uportal` directly into Home Assistant's Energy Dashboard: `last_actual_value` only updates on the monthly billing read date (for example, `12,186.0 kWh` on the 25th of the month). To give Home Assistant a cumulative `total_increasing` electricity counter that updates every 30 minutes as smart meter data arrives, `sp2mqtt` computes a **live cumulative kWh reading**: ```text live_cumulative_kwh = meter_reading_at_t0 + sum(ami_daily_kwh for date > t0) ``` By taking the official cumulative reading anchored at `last_actual_at` and adding every subsequent day's AMI smart meter intervals (fetching both the previous month's and current month's daily AMI series whenever `last_actual_at` falls in the prior month), `sensor.__electricity_cumulative_kwh` advances smoothly every half hour while staying anchored to the official billing meter read. --- ## System Architecture Like `iplusliving2mqtt`, `sp2mqtt` is written in **Rust** as a single static binary using `tokio`, `reqwest` (`rustls`), and `rumqttc`: ```text +--------------------------+ +--------------------------+ | SP Digital Auth0 IDP | | SP Digital B2C APIs | | (identity.spdigital.sg) | | (b2c.api.spdigital.sg) | +------------+-------------+ +------------+-------------+ | 1. OAuth2 + MFA-OOB | 2. /jarvis/* | & Token Refresh | B2C REST v v +--------------------------------------------------------------+ | sp2mqtt | | | | +----------------------+ +-----------------------+ | | | Auth & Token Manager |-------->| B2C Telemetry Poller | | | | (JWT exp/state.json) | | (5xx Retry/Live kWh) | | | +----------^-----------+ +-----------+-----------+ | | | Interactive | Premise | | | SMS OTP Flow | Telemetry | | | v | | +----------+---------------------------------------------+ | | | Home Assistant MQTT Auto-Discovery Bridge | | | | (Account Auth Controls + 12 Premise Sensors) | | | +----------------------------+---------------------------+ | +-------------------------------+------------------------------+ | MQTT (homeassistant/#) v +-----------------------+ | Home Assistant | | (Energy Dashboard) | +-----------------------+ ``` ### Event-Driven Concurrency & State Coordination Internally, `sp2mqtt` runs three decoupled asynchronous tasks on the `tokio` runtime: 1. **Coordinator & Poller Loop**: A `tokio::select!` state machine that owns the `SpClient` and `PersistedState`. It wakes either on the 30-minute polling timer, on incoming `DaemonCommand` messages (`SendSmsOtp`, `SubmitOtp`, `PollNow`) over a bounded `tokio::sync::mpsc` channel, or on `SIGTERM`/`SIGINT` for graceful shutdown. 2. **Reactive MQTT Bridge**: Manages the persistent `rumqttc` connection, publishes retained discovery configurations, and subscribes to command topics (`sp2mqtt/cmd/#`) and Home Assistant's birth topic (`homeassistant/status`). Whenever the coordinator updates authentication state or premise telemetry in shared `ReadinessState`, it signals a `tokio::sync::Notify` waitlist so the MQTT task immediately flushes updated entity payloads without polling locks. 3. **Observability & Control Server**: A lightweight HTTP server on `:8080` exposing `/healthz`, `/readyz`, `/metrics.json`, and an on-demand `POST /poll` trigger. ### Polite Polling & Proactive Token Refresh Since AMI smart meter data settles in 30-minute intervals, `sp2mqtt` defaults to a polite 30-minute polling cadence (`1800` seconds)—making only a handful of lightweight HTTPS requests per half hour. Before each cycle, the token manager inspects the `exp` claim inside both the `access_token` and `id_token` JWTs (refreshing proactively with a 5-minute safety margin) and persists updated credentials atomically via a temporary file (`state.json.tmp` → `rename`) so an unexpected pod eviction never corrupts the cached refresh token. Transient upstream `5xx` gateway responses are handled with bounded exponential backoff. --- ## Home Assistant Entities Once connected to the MQTT broker, `sp2mqtt` registers two devices in Home Assistant with zero manual YAML configuration: ### 1. Account & Authentication Device (`SP Services Account`) | Entity | Type | Description | | :--- | :--- | :--- | | **Auth Status** | `sensor` | `Authenticated`, `MFA Required`, `OTP Sent (...)`, or `Error` with diagnostic attributes. | | **Authenticated** | `binary_sensor` | `ON` when tokens are valid; `OFF` if interactive MFA is needed. | | **1. Send SP SMS OTP** | `button` | Initiates Auth0 login and triggers an SMS OTP challenge to your registered phone. | | **2. Submit SP 6-Digit SMS OTP** | `text` | Masked input box to submit the 6-digit SMS code directly from the Home Assistant UI. | | **Refresh Utility Usage Now** | `button` | Triggers an immediate on-demand telemetry poll across all premises. | ### 2. Per-Premise Utility Device (`SP Premise `) | Entity | Class / State Class | Unit | Description | | :--- | :--- | :--- | :--- | | **Electricity Cumulative (Live)** | `energy` / `total_increasing` | `kWh` | **Energy Dashboard primary sensor**: Official meter reading + subsequent AMI intervals. | | **Electricity Meter Reading** | `energy` / `total_increasing` | `kWh` | Official cumulative meter reading from `smrd-uportal`. | | **Electricity Today (AMI)** | `energy` / `total` | `kWh` | Today's smart meter consumption across 48 half-hour slots. | | **Electricity Yesterday (AMI)** | `energy` / `measurement` | `kWh` | Yesterday's smart meter total (with full 48-slot breakdown in attributes). | | **Electricity Month-to-Date (AMI)** | `energy` / `total` | `kWh` | Current month's daily AMI total (with daily bars in attributes). | | **Electricity Latest 30m Slot** | `energy` / `measurement` | `kWh` | Consumption during the most recently completed 30-minute AMI interval. | | **Electricity Latest 30m Power** | `power` / `measurement` | `W` | Average load (`kWh * 2000 W/kWh`) during the latest 30-minute slot. | | **Electricity Billed Month** | `energy` / `measurement` | `kWh` | Latest monthly billed electricity usage (`Actual` / `Estimated` + neighbor average). | | **Water Meter Reading** | `water` / `total_increasing` | `m³` | **Energy Dashboard water sensor**: Cumulative water meter reading. | | **Water Billed Month** | `water` / `measurement` | `m³` | Latest monthly billed water consumption. | | **Gas Meter Reading** | `energy` / `total_increasing` | `kWh` | **Energy Dashboard gas sensor**: Cumulative piped town gas meter reading. | | **Gas Billed Month** | `energy` / `measurement` | `kWh` | Latest monthly billed gas consumption. | --- ## Kubernetes Runtime Architecture Running a stateful IoT bridge in Kubernetes that also supports interactive MFA requires careful alignment between container security, volume lifecycle, and probe semantics: - **Hardened Distroless Runtime**: `sp2mqtt` is compiled in a multi-stage build and packaged onto **`gcr.io/distroless/cc-debian13:nonroot`**. The pod runs as UID/GID `65532` with `readOnlyRootFilesystem: true`, `allowPrivilegeEscalation: false`, `automountServiceAccountToken: false`, and all Linux capabilities dropped—mounting only a read-only ConfigMap at `/etc/sp2mqtt` and a tiny PersistentVolumeClaim at `/data` for `state.json`. - **Single-Writer Rollout Strategy (`Recreate`)**: Because Auth0 refresh tokens and installation metadata are persisted on a `ReadWriteOnce` volume, the Deployment uses `strategy.type: Recreate` so two replicas never race to refresh or overwrite the same OAuth2 session concurrently during rollouts. - **Decoupled Readiness vs. Auth State**: If `/readyz` required an active `access_token`, Kubernetes would mark an unauthenticated pod unready—and if a liveness probe were tied to authentication, the kubelet would enter a `CrashLoopBackOff` right while the user is waiting for an SMS OTP. Instead, `/healthz` verifies that the Tokio runtime is responsive, while `/readyz` checks that the MQTT broker connection is active (`mqtt_connected: true`). That way, the pod stays healthy and ready so it can expose the **SP Services Account** MFA controls in Home Assistant while awaiting initial OTP verification. - **MQTT Last Will & Birth Synchronization**: The MQTT client registers a Last Will and Testament (`sp2mqtt/status` → `offline`) and subscribes to `homeassistant/status`. If Home Assistant restarts while `sp2mqtt` is already running, receiving the `online` birth message triggers an immediate re-broadcast of all discovery schemas and cached premise states so sensors never show up as `unavailable`. If you're in Singapore and interested in how household smart meter, water, and gas telemetry can be bridged into Home Assistant's Energy Dashboard, feel free to reach out at [me@dj.vg](mailto:me@dj.vg).