Back to list

Development Update — August 5

A heavy day across the whole stack. The dmsg server fleet learned to serve WebTransport on the socket it already listens on, so browser visors stop being stuck on the wss fallback. The decommissioned standalone Uptime Tracker was purged from every CLI command that still dialed its dead PK, with all uptime queries repointed to the TPD-integrated tracker over CXO. The native and browser edges kept converging — a shared transport-manager builder, a shared service-PK set, and a shared auto-transport policy that finally makes dmsg the genuine last resort on both. On top of that: discovery readouts that show real versions and inferred load, a guided tour that actually walks the UI, and a large mobile push — the phone’s Fleet window, SkyVPN over Android’s TUN, and a skychat that can move between devices. On the Skycoin repo the day was light: a batch of web-subproject dependency bumps and a README/security-policy pass that brings the project into line with skywire.

Skywire: dmsg-over-WebTransport, On By Default

Browser wasm-visors were staying on wss instead of upgrading to WebTransport because only 1 of 9 deployment dmsg servers advertised address_wt. 3739 feat(dmsg-server): WebTransport default-on makes WT free: ServeUnifiedQUIC serves dmsg-over-QUIC and dmsg-over-WebTransport on one UDP socket via a shared quic.Transport, ALPN-demuxed (skywire QUIC ALPN → native dmsg, h3 → WebTransport), auto-generating the self-signed WT cert — no extra port, no discovery topology change, reachable wherever dmsg-over-QUIC already is. 3747 fix(dmsg): WT never advertised — single ALPN-demux listener is the diagnosed follow-up: ServeUnifiedQUIC was calling both tr.Listen() and tr.ListenEarly() on the same transport, but a quic.Transport allows exactly one listener (errListenerAlreadySet), so the WT ListenEarly always failed and setAdvertisedWT was never reached — WT was dead on every server that ran the new path. It’s now one tr.Listen with a GetConfigForClient that returns the dmsg-identity cert or the WebTransport cert per ClientHello, and the accept loop dispatches each conn by negotiated ALPN. A new TestServeUnifiedQUIC_QUICThroughDemux runs two native QUIC clients plus a bridged A→B stream through the demux to guard the real regression risk: dmsg-QUIC must still handshake. 3746 fix(dmsg): browser wss churn — only converge wss→WT for WT-capable servers stops the client from tearing down a working wss session toward a server that can’t actually offer WT yet, so the convergence only fires once a server genuinely advertises it. Together these close the wss→WT gap fleet-wide as servers redeploy.

3769 fix(transport): rank direct WT/WS above NAT-traversing WEBRTC corrects the preference order. WEBRTC is a NAT-traversing DataChannel signalled over dmsg — it reaches any visor, but only via ICE and heavier than a true point-to-point link — yet it sat above WS and WT in PreferenceOrder, so EnsureBestTransport (the route-setup hook the wasm edge uses for app dials) and the router’s candidate selection picked WEBRTC before a direct WT, the inverse of the intent. The order is now STCPR > QUIC > SUDPH > STCP > WT > WS > WEBRTC > DMSG, so a browser edge makes WEBRTC only to peers no direct carrier can reach.

Skywire: The Standalone Uptime Tracker, Retired

The standalone Uptime Tracker (PK 022c424c…, dmsg://…:80/uptimes?v=v2) is offline on the deployment, and every CLI command that read the “online visor” set from it broke — most visibly pv -t, which dialed the dead delegated server, got dmsg error 202, then built invalid JSON from the empty result and printed nothing. 3763 fix(cli): deprecate standalone uptime tracker routes all CLI uptime queries to the TPD-integrated tracker — the transport-discovery base with ?v=v3, which cxoFeedForURL already mirrors over CXO (feed tpd-uptime) — so fetches ride the existing CXO→DMSG→HTTP chain and never touch the decommissioned service. /uptimes?v=v3 is a strict superset of v2 (every entry still carries {pk, on, version, daily} plus a per-day bitmap), so no consumer needed a payload adaptation; a shared FetchIntegratedUptimes helper backs the converted sites (pv, sd, ut, vpn list, proxy list/test, visor whois, log), and every jq-join now guards an empty fetch so an offline tracker degrades to “no online visors” rather than invalid JSON. The always-red standalone-UT row is dropped from svc health. 3766 fix(cli): pv drops the now-redundant --uturl/--cdu (uptime derives from --tpdurl), and 3772/3773 switch pv and sd/proxy/vpn to fetch uptimes/days/1 instead of days/30 — they only need the online filter, so pulling one day of timeline instead of thirty is a far cheaper fetch.

The other side of parity: 3767 feat(wasm-visor): uptime-heartbeat parity with the native visor. A long-lived browser tab sent no heartbeat, so the integrated tracker saw it as offline. Rather than stand up a second client, UpdateUptime is added to the dmsg tpdclient the tab already holds, issuing the same signed GET /v4/update over the tab’s established noise-authenticated dmsg stream — over dmsg the TPD authenticates by the KK-handshake PK in RemoteAddr and skips the nonce+signature entirely (#3725), so the heartbeat needs nothing beyond riding the session. startUptimeHeartbeat mirrors the native 5-minute cadence, firing immediately so a reloaded tab claims the current slot.

Skywire: Discovery and Load Readouts

disc.Entry.Version was a vestigial protocol constant "0.0.1" that nothing checks for compatibility, so mdisc servers showed 0.0.1 for every server instead of the real build. 3775 feat(dmsg): advertise the server build version in its discovery entry carries buildinfo.Version() there instead (a Version field on dmsg.ServerConfig, stamped on the registration cycle’s create and update paths), keeping pkg/dmsg decoupled from skywire’s buildinfo — so mdisc servers shows each server’s real version straight from /all_servers with no /health probe. svc health deliberately keeps its direct /health probe, since that’s what catches an unreachable server, which reading the entry would mask. 3774 feat(cli): mdisc servers — inferred connected-client count adds a connected~ column: discovery doesn’t publish a client count, but every server runs the same 2048 max, so connected ~= 2048 - available_sessions. It reads /all_servers (so a saturated server, dropped from /available_servers exactly when it’s most interesting, still appears) and sorts most-loaded first — a pv -t-style load readout. On the visor side, 3768 feat(cli): pv -y breaks the public-visor transport count down by type — total=451 stcpr=14 … sudph=437 … in a fixed-column table friendly to awk/graphing, aggregated from the same /all-transports data pv -t already fetches.

Skywire: Auto-Transport Convergence — dmsg as Last Resort

A run of fixes finally makes dmsg the genuine last-resort data-plane carrier on both visor kinds. 3760 fix(wasm-visor): route-setup hook creates a p2p direct transport (webrtc), not dmsg is the root of the boot storm — the wasm visor registered zero route-setup hooks, so on an app dial the router could only route over existing transports, queried the finder (which can’t yet path to an arbitrary exit), got transport not found, and retried 6× while re-racing. It adds the equivalent hook via a shared primitive: Manager.PreferredDirectCreateOrder() filters the global preference to the direct clients the manager actually initialised (browser → [webrtc, ws, wt]), and Manager.EnsureBestTransport creates a direct transport in that order, dmsg only if every direct type fails. This also reverts 3759 (which had dialed exits with EnsureDirectTransport + UseExistingTpOnly, a path that hardcodes STCPR→SUDPH→DMSG and so always landed dmsg on a browser) back to plain dial options, letting the hook do the right thing. On the harness, transport not found dropped from hundreds to ~3 with the data plane entirely p2p. 3761 feat(transport): native route-setup hook + tp add use the full auto-transport preference order carries the same fix to native, where both auto-creation paths had only ever tried STCPR→SUDPH→DMSG — dropping to the dmsg relay the moment stcpr+sudph don’t apply, which is always on a NAT’d visor. The native hook’s fallback now calls EnsureBestTransport over the remaining creatable types (with a skip set and a 15s per-type timeout so a hanging SUDPH hole-punch can’t starve cheaper types), and tp add with no type iterates the full PreferenceOrder(). 3748 fix(router): skip route finder when visor has no transports fast-fails before the retry loop when the manager reports zero transports and no background creation is pending — provably no route to find, so it returns no transports available instead of six wasted round-trips.

Two reliability fixes on the browser edge’s default proxy. 3745 feat(wasm-visor): resilient skysocks-lite — sticky reconnect + liveness stops the default from abandoning a working exit on a blip: an everConnected bit per exit branches reportProxyExitDead so an exit that had connected gets a sticky exponential-backoff reconnect on the same key (1s→30s, 5 tries) before rotating, while one that never connected still rotates immediately; a proxyKeepaliveLoop yamux-pings the active exit every 15s and keeps standbys warm. 3770 fix(wasm-visor): tighter autoconnect dial timeouts frees concurrency slots ~3× faster — a hung, advertised-but-unreachable WT endpoint (the fleet’s common case) squatted a slot for a full 25s; the handshake timeout drops to 8s and the AR resolve gets its own 10s budget, so the direct-first pass attempts far more peers per cycle.

Skywire: Shared visorcore for Native and Edge

The transport.Manager was hand-constructed twice — native init_transport.go and the browser cmd/wasm-visor/main.go — each free to drift on ManagerConfig fields (the edge silently omitted Version). 3764 feat(visorcore): shared BuildTransportManager adds visorcore.BuildTransportManagerNewManager → optional InitDmsgClient → optional Serve → dial-only InitClient loop — one seam both shells call, with ARClient/EB kept typed any so addrresolver/appevent never enter the TinyGo graph; native passes Serve:false and the edge Serve:true with DialOnlyClients{WS,WT,WEBRTC}. 3765 feat(visorcore): shared DmsgServicePKs converges the other drift-prone piece: the seven deployment services that run as dmsg direct clients (dmsgd, tpd, ar, rf, sd, conf, ut) never publish to discovery, so each visor must seed their entries or a DialStream 404s — and that set was derived twice, with both comments warning they MUST match after a past omission left the edge’s network view empty. visorcore.DmsgServicePKs(Services) now parses the service URLs to PKs in one place. Both are guarded by the enforced GOOS=js build gate and validated on a live native visor booting healthy on the new path with an all-p2p data plane.

Skywire: The Guided Tour, HV UI, tpviz, and Mobile

The wasm guided tour became navigation-driven: 3752 opens the real tabs and spotlights the thing rather than pointing at a static overview, 3753 has it visit each overview tab, 3755 reflows it front-page-first with node-list cell hooks, and 3757/3758 simulate a managed cluster and add app-window demos for the “Your cluster, live” step. On the hypervisor UI, 3754 adds a dmsg-server carrier column, wasm tab hiding, and services-health graceful degrade, with 3756 populating that carrier column in the node-list tree path; 3750 de-dups the log timestamp/level and adds a wasm fresh-worker reload; 3749 feat(wasmhv): ctl-bridge dials /ctl/rpc so the CLI can drive a wasm tab; and 3751 fixes svc health to read the DMSG server version via a discovery-routed probe. Three tpviz fixes tighten lifecycle: 3742 stops all timers/rAF loops and the WebSocket on unmount, 3743 suspends data sync and rendering while the visualizer isn’t visible, and 3744 removes the window/document listeners on unmount. On the plumbing side, 3762 pin-gates a benign sendLastRoot "no such head" WARN in CXO, and 3740/3741 clear lint regressions from the config-parity and scorecard merges and replicate the open Dependabot bumps (CI actions + postcss).

The mobile app landed several large pieces (no PR numbers — direct commits). Fleet turns the phone into a read-only window onto the visors it runs elsewhere: enabling it hands the phone’s own hypervisor a dmsg client, which starts the RPC listener remote visors dial into, and the one new action — POST /api/visors/{pk}/restart, wrapping the existing Reload RPC — answers 202 rather than 200 because Reload tears down the very RPC transport the call rides. A polling bug is recorded honestly: a 5s visors-summary poll made each visor fan a Summary RPC to every remote over dmsg, exceeding the server’s own 5s budget and stacking calls until the stream broke, cycling a peer through evict/redial once a minute while the summary cache still rendered it green — fixed by polling every 15s, a three-try 503 restart, and rendering last_seen_at under a green dot once a snapshot is over 45s old. SkyVPN puts the whole phone through a Skywire exit — the one part that needed new Go, because Android hands out /dev/net/tun only through VpnService after user consent, so SkyVpnService creates the interface and passes its descriptor to the child visor over an abstract unix socket as SCM_RIGHTS; the killswitch is real rather than advisory because the service keeps its own copy of the descriptor, and addDisallowedApplication keeps the tunnel-carrying dmsg traffic (same UID as the app) out of the tunnel it creates. skychat gained a Settings screen and cross-device portability: export writes one JSON document (address book + every stored message), import merges it back, drawing the line at data vs identity — messages and the names you gave keys travel, the keypair and group membership do not — with the reply honestly counting records dropped by the retention window and the per-peer cap so “imported 2000” while 500 survive can’t mislead someone into wiping their old device too early. Smaller skychat fixes kept a file in Saved Messages from being sent to a non-Saved key, added playback speed, and stopped the media-controls WebView from posting itself.

Skycoin: Dependencies and Policy Alignment

2990 build(deps): batch web-subproject Dependabot bumps replicates all nine open Dependabot PRs — the gui and skycoin-web npm groups plus hono and fast-uri across electron/explorer/gui/skycoin-web — in one branch so they land together instead of through nine sequential rebases, each hunk the exact gh pr diff from its Dependabot PR. 2989 docs: SECURITY.md + README badge alignment brings the repo into line with skywire: it adds a SECURITY.md (routing reports to the @skywire Telegram channel and GitHub’s “Report a vulnerability” flow), removes the stale Responsible Disclosure section that pointed at an unmonitored address and a PGP key expired since 2023, and swaps the retired Go Report Card / GoDoc badges for the GitHub-release, OpenSSF Scorecard, go.mod-version, and Telegram badges skywire already carries.