SDK — building a backing app

A “backing app” is anything that renders into a tmnl tab without speaking ANSI. It connects to tmnl over a Unix socket, sends typed Frames of cells, and receives Input / Resize events back. That’s the shape mnml uses to appear as a native tab; the shape mixr uses; the shape anyone writing a new TUI for tmnl uses.

The bytes on the wire are documented in tmnl-protocol and walked through in Native tabs. This page is about the published helper layer that wraps those bytes — a Client struct, a single poll() loop, and a small Event enum that filters out everything you don’t care about. A complete echo client fits in ~25 lines of code.

The page closes with the capability bitmask on Hello (protocol v6+): how features opt in additively across protocol versions, what Caps::CLIENT_COMMANDS / FOCUS_EVENTS / HOVER_EVENTS / IME_EVENTS mean today, how to opt in via Client::connect_with_caps, and how to read the server’s reply via Client::peer_caps.

What the SDK gives you

The SDK lives in tmnl_protocol::client (re-exported from the crate root as the client module). It exposes one struct and two enums:

use tmnl_protocol::client::{Client, Event, ControlFlow};

• Client owns the connection, a double-buffered cell grid, the frame sequence counter, and the default colors used by clear() and put_str().
• Event is the user-facing event enum — what the app gets back from poll(). See The Event enum below.
• ControlFlow is Continue or Quit, returned from the per-tick callback in Client::run_loop. Apps that drive their own loop via poll() ignore this.

The SDK is shipped from tmnl-protocol >= 0.0.6 (the same release that introduces this module). Add it to Cargo.toml:

[dependencies]
tmnl-protocol = "0.0.9"

Version notes worth knowing before you upgrade:

• 0.0.8 added Client::connect_with_caps + Client::peer_caps so apps can advertise (and read) capability flags through the handshake. See Opting into capabilities.
• 0.0.9 added the rich native-mode input variants (InputEvent::Focus / Hover / Ime) and made InputEvent no longer Copy — the Ime variant owns a String. Any send_input(ev) site needs &Message::Input(ev.clone()) instead of a by-value copy. See Rich native-mode input.

A complete backing app

hello_client — the canonical minimal example shipped at examples/hello_client.rs — is the whole template:

use tmnl_protocol::client::{Client, ControlFlow, Event};
use tmnl_protocol::{InputEvent, KeyCode};

fn main() -> std::io::Result<()> {
    let socket = std::env::args()
        .nth(1)
        .expect("usage: hello_client <socket-path>");

    let mut client = Client::connect(&socket)?;
    client.set_title("hello")?;

    let mut typed = String::new();
    client.run_loop(|c, ev| {
        if let Event::Input(InputEvent::Key(k)) = ev
            && k.press
        {
            match k.code {
                KeyCode::Esc => return ControlFlow::Quit,
                KeyCode::Enter => typed.clear(),
                KeyCode::Backspace => { typed.pop(); }
                KeyCode::Char(ch) => typed.push(ch),
                _ => {}
            }
        }
        let (cols, rows) = c.size();
        c.clear();
        let prompt = format!("> {typed}");
        c.put_str(rows / 2, 2, &prompt);
        c.set_cursor(rows / 2, (2 + prompt.chars().count() as u16).min(cols - 1));
        let _ = c.flush();
        ControlFlow::Continue
    })
}

That’s it. Connect, set a title, loop over events, draw cells. The same file pre-SDK was 188 lines of hand-rolled handshake, message reading, frame construction, and seq-bumping. The SDK collapses 90% of that into Client::connect + Client::flush.

Run it the usual way:

# Terminal A — tmnl's stub server
cargo run --example fake_server  -- /tmp/dev.sock

# Terminal B — your app under development
cargo run --example hello_client -- /tmp/dev.sock

Or attach to a real tmnl with --no-launch (see Native tabs).

What Client::connect actually does

Client::connect(socket) is more than UnixStream::connect. The handshake sequence is:

1. Open the socket — UnixStream::connect(socket).
2. Clone the stream into a separate reader / writer pair so reads and writes don’t fight over one buffered I/O state.
3. Send Message::Hello { version: PROTOCOL_VERSION, caps: Caps::empty() }. The default caps bitmask is empty — see Capability negotiation for what “opt-in caps” mean and Opting into capabilities for the connect_with_caps variant that advertises a non-empty set.
4. Drain the reader until the first Message::Resize arrives. That’s the signal that tmnl knows the cell grid dimensions. While draining, capture the server’s Hello caps so peer_caps reflects what the server itself opted into. Any further Hello echo is silently discarded; a Quit returns ConnectionAborted.
5. Apply the Resize — grow the local cell buffer to cols × rows blank cells at the default fg / bg.
6. Return the Client ready to draw.

After connect() returns, the grid dimensions are known via client.size(), the server’s advertised caps via client.peer_caps(), and the app can issue its first clear() + put_str() + flush() cycle.

The mutation API

The SDK exposes a small set of grid mutators:

| Method | Purpose | | --- | --- | | clear() | Fill every cell with the default fg / bg. Apps typically call this at the start of each render tick. | | put_str(row, col, s) | Write s at (row, col), one cell per char. Clips at the right edge. Uses default fg / bg. | | put_str_styled(row, col, s, fg, bg) | Variant with explicit per-call fg / bg. | | put_cell(row, col, cell) | Stamp one styled WireCell. Useful for hand-rolled chrome where attrs (bold / italic / underline) matter. | | cell_at(row, col) -> Option<WireCell> | Read back the current cell. Used by tests and apps that render incrementally instead of clearing every frame. |

Colors are packed u32 — (R<<24) | (G<<16) | (B<<8) | A. Build them with the crate’s pack_rgba_u8(r, g, b, a) / pack_rgba(r, g, b, a) helpers, or use the constants the SDK ships with.

Default colors

Client::set_default_colors(fg, bg) overrides the colors used by clear() and put_str(). The shipped defaults are a slate-grey background close to mnml’s bg_dark and an off-white foreground:

// From src/client.rs:
const DEFAULT_BG: u32 = 0x1012_16ff; // pack_rgba_u8(0x10, 0x12, 0x16, 0xff)
const DEFAULT_FG: u32 = 0xe6e8_ecff; // pack_rgba_u8(0xe6, 0xe8, 0xec, 0xff)

When tmnl is hosting your app and wants to hand you its active palette, the event loop surfaces a Palette { bg, fg, accent } event — that’s the cue to set_default_colors(fg, bg) so your chrome blends into tmnl’s.

Cursor state

The cursor is part of the Frame payload, not a separate message:

| Method | Effect | | --- | --- | | set_cursor(row, col) | Position the cursor. Clamped to grid bounds. | | set_cursor_visible(on) | Hide / show the cursor. Default is shown. | | set_cursor_shape(shape) | 0 block, 1 bar, 2 underline. |

flush() reads these values into the outgoing Frame. tmnl overlays the cursor on the correct cell during render — your app never draws the cursor glyph itself.

The event loop

Two equivalent styles, depending on how much control your app wants.

Style 1: poll()

Client::poll() blocks until the next user-facing event arrives. It auto-handles the messages your app doesn’t care about — Hello echoes, server-direction messages your app isn’t supposed to receive — and applies Resize to the local cell buffer before surfacing the event, so by the time you observe Event::Resize, client.size() already reflects the new dimensions:

loop {
    match client.poll()? {
        Event::Input(input) => handle(input),
        Event::Resize { cols, rows } => log::info!("resized to {cols}x{rows}"),
        Event::Quit => break,
        Event::Palette { fg, bg, .. } => client.set_default_colors(fg, bg),
        _ => {}
    }
    // draw + flush
    client.clear();
    /* ... */
    client.flush()?;
}

Style 2: run_loop

For simple apps, Client::run_loop(|client, ev| -> ControlFlow) is a one-call event loop. The callback receives each event and a &mut Client so it can draw and flush. Return ControlFlow::Quit to break the loop; returning Continue keeps it spinning. Event::Quit from the server exits without invoking the callback for that event:

client.run_loop(|c, ev| {
    if matches!(ev, Event::Input(InputEvent::Key(k)) if k.press && k.code == KeyCode::Esc) {
        return ControlFlow::Quit;
    }
    c.clear();
    c.put_str(0, 0, "hello");
    let _ = c.flush();
    ControlFlow::Continue
})?;

run_loop is the hello_client style. poll is what mnml’s blit backend uses, because it interleaves the protocol read with its own multiplexed input / ratatui-redraw / file-IPC state machine.

The Event enum

Client::poll() returns one of:

| Variant | Meaning | | --- | --- | | Event::Input(InputEvent) | Key press / mouse event from tmnl. InputEvent::Key and InputEvent::Mouse are decoded already — no ANSI parsing on the app side. | | Event::Resize { cols, rows } | The grid was resized. The local cell buffer has already been re-sized; the app should redraw + flush. | | Event::Quit | tmnl is closing the connection. Drain and exit cleanly. | | Event::Palette { bg, fg, accent } | tmnl handed the app its active chrome palette. Re-theme to match by calling set_default_colors(fg, bg). | | Event::RunClientCommand(id) | tmnl is invoking one of your registered commands by id (see capability CLIENT_COMMANDS below). | | Event::ListClientCommands | tmnl is asking for your command registry — respond by sending Message::ClientCommands(Vec<CommandInfo>) on the raw layer. |

Server-direction messages (Frame, Title, OpenPane, …) are filtered out — they aren’t yours to receive on the client side. Raw-protocol clients can still see them via read_message.

Flushing a frame

Client::flush() serializes the current cell buffer into a Message::Frame and ships it over the wire:

• One DiffRun { start: 0, cells: <whole grid> } — sends every cell every flush. Tmnl is fine with this; it’s the easiest correct approach.
• seq increments automatically. The next flush() carries the bumped value.
• Cursor state (row, col, shape, visible) rides on the same Frame.

For apps that know exactly which cells changed, the raw layer accepts multiple DiffRuns per frame — the SDK’s flush() doesn’t try to compute a diff for you. Drop down to write_message(&mut writer, &Message::Frame(...)) directly if minimum bytes-on-the-wire matters.

Capability negotiation

Hello carries an opt-in capability bitmask as of protocol v6.

The problem the bitmask solves: protocol additions (OpenPane, Palette, OpenPaneTransfer, RunHostCommand, ListClientCommands, …) historically got detected by message tag — a renderer that didn’t know a tag rejected it. Useful as an error path; not useful for “the renderer wants to know whether to send a feature in the first place.” Capability bits let both sides advertise what they support up-front, and the server can quietly skip features the client hasn’t opted into instead of probing blindly.

The Caps type

Caps is a thin wrapper around a u64:

use tmnl_protocol::Caps;

let nothing = Caps::empty();
let commands = Caps::CLIENT_COMMANDS;
let both = nothing.union(commands);
assert!(both.contains(Caps::CLIENT_COMMANDS));

let shared = peer_caps.intersection(my_caps);

Set ops (union, intersection, contains) are const. The full API is empty() / from_bits(u64) / bits() / contains() / intersection() / union().

The flags

| Flag | Bit | What it gates | | --- | --- | --- | | Caps::CLIENT_COMMANDS | 1 << 0 | ListClientCommands → ClientCommands → RunClientCommand round-trip. Tmnl’s palette aggregates the hosted app’s commands alongside its own. See Rich native-mode input — the gated probe lives below. | | Caps::FOCUS_EVENTS | 1 << 1 | Server delivers InputEvent::Focus(FocusEvent) events. See Rich native-mode input. | | Caps::HOVER_EVENTS | 1 << 2 | Server delivers InputEvent::Hover(HoverEvent) cell-level pointer events. | | Caps::IME_EVENTS | 1 << 3 | Server delivers InputEvent::Ime(ImeEvent) composition events. | | Caps::INLINE_IMAGES | 1 << 4 | The client may send InlineImageCreate / InlineImageUpdate / InlineImageDestroy to overlay raster images on the cell grid. Added in protocol v7. See Inline images. |

Adding a new flag is straightforward: claim the next bit in the Caps impl, gate the new behaviour on peer_caps.contains(CAP_NEW), and bump the docstring listing known flags.

Opting into capabilities

Client::connect_with_caps(socket, caps) advertises the given capability set in the outbound Hello. Client::connect(socket) delegates to connect_with_caps(socket, Caps::empty()) — the no-opt-in default.

use tmnl_protocol::client::Client;
use tmnl_protocol::Caps;

// Default: no opt-in.
let mut a = Client::connect(&socket)?;
assert_eq!(a.peer_caps(), Caps::CLIENT_COMMANDS); // tmnl advertises this

// Advertise specific caps so the server starts delivering Hover events.
let mut b = Client::connect_with_caps(&socket, Caps::HOVER_EVENTS)?;
# Ok::<_, std::io::Error>(())

The server treats the latest Hello it received as authoritative — sending a follow-up Hello with a different caps set via write_message is supported if your app needs to renegotiate mid-session.

Reading the server’s caps

Client::peer_caps() returns whatever the server advertised on its incoming Hello. Captured during the connect handshake; same value for the life of the connection (the SDK doesn’t surface re-Hello mid-session yet):

let client = Client::connect(&socket)?;
if client.peer_caps().contains(Caps::CLIENT_COMMANDS) {
    // OK to expose a "Send commands to tmnl" path.
}
# Ok::<_, std::io::Error>(())

A pre-v6 server’s Hello carries no caps trailer; the SDK decodes that to Caps::empty(). So peer_caps().contains(X) is always safe — a missing trailer reads as “no caps advertised.”

What advertises what today

| Side | Caps | | --- | --- | | tmnl-as-server | CLIENT_COMMANDS | | mnml-as-blit-client | CLIENT_COMMANDS (so its commands surface in tmnl’s palette) | | mixr / pane-host children | Caps::empty() (no upward command registry) | | Client::connect default | Caps::empty() | | Client::connect_with_caps(_, caps) | The caller-supplied set |

Server-side gating in tmnl

Tmnl’s server reads the incoming Hello’s caps into an Arc<AtomicU64> field on Server and exposes it via Server::peer_caps(). Features that depend on a cap call a _if_supported variant:

src/server.rs

pub fn send_list_client_commands_if_supported(&self) -> bool {
    if self.peer_caps().contains(Caps::CLIENT_COMMANDS) {
        self.send_list_client_commands();
        true
    } else {
        false
    }
}

The palette aggregator in App calls the gated variant for every Native pane when the palette opens, so panes whose client never advertised CLIENT_COMMANDS skip the round-trip entirely — no wasted ListClientCommands message sitting in a reader thread that will never reply. Concretely: mnml panes get the probe; mixr / pane-host child panes don’t.

Back-compat on the wire

The v6 Hello payload is version (4 bytes) + caps (8 bytes). Pre-v6 senders only wrote the 4-byte version.

The decoder probes the remaining payload length. When the 8-byte trailer is present it reads it; when it’s absent it defaults caps to Caps::empty():

// From tmnl-protocol src/lib.rs decode_payload:
MSG_HELLO => {
    let version = c.u32()?;
    let caps = if c.remaining() >= 8 {
        Caps::from_bits(c.u64()?)
    } else {
        Caps::empty()
    };
    Ok(Message::Hello { version, caps })
}

A v5 client talking to a v6 renderer still works — the renderer just sees caps = empty and skips any opt-in feature paths. Same on the reverse direction. The protocol grows additively without forcing a flag day.

Rich native-mode input

InputEvent::Key and Mouse are the two variants every backing app sees by default. Three more variants — Focus, Hover, Ime — arrived in tmnl-protocol 0.0.9 to carry the kinds of OS-level input that shell-mode terminals can’t represent at all. Each is opt-in via a matching Caps flag; servers don’t deliver these to clients that didn’t advertise the flag, so the input channel stays quiet for apps that don’t care.

Focus(FocusEvent { gained })

Window-focus gained on the tab (gained = true) or lost (gained = false). Fires when the user clicks into the tmnl window, returns via Cmd+Tab, or switches to another app. Gated by Caps::FOCUS_EVENTS.

Use case: a media app that pauses on blur and resumes on focus; an editor that flushes a save buffer when the window backgrounds.

Hover(HoverEvent)

Cell-level pointer tracking in three phases:

pub enum HoverEvent {
    Entered { col: u16, row: u16 },
    Moved   { col: u16, row: u16 },
    Left,
}

Entered fires when the pointer first crosses a cell inside the tab area, Moved for every cell traversal while inside, Left when the pointer leaves the tab area entirely. Coords are in cell-grid space — same units as MouseInput.col / row. Gated by Caps::HOVER_EVENTS.

Use case: tooltips on hover, link previews, button highlight states without the user having to click. Servers gate this on opt-in because the event volume is high — without the gate every pointer motion across the cell grid would land in your event queue.

Ime(ImeEvent)

IME composition with three phases:

pub enum ImeEvent {
    Preedit(String),  // in-flight composition (may be empty)
    Commit(String),   // finalized text
    Cancel,           // aborted mid-composition
}

Preedit carries the in-flight composition string the user is typing — "n" → "ni" → "你" in pinyin, for example. Commit arrives once the IME finalizes; Cancel fires if the user aborts. Gated by Caps::IME_EVENTS.

Without the cap, servers still deliver finalized IME text through ordinary Key(Char) events — so most apps don’t need to opt in. Apps that want preedit feedback (a text editor, a search box with live preview) opt in to draw the composition string before the user commits.

Preedit can carry an empty String — IMEs use that to clear an in-flight composition without committing. Treat empty preedit as “clear my preedit state,” not “ignore.”

A worked Hover example

Client::connect_with_caps(socket, Caps::HOVER_EVENTS) advertises the capability in the outbound Hello:

use tmnl_protocol::client::{Client, Event};
use tmnl_protocol::{Caps, HoverEvent, InputEvent};

let socket = std::env::args().nth(1).unwrap();
let mut client = Client::connect_with_caps(&socket, Caps::HOVER_EVENTS)?;
client.set_title("hover demo")?;

loop {
    match client.poll()? {
        Event::Input(InputEvent::Hover(HoverEvent::Entered { col, row })) => {
            log::info!("entered cell ({col}, {row})");
        }
        Event::Input(InputEvent::Hover(HoverEvent::Moved { col, row })) => {
            log::trace!("hover at ({col}, {row})");
        }
        Event::Input(InputEvent::Hover(HoverEvent::Left)) => {
            log::info!("pointer left tab");
        }
        Event::Quit => break,
        _ => {}
    }
    // …draw + flush…
    # break;
}
# Ok::<_, std::io::Error>(())

Combine flags with Caps::union:

let caps = Caps::HOVER_EVENTS
    .union(Caps::FOCUS_EVENTS)
    .union(Caps::IME_EVENTS);
let mut client = Client::connect_with_caps(&socket, caps)?;
# Ok::<_, std::io::Error>(())

Mnml currently opts in to only CLIENT_COMMANDS

Mnml’s blit client advertises Caps::CLIENT_COMMANDS on its Hello so mnml commands surface in tmnl’s palette. It does not opt into Focus / Hover / Ime — mnml’s own input pump synthesizes focus tracking from chord context and doesn’t need OS-level hover (its cursor renders inside the cell grid, not as a system pointer overlay).

Backing apps that need any of those three call connect_with_caps with the right flags and the server starts emitting the corresponding InputEvent variants.

How the renderer emits these — the App-side story

Tmnl’s winit pump is the source of Focus / Hover / Ime events. The protocol contract is “advertise the cap, receive the event”; this section is the other half of that contract — how tmnl’s App decides when to emit, to which pane, and whether to emit at all.

A single helper is the chokepoint for every rich-input dispatch:

src/app.rs

fn forward_input_if_cap(
    &mut self,
    tab_idx: usize,
    pane_idx: usize,
    cap: tmnl_protocol::Caps,
    ev: tmnl_protocol::InputEvent,
) {
    // No-op if the pane isn't Native, no client is connected,
    // or the client never advertised `cap` on its Hello.
    // Otherwise calls server.send_input(&ev).
}

Every rich-input emission path calls this. The cap check lives in exactly one place, so a peer that opted into FOCUS_EVENTS but not HOVER_EVENTS reliably sees only the events it asked for — there’s no duplicate gate to drift between subsystems.

Focus emission

WindowEvent::Focused(gained) walks the entire tab tree and fires InputEvent::Focus(FocusEvent { gained }) to every Native pane whose peer advertised Caps::FOCUS_EVENTS. A window can host several Native panes spread across tabs — each carries its own peer-caps cell on its Server, so the per-pane cap check matters even within one window-blur event.

Window-blur also fires a synthetic Hover::Left to whatever pane was last tracked as hovered, then clears the cell. The rationale: if the user alt-tabs away mid-hover, the peer shouldn’t keep painting a hover highlight on a cell the pointer has effectively left. Re-enters fire on the next CursorMoved.

Hover emission — the state machine

WindowEvent::CursorMoved runs through App::emit_hover_after_cursor_move, a four-state machine driven by last_hover_native: Option<(pane_idx, col, row)>:

| Previous | Current | Emitted | | --- | --- | --- | | None | None (chrome, divider, Shell pane) | nothing | | None | Some(pane, col, row) (Native pane) | Entered { col, row } to the new pane | | Some(prev) | None | Left to the previous pane | | Some(prev) | Some(pane), prev != pane | Left to previous, then Entered to new | | Some(prev, c0, r0) | Some(prev, c1, r1), (c0,r0) != (c1,r1) | Moved { c1, r1 } | | Some(prev, c0, r0) | Some(prev, c0, r0) (same cell) | nothing — per-pixel-within-cell moves are dropped |

The “same pane, same cell” suppression is what keeps the wire quiet. A trackpad gesture at 120 Hz across one cell would otherwise pump 120 Moved events per second through a serial channel to a client that only cares about cell-level granularity. Hover-aware peers see one event per cell traversal — enough resolution for tooltips and cell highlights, none of the noise.

last_hover_native is tracked regardless of any peer’s caps so a mid-hover cap upgrade (a follow-up Hello advertising HOVER_EVENTS mid-session — the SDK doesn’t expose this today but the raw layer does) doesn’t fire a spurious Entered because the state machine “forgot” the pointer was already parked on a cell.

WindowEvent::CursorLeft fires Hover::Left to the tracked pane and clears the cell — the symmetric counterpart to Entered on first window-enter.

IME emission

WindowEvent::Ime routes to App::handle_ime. Three branches:

• Ime::Preedit(text, _cursor) — the in-flight composition. If the focused pane is a Native pane whose client advertised Caps::IME_EVENTS, fire Ime::Preedit(text) over the wire. Empty preedit (the IME clearing the composition without committing) is a valid Preedit("") event — peers should treat it as “clear my preedit,” not “ignore.”
• Ime::Commit(text) — the finalized string. IME-aware Native panes get a single Ime::Commit(text) event. Legacy peers (no IME_EVENTS cap) get the historical fallback — per-char Key(KeyCode::Char(c)) events, one per char in the committed string. So apps that never opt in keep working — they just don’t see preedit state.
• Ime::Disabled — IME composition was aborted (the user hit Escape mid-preedit, switched IME engines, …). Fire Ime::Cancel to IME-aware Native panes and clear the in-flight preedit cell.

The dispatch site for each branch is forward_input_if_cap (or the inline cap-aware fan-out for Commit, which has the legacy fallback path baked in).

Putting it all together — the files_client worked example

examples/files_client.rs is the reference client that exercises all three rich-input cap flags. It’s a ~340-line TUI file picker — directories first, then files, alphabetical — and it’s the second reference client (after mnml) for the SDK. The PLAN.md target was “prove the protocol isn’t just mnml’s renderer with an API,” and this delivers on it.

// examples/files_client.rs — the opt-in
use tmnl_protocol::client::Client;
use tmnl_protocol::Caps;

let caps = Caps::FOCUS_EVENTS.union(Caps::HOVER_EVENTS);
let mut client = Client::connect_with_caps(&socket, caps)?;

Two-axis row highlighting is the central interaction. kb_idx tracks the keyboard selection (↑ / ↓); hover_idx tracks the most recent Hover::Entered / Moved cell. Each gets a distinct background:

• Keyboard row (kb_idx) — warm yellow fill, dark text. The “where Enter goes” row.
• Hover row (hover_idx) — neutral grey fill, normal text. The “where the pointer is” row.

Both can be present at once — keyboard selection at row 3, pointer hover at row 7 — and the user sees them as distinct visual cues. The Hover::Moved events update hover_idx cell-by-cell, repainting only when the row index changes. Hover::Left (window-blur or CursorLeft) clears hover_idx back to None so the highlight disappears the moment the pointer wanders off.

The header row reflects Focus:

let mut header = format!("📁 {}", st.cwd.display());
if !st.focused {
    header.push_str("  (blurred)");
}

Focus(gained = false) flips state.focused; the next render paints the (blurred) hint after the path. Cosmetic — the picker still works — but it proves the focus signal landed at the peer and the peer reacted.

Input handling is plain — ↑ / ↓ move the keyboard cursor, ← goes to the parent dir, → enters the focused directory, Enter opens the focused file (prints its absolute path to stderr and exits 0) or descends if it’s a directory, Esc / q cancels (exits 1).

Run it locally against fake_server:

# Terminal A — tmnl stub
cargo run --example fake_server  -- /tmp/files.sock

# Terminal B — the picker
cargo run --example files_client -- /tmp/files.sock

The fake_server stub doesn’t fire real Hover events (it has no window, no pointer), so to exercise the hover path you need real tmnl hosting the socket. Launching the picker against a tmnl process with HOVER_EVENTS advertised on the server side surfaces the live cell tracking.

InputEvent is no longer Copy (0.0.8 → 0.0.9 breaking change)

The 0.0.9 release added the Ime variant, which owns a String. That makes InputEvent Clone but no longer Copy. Code that matched on InputEvent by value continues to compile, but code that forwarded an InputEvent to another writer needs a one-line update:

// Pre-0.0.9 — InputEvent: Copy
write_message(&mut s, &Message::Input(ev))?;

// 0.0.9+ — InputEvent: Clone
write_message(&mut s, &Message::Input(ev.clone()))?;

Tmnl’s own server (Server::send_input) clones at the single dispatch site — the event lives on the stack for one call, not a hot loop where the clone would matter.

Inline images

Protocol v7 adds three client→server messages that let a backing app overlay raster images on the cell grid: InlineImageCreate, InlineImageUpdate, InlineImageDestroy. Tmnl decodes the bytes once on receipt, uploads them to a wgpu texture, and renders the result as a textured-quad pass over the cell grid each frame. The image is anchored at a cell coordinate inside the pane — scroll the pane and the image moves with the cells underneath.

The feature is gated by Caps::INLINE_IMAGES. Tmnl advertises this in its server Hello from v7 onward; clients that want to send images should advertise it on the outbound Hello so the server knows it can expect the messages, and gate sends on peer_caps() so pre-v7 servers don’t see ignored traffic.

Sending an image

The SDK exposes three thin wrappers that mirror the protocol messages:

use tmnl_protocol::client::Client;
use tmnl_protocol::{Caps, ImageFormat};

let mut client = Client::connect_with_caps(&socket, Caps::INLINE_IMAGES)?;
if !client.peer_caps().contains(Caps::INLINE_IMAGES) {
    // Pre-v7 server — drop the feature.
    return Ok(());
}

let png_bytes: Vec<u8> = std::fs::read("logo.png")?;
client.send_inline_image_create(
    /* id        */ 1,
    /* anchor_col*/ 4,
    /* anchor_row*/ 2,
    /* cell_w    */ 12,
    /* cell_h    */ 6,
    /* format    */ ImageFormat::Png,
    /* bytes     */ png_bytes,
)?;
# Ok::<_, std::io::Error>(())

The id is client-allocated and opaque to tmnl — pick whatever scheme suits the app (a monotonic counter is fine). The same id later identifies the image for Update or Destroy. Anchor + cell box are cell coordinates relative to the pane’s top-left; the image scales to fit cell_w × cell_h cells regardless of the source PNG’s pixel dimensions.

Replace an image’s bytes without moving it:

let new_png = re_render_chart();
client.send_inline_image_update(1, new_png)?;

Remove an image:

client.send_inline_image_destroy(1)?;

Tmnl quietly drops any Update or Destroy whose id was never created — no error path to handle.

What renders, and how

On Create / Update, tmnl decodes the bytes (image crate, PNG-only today — the ImageFormat enum reserves room for more formats later) into RGBA8 and uploads to a wgpu::Texture tied to the pane. On every frame, the pane’s compositor walks inline_images and emits one textured-quad instance per image into a shared instance buffer; the quad alpha-blends over whatever the cell grid painted underneath, so transparent pixels in the PNG show the cell content through.

A decode failure leaves the image’s decoded slot as None. The cell-grid compositor then paints a dim-red placeholder block with a nf-md-image_broken glyph at the image’s anchor rect so the user can see something went wrong without the host crashing.

The image is anchored at the pane’s top-left, so it moves with the pane on splits and resizes. If the anchor scrolls off-screen entirely (zero- size after clipping), the GPU draw is skipped — no leak into chrome.

Worked example

The inline_image_client example in the tmnl repo ships an end-to-end demo. It encodes a small gradient PNG in memory via the image crate, sends it, and cycles the gradient or destroys it on keypress. Run it against a real tmnl process (the fake_server stub doesn’t render images):

# Terminal A — start tmnl, note the socket path
tmnl
# (the launcher prints something like: socket=/tmp/tmnl-1234.sock)

# Terminal B — point the client at it
cargo run --example inline_image_client -- /tmp/tmnl-1234.sock

Keys:

• space — re-encode the gradient with a shifted hue and send Update (or re-Create if you previously destroyed it).
• d — send Destroy. The image disappears; the placeholder doesn’t paint because nothing’s tracked anymore.
• Esc — quit.

The image appears as a 10-cell × 4-cell block at column 2, row 2 of the tab. The block is real pixels from the GPU pass — not a placeholder.

Lifecycle + caching

Inline images live on the receiving pane and disappear when it closes (no per-pane “save these forever” semantics — Destroy is explicit, or pane close drops them). Tmnl’s GPU texture cache survives a tab switch: each image gets a stable gpu_key assigned at creation, the cache keys on that, and only entries whose key drops out of the live set across all tabs get evicted. So flipping between two tabs that each show an image doesn’t thrash the GPU; only Destroy or pane close frees the texture.

The image_storage_gpu_limit config knob caps the GPU byte total for the Kitty-graphics path; the v7 inline-image path is not bounded by it today (a typical SDK client sends a small handful of images, not the hundreds-of-frames-per-second flow Kitty’s animation protocol allows). A future cap-on-the-v7-side would be cheap to add if a real consumer ever needs it.

Smoke tests for the SDK

Three integration tests in tests/sdk_roundtrip.rs spin up an in-process UDS server that mimics tmnl’s handshake (Hello with caps → Resize → optional Input) and drive the published Client against it. They’re the runnable reference for end-to-end SDK use:

| Test | What it asserts | | --- | --- | | sdk_handshake_resize_then_first_event | connect_with_caps(Caps::CLIENT_COMMANDS) round-trip — initial Resize lands in client.size(), server’s advertised caps land in client.peer_caps(), an inbound Key('a') arrives as Event::Input, and a Frame flushed from the client is observed by the server reader thread with seq = 0 and a non-empty cell count. | | sdk_connect_default_advertises_empty_caps | The plain Client::connect path’s Hello carries Caps::empty() — the server’s reader thread sees that exact value. | | sdk_resize_event_updates_size_and_resizes_grid | The initial Resize propagates to client.size() so a clear() + put_str(0, 0, "x") + flush() works first frame. |

The test file is also a copy-pastable template for any backing app that wants a smoke test of its own — spawn_mock_server is a generic mock that scripts in a Vec<InputEvent> to deliver after the handshake and returns a channel of ServerEvents the test can assert on.

Run the tests:

cd tmnl-protocol
cargo test --test sdk_roundtrip

When to drop to raw protocol

The SDK is the right starting point for ~95% of backing apps. The raw layer is exposed for the remaining few:

• Sophisticated diff-based frame construction. The SDK ships every cell every flush; an app that does its own dirty tracking will want to emit multiple DiffRuns.
• OpenPane / OpenPaneTransfer / OpenFile / RunHostCommand. These are app → server messages the SDK doesn’t expose helpers for yet. Use write_message(&mut writer, &Message::OpenPane { command, args }) directly.
• Mid-session Hello renegotiation. The SDK only sends one Hello (during connect_with_caps). The server treats the latest Hello it received as authoritative, so changing the caps set after connect needs a hand-rolled write_message.

The raw API is read_message / write_message / send_message_with_fd / read_message_with_fd — see examples/fake_server.rs for a hand-rolled walkthrough that doesn’t use the SDK at all.

Smoke-testing without a GPU

Two example binaries fake each side of the protocol so you can develop without launching real tmnl:

# Terminal A — tmnl stub: binds the socket, scripts input, prints frames.
cargo run --example fake_server -- /tmp/dev.sock

# Terminal B — your app under development.
cargo run --example hello_client -- /tmp/dev.sock

fake_server is hand-rolled raw protocol — no SDK. It’s the reference for SDK authors who want to see the wire dance directly. fake_client exercises tmnl’s server side without a backing app.

Where to go next

• Native tabs — what a native tab actually is, the full message catalog, and how a backing app gets launched as a tab.
• tmnl-protocol — the wire-format crate. The SDK lives in src/client.rs; Caps and the v6 wire layout in src/lib.rs.
• examples/hello_client.rs — copy this file as the starting point for your own backing app.
• examples/files_client.rs — the second reference client; a working TUI file picker that exercises FOCUS_EVENTS + HOVER_EVENTS end-to-end.
• examples/fake_server.rs — raw-protocol reference for SDK authors needing to see the wire format without abstraction.
• mnml — the reference client; src/blit/mod.rs is the BlitBackend pattern for ratatui apps. mnml advertises Caps::CLIENT_COMMANDS on its Hello so its command palette surfaces in tmnl’s.
• FEATURES.md — the shipped-feature inventory.