Very cool! But I’m just a bit sad that this is “just” a set of effects implemented on top of the same old standard ANSI escape codes as primitives.
I’ve been waiting for one or more terminal emulators to get together and add some ridiculous new escape codes, such that animated effects (or things like collapsible sections, font size, “form-input-ness” and “form-button-ness” ala 3270, etc) all get treated as part of the state of a given character cell / run of character cells.
Heck, it’s 2024; an application should be able to use an escape code to set a soft-word-wrap mode for longer-than-viewport-width text, and have the TTY render text accordingly, rather than everything being hard-wrapped (baked mode) or not wrapped at all (raw mode) and only being able to get soft-wrapping using a pager!
(There’s precisely one thing like this I’m aware of terminal emulators adding in recent memory: clickable links, ca. six years ago.)
> (There’s precisely one thing like this I’m aware of terminal emulators adding in recent memory: clickable links, ca. six years ago.)
And they are often disabled by default, as a potential security risk. We don't get to have fun things, do we? (also worth read: CVE-2003-0063, abusing escape seq is unfortunately a valid concern against adding more stuff).
On the other hand, more and more emulators are adding support for various graphic protocols (sixel, iTerm2 format, kitty format).
> I’ve been waiting for one or more terminal emulators to get together and add some ridiculous new escape codes[...]
Well, it's not much, but mintty apparently has some interesting stuff like audio support[0], and codes for font size and font family[1][2].
iTerm2 also has a bunch of custom escape sequences of varying level of usefulness starting from displaying fireworks animation on cursor position to sending system notifications[3] (although sadly I could not get the last one to work for me).
For some semblance of forms, you can check bubbles[4] and gum[5] (binary to easily incorporate the components into shell scripts).
> And they are often disabled by default, as a potential security risk.
Well, that and a clickable link would conflict with mouse reporting if both were active at once.
Given that mouse reporting exists, and "has precedence" due to its age, I think iTerm's choice — to style the anchor-SGR text either way, but to only make them actually act like links rather than text if you hold a modifier key — is the only "correct" behavior for rendering anchor-SGR text, regardless of security concerns.
But that really wouldn't be true for most other potential novel graphical-rendition "styles." Anything that's non-interactive could certainly be on by default.
> [...] waiting for one or more terminal emulators to get together and add some ridiculous new escape codes [...]
I'm definitely of the opinion[0] that we haven't yet reached the limits of the "terminal emulator" UX paradigm.
The past few years do seem to have seen a resurgence in terminal emulator innovation due in part to a combination of new languages, the prevalence of GPUs, and a realisation that many of the existing terminal emulators weren't interested in any innovation in certain directions.
I've particularly been interested in the possibilities provided by the Terminal Graphics Protocol (which I discuss more in the linked comment).
A couple of years ago I switched to WezTerm[2] due to a combination of its graphics support, implementation language (Rust) and that its main developer seems to be interested in a combination of both solid support for existing standards & opportunities for innovation.
WezTerm also provides opportunities for customisation both in terms of shell integrations and of the application itself[3].
> [...] new escape codes [...]
Also, on this aspect, it may not even be necessary to create new escape codes--recently I discovered the `terminfo(5)` man page actually makes a pretty interesting read[7], in part because it lists some existing escape codes that seem like they have potential for re-use/re-implementation in the current day's more graphic-based systems.
---- footnotes ----
[0] As I mentioned in a recent comment on a thread[1] here:
"Motivated by the thought that at the current point in time perhaps the 'essence' of a 'terminal' is its linear 'chronological' presentation of input/interaction/output history rather than its use of 'text'."
[3] While I'm definitely not a fan of the choice of Lua as the extension language, I have now at least hit my head against the wall[4] with it enough that I can actually get more complex custom functionality working.
[4] I've started to write up some of my Lua-related[5] notes & more general WezTerm[6] notes so hopefully it'll eventually be an easier road for others. :)
[7] As one does. :) It was a fascinating/amusing time capsule in terms(!) of mentions of weird hardware terminal quirks that at one time ("before my time") needed to be worked around; interesting escape code discoveries; and, the mention of a term I had not thought of for decades but was at one time of importance: NLQ! :D
> [0] As I mentioned in a recent comment on a thread[1] here:
> "Motivated by the thought that at the current point in time perhaps the 'essence' of a 'terminal' is its linear 'chronological' presentation of input/interaction/output history rather than its use of 'text'."
I somewhat object to this.
If you think of a TTY/PTY as an Abstract Data Type, it's a very specific (and clever) one. It's not just a transcript of bytes that flowed through the wire.
Rather, a TTY/PTY is five distinct but inter-related things:
1. a (passive) character-cell viewport grid, of (at any given time) a fixed size, where each cell holds not only a character as data, but also a set of metadata/annotations attached to it (the particular metadata being dependent on the implementation)
2. an active "brush" state for the viewport grid — this includes things like a cursor position, an active foreground and background color, etc.
3. a byte pipe "but better" — in that it's actually a multi-subscriber byte message-queue, plus (only in baked mode) message persistence into an expandable ring buffer, with logical line-oriented consumer-group cursoring to trigger forward-truncation of said buffer
4. a rendering agent with a rendering ruleset (or, in old-school TTY terms, a "line discipline") that sits as one consumer of the byte pipe; reacts to characters in the byte pipe by writing them to the cursor position in the viewport grid; and reacts to in-band messages by manipulating the "brush", making big changes to the viewport grid (e.g. clearing it entirely), and/or swapping out the rendering ruleset itself
5. (the part everyone forgets about) a replay stream-conversion function, that can convert the current state of the viewport grid + its brush, into a stream of characters + in-band messages, such that the rendering agent would parse that stream back into the current viewport + brush state
The clever function you get from this set of components, can be seen in the case of attaching a "client" TTY/PTY to an existing persistent "server" TTY/PTY, such as is created by something like tmux or docker — or even by /bin/login on an old-school terminal server that you've dialed into.
When a client like telnet, ssh, docker-exec, tmux-attach, etc. attaches to an existing backing PTY (either locally or remotely), the client either establishes its own "replica PTY" or reuses the one it's running attached to, and then wants to establish replication from the backing PTY into this replica PTY.
To get a coherent replica, that doesn't just paint garbage at a random position with a random brush, several things need to happen:
• the client needs to reset its own local PTY a known-neutral brush state;
• the client needs to set an ACK position in the byte-stream, and a memorized brush state, for the remote PTY (usually both of these are done implicitly by the syscall on the remote end that opens the PTY for reading);
• the client needs to fetch from the backing PTY, an implementation-neutral byte stream representing the current contents of the character viewport grid of the PTY — and replay that (i.e. it needs to trigger the replay stream-conversion function) — this also happens implicitly, such that when the remote end open(2)s its its PTY, the current state is serialized and dumped into a buffer for read(2)s on the PTY to read;
• the client needs to set up its local terminal's brush to match the memorized one that was set as of its ACK position on the remote.
Once these four things happen, the client can in theory simply shunt the remote PTY's byte-pipe into the local PTY's byte-pipe, select(2)ing and write(2)ing in a loop on a thread; whereupon the local PTY's rendering agent will handle reducing that stream to the visual grid.
In practice, though, the rendering rules / line disciplines / syntaxes of local vs remote PTY aren't guaranteed to match (especially if the local PTY is actually a real physical TTY); and so in practice, clients doing PTY replication like this actually have their own translation function — logic that acts as a hybrid of a rendering agent and a replay stream-conversion function, taking the byte-stream from the remote, reducing it into a model of immediate state-change effects, and then re-serializing those immediate state-change effects as a byte-stream that the local PTY/TTY will understand. (With the first half of this usually being app-specific code, and the second half of this usually being done using some version of libcurses, to know exactly what rendering-rule syntax the local PTY/TTY claims to understand.)
---
Given all that, you could generalize a TTY/PTY to something other than "text". I've often myself considered a PTY where the "lines" are each (a binary, minimal encoding of) HTML body markup; or even where the "lines" are JSON-encoded log events with parent-node-IDs, and the "rendering" is of a default-collapsed hierarchical event history.
But you'd be losing a lot if you didn't bring across the concept of there being a message-stream + a "canvas" + a brush-state for that "canvas" + rules for two-way conversion between the message-stream and "canvas." Without the stateful canvas, you would need an unlimited-size transcript — and potentially hours of replay — to be able to "attach" a client to an existing backing PTY.
I’ve been waiting for one or more terminal emulators to get together and add some ridiculous new escape codes, such that animated effects (or things like collapsible sections, font size, “form-input-ness” and “form-button-ness” ala 3270, etc) all get treated as part of the state of a given character cell / run of character cells.
Heck, it’s 2024; an application should be able to use an escape code to set a soft-word-wrap mode for longer-than-viewport-width text, and have the TTY render text accordingly, rather than everything being hard-wrapped (baked mode) or not wrapped at all (raw mode) and only being able to get soft-wrapping using a pager!
(There’s precisely one thing like this I’m aware of terminal emulators adding in recent memory: clickable links, ca. six years ago.)