Feature request: uefi support

[joholl]

Problem

This crate is great! It would be even greater, if it supported uefi. Rust nightly contains much of std already, see this example in the official docs.

In the end, I would love to use ratatui on Linux/Windows/UEFI with crossterm as a backend.

Solution

What we already have:

• std already supports printing to stdout (e.g. via print!()), but there is also raw access to the UEFI protocol via EFI_SIMPLE_TEXT_OUTPUT_PROTOCOL.OutputString()
• the UEFI console supports a few control characters (LF, CR, BS, ...) and some ANSI escape codes
• the UEFI protocols provide an API for clearing the screen, querying for terminal size, setting the cursor position etc.

Open questions:

• I have not found features to enable alternate screen and raw mode (but I also lack deeper understanding of UEFI and what these modes do exactly)
• I have not had a closer look at input/events

I do not know if this feature request makes sense because I have not a full understanding of UEFI, crossterm, ratatui and terminals in general.

Development

If you quickly want to spin up a Rust application using stdout on UEFI (qemu):

sudo apt install ovmf  # for qemu uefi image
rustup target add x86_64-unknown-uefi  # install cross-compile toolchain

cargo init
cargo add r-efi
# copy this example into main.rs: https://github.com/rust-lang/rust/blob/master/src/doc/rustc/src/platform-support/unknown-uefi.md#example-hello-world-with-std
cargo build --target=x86_64-unknown-uefi
qemu-system-x86_64 -m 4G -cpu max -smp 4 -bios /usr/share/ovmf/OVMF.fd -nographic -no-reboot -nic none -drive file=fat:rw:./target/x86_64-unknown-uefi/debug,format=raw,media=disk
# type "<ESC>" to drop into shell
# type "fs0:" to change drive
# type "ls" to find your binary name: <...>.efi
# type "<...>.efi" to run application

PoC code for getting the terminal dimensions via query_mode:

let mut columns: usize = 0;
let mut rows: usize = 0;
let r = unsafe {
    let con_out: *mut simple_text_output::Protocol = (*system_table).con_out;
    let query_mode: extern "efiapi" fn(
        _: *mut simple_text_output::Protocol,
        usize,
        *mut usize,
        *mut usize,
    ) -> efi::Status = (*con_out).query_mode;

    let mode_number = 0;

    query_mode(
        con_out,
        mode_number,
        &mut columns as *mut _,
        &mut rows as *mut _,
    )
};
assert!(!r.is_error());

[jvantuyl]

@joholl Short Version: The request does make sense but probably only at a lower level. Most of what you seek would likely be best handled by building out terminal functionality in the UEFI code.

Long Version

Well, you asked for it. Specifically: "I have not a full understanding of UEFI, crossterm, ratatui and terminals in general." Allow me to fill in some of the blanks here. Apologies that this is so long, but terminals have quite a history and consider this to be a sort of crash course.

I hope to provide enough context to place all of this within the bigger world of terminals, etc. Once you understand why all of these features are here, a lot of what you need to do to get this working should crystalize.

What is a Terminal?

Terminals, in the Crossterm / Unix sense, are hardware devices or software programs that speak some various flavors of protocol to your OS. This is complicated somewhat by a few things like...

• most modern terminals all run in software
• but some older terminals are hardware devices
• OS consoles are a weird amalgam of the two, providing low-level input / display hardware support along with terminal handling
• in the interests of backwards compatibility, many features are shared (or almost shared) by the above

On top of that, there's often issues of how you're talking to the terminal:

• tty devices provide access to a serial terminal or console driver impersonating one that probably means more functionality to tell the OS what to do
• pty devices allow you to talk over the network or to software terminal emulators (which is even more complex)
• then you have Windows, which doesn't really provide exactly the same interface
• and your UEFI situation is even weirder (as I'll describe below)

Terminals (Like Ogres) Have Layers

All of this meshes together such that you can expect the following layers:

• application
• terminal
• operating system
• display / input

Application Level

At a high-level, most applications just want to print out text. Most everything you work with just knows to write strings to something (e.g. a file descriptor on Unix, a file handle on Windows, a protocol in UEFI). In the simplest model, those characters just get printed out.

This isn't really very remarkable, but it's important to understand it when you are trying to sort out what's going on. At this level, you're concerned about things like character encodings (ASCII, Unicode, EBCDIC, etc.) And often you're only concerned about a small subset of characters in those encodings (Latin-1, Klingon, etc.).

Most critically, this is where you'll send the control codes and sequences that other layers will intercept and interpret. Or, if things aren't right, they'll just display them wrong. Providing this layer is Crossterm's entire job—and to do that it (or your support code) needs to know a lot about those other layers.

As a matter of history, printing itself has some cursor positioning that varies a bit for certain control codes. So even though this level isn't concerned with all forms of cursor positioning, it implicitly does some.

For example, on old teletypes, backspace would move the carriage back one character. They actually supported underlines by printing the text, sending backspaces to back up to the beginning, and then printed underscores. They would also do bold the same way by backing up and printing it another time!

There is even some ambiguity at this level. You know how Windows uses two characters (carriage return + linefeed) for line endings and Unix uses just one (only a linefeed)? This is because Windows preserved the old teletype interpretation of carriage return just positioning back at the first column and linefeed advancing the paper. Unix just decided to use only the linefeed. When this is set wrong, you can see things like "stair-stepping" in text.

Terminal Level

When you read about ANSI codes, that is usually a terminal-level thing. Your application will need to know how to write these codes and the consuming software / hardware hopefully knows how to handle them. Alternate Screen-mode is at this level. Notably, raw mode is not. Clearing the screen and selecting colors is.

The general gist of this layer is that you're telling the remote end how to do something more than just spit out characters. This will be things like colors, cursor positioning, etc.

This may be implemented in hardware, in a(n) OS driver(s), or even an emulator program running on a GUI. Emulators will have codes for things like color palettes, handling cut-and-paste buffers, etc.

OS Level

When you start hearing about things like cooked-mode, raw-mode, echoing mode, and break (i.e. ^C) handling, that's OS-level. These are things that have less to do with the terminal you're talking to and more to do with what the OS tries to do for you. Remember the thing about different line ending codes in the Application Level? You can even ask the OS to translate this for you sometimes.

For example, the original terminals were actually teletypes. That is, they were like a type-writer that the computer could also print back to. The "line buffering mode" settings actually emulate this and it's sort of the natural behavior for most Unix tools. Line buffering plus application-level text writing / reading gets you most of what you experience at the shell prompt.

You'll also get stuff at this level like "process control". ^C, ^Z, and friends happen here.

Display / Input Level

Finally, lowest-level physical aspects of presenting your text and getting input come into play. When you're dealing with a console driver, the OS has some subsystem that knows how to draw the characters. Some systems have hardware do this for them, whereas others manage their own graphics hardware and draw the text themselves.

These systems might have escape codes for resizing the screen when it changes text modes or advanced codes for drawing graphics. They can even have codes for things like cursor shape / flash / flash rate or changing fonts.

If you're dealing with a serial terminal, you have entirely different physical concerns. Here you may actually have functionality to set the baud rate and character encoding. Some serial systems actually use control characters for flow control. Others have "clear-to-send" / "data-terminal-ready" (CTS/DTR) bits.

This may also be an entirely imaginary "physical" layer like you'll find in a Terminal Emulator program. Programs like xterm and konsole on Linux; iTerm or Terminal on Mac; whatever monstrosity Windows presents as a console; something even weirder under WSL; etc. Most of what they do will be at the "terminal level", but you'll still find things down here like window resizing, mouse events, and window titles.

Why So Many Layers?

Most all of the above exists because of the biggest annoyance in the field of computing—backwards compatibility. Some of this stuff has its root all the way back before WWI (in the 1920s and even a bit before)! So you're actually looking at about 100 years of backwards compatibility!

Each of the above are layers are the end-result of series of often disconnected design decisions. From the ENIAC all the way to your Macbook—and every IBM Mainframe in between—decisions were piled on top of each other. There's more history here than I can easily recount, but I've attached a mock history to the end of this document.

What's Your Stack?

The easiest way to understand how you need to work with the above layers is to define your target environment. In modern parlance, what is your "stack"? The following are all common "stacks" these days:

• Application -> OS tty -> OS Serial Driver -> Serial Modem -> (phone line) -> Serial Modem -> Hardware Terminal
• Application -> OS pty Driver -> SSH -> (OS TCP Stack) -> (network) -> (OS TCP Stack) -> SSH -> OS pty -> Terminal Emulator -> GUI
• Application -> OS tty Driver -> OS Console Driver -> Graphical Display Hardware
• Application -> OS pty Driver -> Terminal Emulator -> (GUI) -> (OS Graphics Driver) -> (Graphical Display Hardware)

In each of the above, every single layer (except a few marked in parenthesis) may interpret certain control codes or have external APIs to provide certain functionality. For example:

• An OS Serial Driver may have out-of-band hardware flow control which itself would be exposed via the OS driver.
• An actual Serial Modem may have in-band software flow control using control codes (e.g. XON / XOFF).
• The OS Serial Driver will likely have functionality to set serial parameters like baud rate, character size, stop bits, parity, etc.
• SSH has its own escape codes to handle when there are network issues or other protocol-level things.
• An OS pty or tty Driver may catch control characters (such as ^C and ^Z) to send signals to processes.
• An OS pty or tty Driver may do line-buffering other other input handling.
• A Terminal Emulator program may have its own codes to interact with its environment (i.e. a GUI)
• A Console Driver may have its own codes for controlling fonts, beep frequency / length, and display mode (i.e. how many columns / rows, color capability, etc.).
• An OS driver (or in some cases a library) may have an interface for echo control, secure attention, line buffering, etc.

Where is Crossterm In All This?

Most interestingly, Crossterm runs inside of the "Application" of these stacks. It's handling a dizzying array of control codes and features to handle everything you see there. And, in real programs, it often works in concert with other setup code to work with things like serial ports, ttys, and ptys.

It's honestly kind of crazy that Crossterm can find a subset of functionality that works across all of these things. In most use cases, it "stands on the shoulders of giants". The OS and emulators do most of the work for it. Your use case isn't much like that.

Make It All Make Sense!

Let's define your stack. You're probably going to be implementing one of these:

• Application -> UEFI Rust Libraries -> UEFI Text Protocol -> UEFI Implementation
• Application -> UEFI Rust Libraries -> UEFI Graphical Protocol -> UEFI Implementation

Notice what's missing above that was in every other stack? The OS. There's going to be no signal handling, buffering mode, "cooked" / "raw" handling, keyboard handling. For that matter, there's no Terminal Emulator or Hardware either. So that means no ANSI codes, limited control codes, etc.

There's no terminal emulator or OS at boot time. That's good in that you don't have so much baggage history or baggage to deal with. That said, Crossterm kind of expects some of it, so you'll have to figure out what to provide to Crossterm and what you might be able to workaround with a custom backend.

Considering the features / issues you've mentioned:

• Alternate Screen Mode: This is a function of the terminal (or emulator) at the Terminal Level. It's an escape code sent to a (potentially remote) terminal that's an escape code and isn't always supported.

• Raw Mode: This is an OS Level feature. It's more about how to present terminal interaction to your application and is there to help support different ways of talking to terminals.

• Rust's stdout on UEFI: This talks to the OS when you're in Linux or Windows; but there is no OS at boot time. These libraries are either providing their own functionality or wrapping the UEFI Implementation here.

• UEFI Protocols: This is very much Input / Display Level. What you're going to be dealing with is less "terminal codes" and more "interacting with a low-level display driver" like you would interacting directly with an OS Console Driver. You'll need to implement some stripped-down version of a OS Level functionality and most of the Terminal functionality.

• ANSI Codes: These are not necessarily available on every terminal. More importantly, UEFI does not provide this. The codes you found in the reference are present as an example only. You will find various (most?) UEFI implementations don't implement them—at all.

What Needs To Be Done, Then?

1. Raw Mode. You've already got something like raw mode. In fact, you can't turn it off and you're going to have to build functionality onto what you've got to even get to the point that there's a facsimile of "raw mode" that Crossterm could use. This is very likely better done in the Rust libraries than in a Crossterm backend, but you could do it if you wanted to.

2. Alternate Screen. This is a terminal concept, so you don't have it. If you want it, you'll have to implement it. That said, you're not cooperating with any other programs really, so it's not so needed.

3. EFI_SIMPLE_TEXT_OUTPUT_PROTOCOL: Some UEFI systems may not provide this. Oddly, it is character oriented and specifies Unicode. This is almost unique amongst console requirements (though terminals often deal with it). Implementing this will basically be implementing something like an OS console driver, so you're actually writing most of the terminal layer yourself.

4. EFI_SIMPLE_TEXT_INPUT_PROTOCOL: You'll want the extended protocol. You need it to get the shift-key statuses and to control various things.

5. EFI_SIMPLE_TEXT_INPUT_EX_PROTOCOL: This also may or may not be supported. I suspect it'll be supported most places you have the basic one. Again, this is low-level and you'll have to build it up to implement something like terminal input.

So... yeah. There's not a lot to be done here in Crossterm. You really need to build up the Rust UEFI support to provide:

• Basic Terminal Input
• Basic Terminal Display
• Emulating Raw Mode

To that end, a new Crossterm backend may help take out some of the other assumptions that most terminals meet but would be hard to implement. But you'll probably do better to keep as much as possible in the UEFI code. It would certainly be more widely useful.

I hope that helps. And, if you do undertake this work, good luck.

Appendix A: A (Not So) Brief (or Entirely Accurate) History of Text Processing Equipment

In the beginning, $DIETY created people. And he saw that the people wanted to communicate much further than they could hear each other. And he created teletypes and it was good.

And then he created computers. And he looked down and saw that computers had no mouth and could not speak, so he adapted teletypes into teleprinters. And he saw that computers could only speak in bits and that users could only speak in glyphs. And he said, "Let there be character encodings!" And there were character encodings. And it was good.

And he saw that the characters were without formatting and the teletypes couldn't really print much more than a line of text. And he said, "Let there be control codes!" And control characters such as "Carriage Return" and "Line Feed" and "Tab" and "Form Feed" came into being. And they were simple characters, and now the teleprinters could print about freely. And they were happy for a time.

And in these early times, IBM had to "Think Different" and they create some diabolical alternate encodings that they forced upon their customers. And these encodings were seen to be proprietary and unwieldy. In his anger, he consigned them to Corporate Hell, where they toil to this day, underpinning the government, financial, medical, and insurance systems.

And the computers were confused, for the humans in their haste had made many typos because they couldn't see what they were typing. And they took pity on the humans and echoed back the characters they were typing. And thus echo mode was born.

And it came to pass that humans humans realized that they had things to hide; and in their shame they created passwords. And computers gifted them with the ability to turn off echo mode, so their passwords weren't displayed.

The computers tired of handling each character on their own, and they cried out to their creator, "Can we just get a whole line?" And the teletypes were taught to buffer, and line mode came to be.

Soon the people had created great industries. They consumed entire forests, creating tractor-feed paper to feed to the hungry printers and teletypes. They looked upon the barren landscapes that had once been verdant jungles and despaired. They cried out, "We haven't invented recycling yet. What do we do with all of this paper?"

And they made monitors and keyboards. They integrated them together, creating terminals. And they replaced the teleprinters with them. As punishment, the teleprinters were stripped of their keyboards and banished to the land of Hewlett-Packard, where they would have buggy drivers until the end of days.

The terminals were taught to speak the same control codes that the printers had. And they had more added... so many more. But eventually, they exhausted their supply of bits, and could mint no more new characters.

And they saw that it was wasteful, as some characters signified inputs that had no glyph to display (i.e. ESC) and others signified outputs that would never be typed (i.e. BEL). And they were trapped, for they had no more bits to use.

They cried out, "Let us escape from these bonds. Let us use multiple characters to signal new inputs and outputs." And escape they did, by creating escape codes. They could now print out the Escape character and define sequences of characters to control new functionality.

And the monitors and the keyboards and the computers and the people were happy for a time. Even the printers rejoiced and hummed as they munched page after page of paper. And their ink flowed like wine, even when you weren't printing. And Hewlett-Packard became rich.

But in time, the people became divided. And in this division they created new, incompatible, competing standards and codes. Some tribes of people strayed far from their computers as they had with the teletypes of old. They invented modems to sit in between the computer serial port and the terminal.

Some left out the terminal altogether, integrating keyboards and monitors directly with computers. They retained vestigial aspects of the terminals that handled by video cards and the keyboard interface. Soon these devolved until some computers didn't even have a native text mode and drew their own characters.

And $DIETY looked down on this and turned away in disgust. And he declared that the world of terminals was outside of his sight—and thus outside of his love. It came to pass that the fractious groups of humans had to fight among themselves to make this all work.

And fight they did. Some created great families of terminals, each with their own dialect and capabilities. Some tried to unify the printers with the terminals again and created escape codes for remote printing.

And computers tried their best to make use of all of this, providing APIs to control how they interfaces with serial ports and dialects of terminal and their own consoles. They even created interfaces to allow programs to talk to other programs, creating the ptys used by software terminals and things like SSH.

In doing this, they also allowed entirely software terminals. Seeing the freedom from hardware limitations, they said "We will create a standard to replace these other N standards." And thus it came to pass that there were N+1 standards.

Humans, in their desire to "just get work done" created vast libraries of data and code to try to sort all of this out. There were descriptions of terminal capabilities / codes like termcap and terminfo. And there were libraries to try drawing on various terminals using this data such as curses, and ncurses. There was even code written to render text directly on graphical framebuffers.

And in the ensuing years, there were oft recreated with specific focuses and varying qualities. Which leads us to the mess you're going to have to sort out to make all of this work.

Appendix B: Some Useful Links

You read all the way down here, eh? Here are some tasty links for your trouble.

• The Linux vt (Virtual Terminal) control code docs
• The Linux utility for changing serial settings.
• A discussion of the differences between TTY and PTY devices
• A decent walkthrough of Linux terminals. (part 1) (part 2)
• A History of Some ASCII Codes
• Everything you never wanted to know about ANSI escape codes
• Some discussion of the alternate screen.
• Some discussion of what happens when line endings aren't translated correctly.
• Info on the cooked and raw modes (and a secret third mode!)
• An illustrated history of the terminal

[joshka]

Get this man a blog!!! :D (serious suggestion as this is a great write up and deserves to have a URL where people can point at permanently)

Solving the underlying problem Ratatui apps in UEFI, I'd suggest solving this by sidestepping crossterm entirely.

• For the display side, write a Ratatui backend that sits on top of the UEFI output code https://docs.rs/uefi/latest/uefi/proto/console/text/struct.Output.html.
• For the input side, either use the input struct directly https://docs.rs/uefi/latest/uefi/proto/console/text/struct.Input.html or write a small wrapper that sits in front of it and crossterm. Choose the second approach if you want to be able to test your app outside of a UEFI context.
• Ratatui likely has a bunch of code which requires std, so you're probably going to spend a lot of time there working around that (guessing here - but factor that into your difficulty estimate)
• let us know how you went by shooting a link to your app on the ratatui discord or forums.

[jvantuyl]

@joshka Thanks for the kind words. I don't really maintain my blog anymore. I tend to put stuff like this in random GitHub comments because I have an inkling that they'll exist long after my blog will be gone. It seems like as good a place as any.

I've added some resources and clarifications to the post. I hope it's useful to people who stumble across it in search engines and whatnot. I'm sure it'll be trained into some AI and memorialized forever... so I guess I've got that going for me.

@joholl The above suggestion is a good one. You could skip Crossterm entirely and go straight to a ratatui backend. The only benefit of implementing the terminal at the lower level is that it opens it up to things other than just ratatui apps.

That said, it's way more work to do that; so nobody (who matters) would blame you for going the expedient route.

[joholl]

Oh wow... Thank you @jvantuyl for the very comprehensive (and no less entertaining) answer!

@jvantuyl @joshka To be honest, I already kinda tried making ratatui crossterm work compile on UEFI. Well, it made me question if I really need want interactive user input enough to try making crossterm (or any other piece of software) become the necessary abstraction layer. Welp, I stopped before reaching anything worth sharing.

More importantly, though, I think I'd rather have a non-interactive tool that works on most UEFI systems than an interactive tool that works on some of them, not quite on some others and not at all on the rest. It is hard for me to judge how fragmented the UEFI landscape is, right now.

Adding UEFI support to crossterm might still be a worthwhile endeavor, perhaps. Feel free to close this issue if you feel like it is sufficiently unlikely (or unworthy of tracking).