Zed Decoded: Linux when?
May 7th, 2024
Take a look at this:
Does anything stick out? Yes, exactly, it's a screenshot of Zed running on Linux!
Wait, what? Zed on Linux? Is it released yet? No, it's not, but it's taking shape, fast.
At the end of January we open-sourced Zed and had zero Linux support. Now, three months later, you can compile & run Zed on Linux and actually use it. And I mean really use it — I've worked in Zed (on Zed!) the whole last week without any problems.
The words "alpha release" seem to appear at the end of the tunnel. We're getting closer.
Today we're going to talk about how Zed on Linux took shape, what the challenges were, who did the work, and what's still left to do.
Companion Video: Linux when?
This post comes with a 1hr companion video, in which Thorsten and Mikayla explore Zed on Linux, dig through the codebase to see how it's implemented, talk about implementation challenges, and how the open-source community helped out big time.
Watch the video here: https://youtu.be/O5XVVnA2LoY
Why not Linux from the start? Or: the Tricky Thing About Platforms
Why didn't Zed work on Linux out of the box? That's not an unreasonable thing to ask and many of you did. After all, Zed's written in Rust, a language that's known for its cross-platform support. So if Rust programs can run on macOS, Linux, and Windows, why didn't Zed?
The tricky thing about cross-platform support is that — in general and in Rust — it only works as long as you're fine with the platform being abstracted away, hidden behind an API that is the same on every platform.
At Zed, though, we want to use each platform as best as we can to build a high-performance application that is and feels native to the platform. That often means talking directly to the platform, in order to use it to the best of its abilities.
On macOS, for example, Zed makes direct use of Metal. We have our own shaders, our own renderer, and we put a lot of effort into understanding macOS APIs to get to 120FPS. Zed on macOS is also a fully-native AppKit NSApplication and we integrated our async Rust runtime with macOS' native application runtime.
If you want your application to have this level of depth and control over its platform integration and have it be cross-platform, what you'll need to build is a framework. A framework that allows you to talk directly to the platform whenever you need, but otherwise abstract it away from you so you don't have to worry about it when you write application-level code.
That's what Zed did. The framework is called GPUI and when we released it as open-source, along with Zed, it came with cross-platform support. Except not really.
GPUI's cross platform support
GPUI does abstract away the underlying platform and assumes there's more in the world than macOS. Here are parts of GPUI's Platform trait, which is all a GPUI application has to interact with:
// crates/gpui/src/platform.rs
trait Platform: 'static {
// [... some methods left out to keep this short ...]
fn background_executor(&self) -> BackgroundExecutor;
fn foreground_executor(&self) -> ForegroundExecutor;
fn text_system(&self) -> Arc<dyn PlatformTextSystem>;
fn run(&self, on_finish_launching: Box<dyn 'static + FnOnce()>);
fn quit(&self);
fn restart(&self);
fn hide(&self);
fn displays(&self) -> Vec<Rc<dyn PlatformDisplay>>;
fn open_window(
&self,
handle: AnyWindowHandle,
options: WindowParams,
) -> Box<dyn PlatformWindow>;
fn window_appearance(&self) -> WindowAppearance;
fn open_url(&self, url: &str);
fn on_open_urls(&self, callback: Box<dyn FnMut(Vec<String>)>);
fn register_url_scheme(&self, url: &str) -> Task<Result<()>>;
fn prompt_for_paths(&self, options: PathPromptOptions, ) -> oneshot::Receiver<Option<Vec<PathBuf>>>;
fn prompt_for_new_path(&self, directory: &Path) -> oneshot::Receiver<Option<PathBuf>>;
fn reveal_path(&self, path: &Path);
// [...]
fn local_timezone(&self) -> UtcOffset;
fn set_cursor_style(&self, style: CursorStyle);
fn write_to_clipboard(&self, item: ClipboardItem);
fn read_from_clipboard(&self) -> Option<ClipboardItem>;
fn write_credentials(&self, url: &str, username: &str, password: &[u8]) -> Task<Result<()>>;
fn read_credentials(&self, url: &str) -> Task<Result<Option<(String, Vec<u8>)>>>;
fn delete_credentials(&self, url: &str) -> Task<Result<()>>;
}This Platform offers nearly everything an application might want to do: execute work on different threads, start/stop the application, manage windows, open URLs, open system dialogues to open files and directories, cursor style, clipboard, credentials, ... and a few more things that I left out to keep this succinct.
From the start, GPUI has had this Platform abstraction built-in. GPUI never was macOS-only. It was always platform-agnostic, as long as the Platform trait is implemented for the platform.
And that's essentially what it meant to get Zed running on Linux: implement Platform for Linux.
So, just implemented a bunch of methods and then Zed works on any platform? Well... yes, kind of.
Here's what the Linux Platform implementation looked like in the middle of February. Again, in excerpts:
// crates/gpui/src/platform/linux/platform.rs @ 266988adea
impl Platform for LinuxPlatform {
// [... some methods left out to keep this short ...]
fn background_executor(&self) -> BackgroundExecutor {
self.inner.background_executor.clone()
}
fn foreground_executor(&self) -> ForegroundExecutor {
self.inner.foreground_executor.clone()
}
fn text_system(&self) -> Arc<dyn PlatformTextSystem> {
self.inner.text_system.clone()
}
fn run(&self, on_finish_launching: Box<dyn FnOnce()>) {
self.client.run(on_finish_launching)
}
fn quit(&self) {
self.inner.state.lock().quit_requested = true;
}
//todo!(linux)
fn restart(&self) {}
//todo!(linux)
fn hide(&self) {}
//todo!(linux)
fn unhide_other_apps(&self) {}
fn prompt_for_new_path(&self, directory: &Path) -> oneshot::Receiver<Option<PathBuf>> {
unimplemented!()
}
fn reveal_path(&self, path: &Path) {
unimplemented!()
}
//todo!(linux)
fn write_to_clipboard(&self, item: ClipboardItem) {}
//todo!(linux)
fn read_from_clipboard(&self) -> Option<ClipboardItem> { None }
}Some methods implemented, some marked with todo!, others panicking with an unimplemented!(). Back in February this file had 11 todo! comments and 9 calls to unimplemented.
Now it has 9 todo!s (different ones) and the word unimplemented in a string.
So, yes, the work was getting rid of the todo!s and implementing the missing methods. But we all know: not all methods are created equal.
While a write_to_clipboard method seems relatively straightforward to implement, things weren't as easy as the lower-case todo! and its defiantly cute exclamation mark might suggest.
Which Linux are you talking about?
A big challenge when building a GUI application for Linux is that there is no such thing as Linux, really. Linux is a kernel and when you install and run it, you're mostly likely doing that through a Linux distribution that gives you the rest of the operating system too: Ubuntu, Debian, CentOS, Arch, Gentoo, and so on.
They nearly all differ in some aspects that are relevant for an application developer. Example: package management. Distributions not only have their own format with which to distribute applications, but they also have different ways of managing dependencies. And there's no standard across distributions (but of course there are competing standards). So far, with Zed, we've avoided the whole packaging topic and just focused on building .tar.gz archives that others can then turn into distribution-specific packages. But just scrolling through Tailscale's Packages site tells us that there's a lot of work to do that we never had to do for macOS.
But even if you focus on only one distribution, say Ubuntu, you still have to decide: do you support X11 or Wayland? Both are — ignoring some technical details (don't send me angry letters) — display servers with which Linux software can draw things on the screen. X11 has been around for a long, long time and Wayland has been trying to replace it, also for a long, long time. That, in turn, makes the question a trick question: you have to support both. They're both widely used. Still used in the case of X11 and gaining users in the case of Wayland, but it's not realistic that 100% of Linux users will be on Wayland anytime soon.
After deciding to support Ubuntu and X11 and Wayland, the next question is: which desktop environment or window manager are you going to support? KDE or GNOME? Qt or GTK? What about tiling window managers and users that want to turn window decorations off? Which audio server are you going to support to get audio calls working? PipeWire? Or still PulseAudio? Do you even have to worry about that or is that choice dictated by the desktop environment you chose?
I could probably go on and find more "actually, there's more than one of X" to list, but you get the point: when building a graphical application to run on Linux, you have to make quite a few technical decisions about which platform combinations to target and how.
And we haven't even touched on the most fundamental of all of these decisions yet: how do we render Zed?
From Metal to... what?
Even if you knew how to package your application, and how to target X11 and/or Wayland, and managed to open an application window in them — how do you draw your application in that window?
You might have missed it, but in the Platform trait above, there's a method to open a new window:
// crates/gpui/src/platform.rs
trait Platform: 'static {
// [...]
fn open_window(
&self,
handle: AnyWindowHandle,
options: WindowParams,
) -> Box<dyn PlatformWindow>;
}In order to implement that, you also have to implement the interface of the thing it returns: PlatformWindow. And that packs a punch.
As of right now, PlatformWindow requires 37 methods to be implemented. The methods range from hooks (on_close, on_fullscreen) to attributes (is_fullscreen, is_minimized) to bread-and-butter (content_size, mouse_position, toggle_fullscreen) to tricky (scale_factor, appearance).
But there's one method on PlatformWindow that's the most fundamental of them all: draw. Another name for it could be where_the_rubber_hits_the_road. Here is its implementation for macOS:
// crates/gpui/src/platform/mac/window.rs
impl PlatformWindow for MacWindow {
// [...]
fn draw(&self, scene: &Scene) {
let mut this = self.0.lock();
this.renderer.draw(scene);
}
}It doesn't look imposing, but it is: Scene is the result of GPUI rendering a frame and this method, draw, is what makes it appear on the screen.
A Scene is the visible parts of the application — text, windows, borders, rectangles, underlines, and so on — turned into data, primitives. Here's the definition of Scene:
// crates/gpui/src/scene.rs, simplified
struct Scene {
shadows: Vec<Shadow>,
quads: Vec<Quad>,
paths: Vec<Path<ScaledPixels>>,
underlines: Vec<Underline>,
monochrome_sprites: Vec<MonochromeSprite>,
polychrome_sprites: Vec<PolychromeSprite>,
surfaces: Vec<Surface>,
paint_operations: Vec<PaintOperation>,
primitive_bounds: BoundsTree<ScaledPixels>,
layer_stack: Vec<DrawOrder>,
}Shadows, quads, paths, underlines, sprites — that's what GPUI boils your application down to when a frame is rendered. And then, in draw, it hands that data over to the renderer.
And the renderer — that's your GPU.
As I mentioned above: on macOS we use the Metal APIs to talk to the GPU and when you follow the definitions there, you'll end up at a fn draw, implemented on MetalRenderer, that contains nearly the complete saga as told in our 120FPS blog post: setting up a rendering pipeline, triple-buffering, syncing up with the OS and the display, all of that.
So how do we implement draw on Linux, without Metal? How do we talk to the GPU on Linux?
Enter stage: the open-source hackers
Less than two weeks after Zed going open-source, flying through the cloud of pull requests and issues that's been stirred up by all the excitement comes @kvark (Dzmitry Malyshau) with a by now legendary PR that makes Zed render on Linux. In less than four thousand lines of code. Sure, it didn't render all of Zed yet, but: wow.
The PR answers the question of how to talk to the GPU on Linux: with blade, kvark's "rendering solution for Rust", that offers a "lean low-level GPU abstraction focused at ergonomics and fun". Blade uses Vulkan under the hood, a graphics API that's similar to Metal in the level of abstraction it offers, to talk to the GPU and render Zed.
After @kvark opened the PR, Mikayla, Antonio, and Nathan reached out to him to talk about the feasibility of using Blade in Zed and shared goals. It didn't take long to get to a shared understanding and, exactly two weeks after open-sourcing Zed, the PR that made it compile and run on Linux was merged.
With that big TODO and question out of the way, the community jumped on the chance to remove the remaining todo!s: @witelokk added support for Wayland, @kvark continued his work on getting all of Zed to render smoothly, @romgrk added support for file dialogues, @apricobucket28 fixed scrolling and selections and many others contributed and fixed a lot of other things, with Mikayla diligently reviewing, coding, managing, leading and keeping an eye on everything.
This list of contributions is incomplete and I'm sure I should've mentioned more people, but the point I want to make is this: Zed on Linux was and is an impressive open-source team effort that surprised all of us at Zed with how fast it got Zed to a working state.
Zed on Linux: the abstractions
Let's take a look at how it works, how X11 and Wayland and Blade are tucked away under the Platform abstraction.
Here's an excerpt:
// crates/gpui/src/platform/linux/platform.rs
impl<P: LinuxClient + 'static> Platform for P {
fn background_executor(&self) -> BackgroundExecutor {
self.with_common(|common| common.background_executor.clone())
}
fn foreground_executor(&self) -> ForegroundExecutor {
self.with_common(|common| common.foreground_executor.clone())
}
fn text_system(&self) -> Arc<dyn PlatformTextSystem> {
self.with_common(|common| common.text_system.clone())
}
fn run(&self, on_finish_launching: Box<dyn FnOnce()>) {
on_finish_launching();
LinuxClient::run(self);
self.with_common(|common| {
if let Some(mut fun) = common.callbacks.quit.take() {
fun();
}
});
}
fn quit(&self) {
self.with_common(|common| common.signal.stop());
}
// [...]
}Platform is implemented on LinuxClient, which itself is a trait too:
// crates/gpui/src/platform/linux/platform.rs
trait LinuxClient {
fn with_common<R>(&self, f: impl FnOnce(&mut LinuxCommon) -> R) -> R;
fn displays(&self) -> Vec<Rc<dyn PlatformDisplay>>;
fn primary_display(&self) -> Option<Rc<dyn PlatformDisplay>>;
fn display(&self, id: DisplayId) -> Option<Rc<dyn PlatformDisplay>>;
fn open_window(
&self,
handle: AnyWindowHandle,
options: WindowParams,
) -> Box<dyn PlatformWindow>;
fn set_cursor_style(&self, style: CursorStyle);
fn write_to_primary(&self, item: ClipboardItem);
fn write_to_clipboard(&self, item: ClipboardItem);
fn read_from_primary(&self) -> Option<ClipboardItem>;
fn read_from_clipboard(&self) -> Option<ClipboardItem>;
fn run(&self);
}And there are two implementations of LinuxClient:
Both of them have an implementation of PlatformWindow too:
But both of them use Blade in their draw method:
// crates/gpui/src/platform/linux/x11/window.rs
struct X11WindowState {
// [... other fields ...]
renderer: BladeRenderer
}
impl PlatformWindow for X11Window {
// [...]
fn draw(&self, scene: &Scene) {
let mut inner = self.0.state.borrow_mut();
inner.renderer.draw(scene);
}
}
// crates/gpui/src/platform/linux/wayland/window.rs
struct WaylandWindowState {
// [... other fields ...]
renderer: BladeRenderer,
}
impl PlatformWindow for WaylandWindow {
fn draw(&self, scene: &Scene) {
let mut state = self.borrow_mut();
state.renderer.draw(scene);
}
}That means the rubber hits the road through these layers of abstractions:
- GPUI asks the
Platformto open a window - The Wayland
LinuxClientuses Wayland to open a window, the X11 client uses X11 - When rendering a frame, GPUI turns the application into a
Scene - It then calls the
PlatformWindowtodrawthatScene - Both the Wayland and the X11 implementations use Blade, which uses Vulkan, to talk to the GPU, to draw the
Scene
That's obviously only scratching the surface. There are a lot more interesting things going in the Linux implementation — did you know that there's multiple clipboards on Linux but only one on macOS? Or here, take a look at this:
Yup, that's a native file dialog, but we don't use GTK in Zed and that clearly is GTK — so, how does that work? You'll find the answer in the companion video in which Mikayla and I dive deeper into this and also touch on the question of whether or not we should use GTK or Qt or something else (spoiler: it's complicated).
So, Linux when?
So what's left to do for Linux? In order to get to an alpha release: not that much, but don't quote me on that. Essentially: fix 86 remaining todo!s of various difficulty levels, get window resizing/moving to work on Wayland, and implement system dialogues for GPUI. We're close, very close.
After the alpha, we'll need to add support for audio calls, drag & drop, storing of credentials, make sure the performance is consistently high, increase stability, and so on. Take a look at the Linux Roadmap tracking issue.
Even though there might still be a lot to do (I don't even want to know how complicated drag & drop can be on Linux— I mean, GNOM— I mean, KDE, no, I mea—) and a lot of unknown unknowns and surprises along the way, one thing's for sure: the fact that we got to the current state of Zed on Linux in three months, with that many high-quality open-source contributions, is pretty amazing.
Want to try out Zed on Linux? You need Rust, some dependencies, and depending on your patience enough CPU and memory to compile it in a reasonable amount of time. Take a look at these instructions. Have fun!