Earlier this year, extensions landed in Zed, adding the ability to add more languages, themes, snippets, and slash commands to Zed.
The extension system was built by Max and Marshall and in April, Max wrote an excellent blog post about some of the intricate engineering challenges behind extensions and how they overcame them. It also explains that extensions are run as WebAssembly (Wasm) modules and why and how Tree-sitter is used within Wasm.
Half a year later, that's still all I really know about how our extensions work — not a lot, considering how much more there is to know. I know, for example, that extensions are written in Rust, but how are they compiled and when? I know extensions can provide language servers to Zed, but how exactly does that work? And how exactly do we run them as Wasm modules inside Zed? How does it all fit together? I need to know!
Two weeks ago, I finally had the chance to sit down with Marshall and Max and ask them everything I wanted to know about extensions. To kick us down the rabbit hole, I asked them: what exactly happens when I install an extension in Zed?
Companion Video: Life of a Zed Extension: Rust, WIT, Wasm
This post comes with a 1hr companion video, in which Marshall, Max, and Thorsten explore extensions in Zed, digging through the codebase to see how Rust, Wasm, WIT and extensions fit together and end up running in Zed.
Watch the video here: https://youtu.be/Ft58q9E0G5Y
The Question
The extension I used as an example to ask "what happens when I install this?" was the zed-metals extension. It adds support for Scala to Zed by adding the Scala Tree-sitter parser, Tree-sitter queries, and — the interesting bit for me in that conversation — it also adds support for the Metals language server.
And it's written in Rust! Yes, here, take a look:
There's not a lot more in that repository. There's an extension.toml file with
metadata about the extension, the Tree-sitter queries, and this single Rust
file.
So the question I wanted to answer was this one: when I install this extension,
when and how and where is the Rust code in that lib.rs file compiled and how
does it end up being executed in Zed when I use the zed-metals extension?
In our conversation Max and Marshall patiently walked me through the all of the code involved so that I can now report: we figured it out!
Installing Extensions
When you open Zed, hit cmd-p/ctrl-p and type in zed: extensions. You'll
see this:
This extensions view shows all extensions available in Zed and which ones you have installed.
So, first things first: where does that list of extensions come from? Its ur-origin is this repository: github.com/zed-industries/extensions.
This repository is the source of truth for which extensions exist in Zed. It contains references to all other repositories of Zed extensions.
(Did you just have a visceral reaction and thought "yuck! the extension registry is a repository?". Hey, we're with you! When I asked Max and Marshall what they think will have to change about the extension system in the future, Max said that this repository will likely have to go. It doesn't scale, but it worked very well so far.)
The repository and the list of extensions it contains is mirrored regularly to
zed.dev, on which we run our Zed API. I'm using "mirrored"
loosely here: not the actual contents git repository is mirrored, only its
contents (you'll see). And when you run zed: extensions, your Zed sends a
request to zed.dev's API and ask it for the list of extensions.
So then what happens when you decide to install an extensions by clicking on
the Install button?
It first has to be downloaded, of course. But the question is: what is being downloaded? The Rust code? Do you download and compile Rust code?
Turns out, no, you don't. And this is where things become fun.
From the extensions repository to your Zed
The extensions repository contains a CI step that ends up executing
the "extensions CLI", a small CLI program that lives in the Zed repository in
the extensions_cli crate.
It's made up of a a single file and its main job is to accept a directory containing a Zed extension and compile it.
Lucky for us explorers, we don't need to use a CI system to that. We can run the
binary manually on our machine. Here's what that looks like when I use it (the
binary is called zed-extension here) to compile metals-zed:
$ mkdir output
$ zed-extension --source-dir ./metals-zed --output ./output --scratch-dir $(mktemp -d)
info: downloading component 'rust-std' for 'wasm32-wasip1'
info: installing component 'rust-std' for 'wasm32-wasip1'
$ ls -1 ./output
archive.tar.gz
manifest.jsonThat produced two files: archive.tar.gz and manifest.json.
The manifest.json contains the metadata you see in the zed: extension view
inside Zed:
$ cat ./output/manifest.json | jq .
{
"name": "Scala",
"version": "0.1.0",
"description": "Scala support.",
"authors": [
"Igal Tabachnik <[email protected]>",
"Jamie Thompson <[email protected]>",
],
"repository": "https://github.com/scalameta/metals-zed",
"schema_version": 1,
"wasm_api_version": "0.0.6"
}It's generated from the extension.toml in the repository.
So what's in archive.tar.gz?
$ tar -ztf ./output/archive.tar.gz
./
./extension.toml
./extension.wasm
./languages/
./grammars/
./grammars/scala.wasm
./languages/scala/
./languages/scala/outline.scm
./languages/scala/indents.scm
./languages/scala/highlights.scm
./languages/scala/config.toml
./languages/scala/overrides.scm
./languages/scala/injections.scm
./languages/scala/runnables.scm
./languages/scala/brackets.scmThere's an extension.toml file that's nearly the same as the one in the
extension repository, but contains some more metadata added at
compile-time, including the version of the Zed extension Rust API the extension
was compiled against — keep that in the back of your head, we'll come back to
it.
The extension.wasm file is the lib.rs file we saw earlier, the
Rust code, compiled into Wasm.
The grammars/scala.wasm is the Tree-sitter grammars compiled into Wasm.
(Max's blog post explains how Tree-sitter and Wasm are compiled
here.)
And then there's a bunch of Scheme files — outline.scm, highlights.scm,
... — that contain Tree-sitter queries which Zed executes at runtime to, for
example, get syntax highlighting for a Scala file.
So, what we know so far: an extension is compiled with a small CLI tool and that
compilation results in two files. A manifest.json with metadata and an archive
that contains two Wasm files and a bunch of small Scheme files.
In CI, the next thing that would happen after compiling the extension, is that
both files are uploaded to a place that's reachable from the zed.dev API. The
code for that lives in the zed-industries/extensions repository, as some
neatly-written JavaScript code in package-extensions.js.
The code goes through all the extensions in the repository, compiles them with
the extensions_cli from the Zed repository
(just like I showed you), and uploads the resulting archive.tar.gz and
manifest.json files to an S3 bucket.
And that bucket — not the zed-industries/extensions repository — is what
gets mirrored by zed.dev and made accessible through its API. Every few minutes,
zed.dev fetches the manifest.json files from the S3 bucket and stores their
contents in a database, along with the URL of the archive.tar.gz files.
We have our first answer: when you install an extension, you don't compile Rust code. You download and unpack an archive that contains Wasm and Scheme code.
But that's skipping over quite a few things. I mean, what exactly happens when an extension is compiled from Rust to Wasm?
Compiling an extension
Let's take a look at a very simple Zed extension written in Rust:
use zed_extension_api::{self as zed, Result};
struct MyExtension;
impl MyExtension {
const SERVER_BINARY_NAME: &'static str = "my-language-server";
}
impl zed::Extension for MyExtension {
fn new() -> Self {
Self
}
fn language_server_command(
&mut self,
_language_server_id: &zed::LanguageServerId,
worktree: &zed::Worktree,
) -> Result<zed::Command> {
let path = worktree
.which(Self::SERVER_BINARY_NAME)
.ok_or_else(|| format!("Could not find {} binary", Self::SERVER_BINARY_NAME))?;
Ok(zed::Command {
command: path,
args: vec!["--use-printf-debugging".to_string()],
env: worktree.shell_env(),
})
}
}
zed::register_extension!(MyExtension);This is a fictional extension I just threw together. All it does is to define
how to run the also fictional my-language-server binary: it looks up the location
of the binary in the $PATH of Zed worktree that's open and returns some
arguments with which to run it.
But MyExtension is an empty struct. It doesn't even have any fields. What
makes it a Zed extension is the fact that it implements the zed::Extension
trait — where does that come from?
It's a dependency added in the Cargo.toml of my fictional extension, down
there, on the last line:
[package]
name = "test-extension"
version = "0.1.0"
edition = "2021"
[lib]
crate-type = ["cdylib"]
[dependencies]
zed_extension_api = "0.0.6"Then next question then is: what is in that zed_extension_api crate? It can't
just be Rust code, right? Because after all, we want the extension to compile
down to and run as Wasm.
And that's where things become really interesting!
The extension_api crate
The zed::Extension trait is defined in the extension_api
crate in the Zed repository, in the extension_api.rs
file, which looks — on first glance — pretty normal:
// Excerpt from `extension_api.rs`
/// A Zed extension.
pub trait Extension: Send + Sync {
/// Returns a new instance of the extension.
fn new() -> Self
where
Self: Sized;
/// Returns the command used to start the language server for the specified
/// language.
fn language_server_command(
&mut self,
_language_server_id: &LanguageServerId,
_worktree: &Worktree,
) -> Result<Command> {
Err("`language_server_command` not implemented".to_string())
}
/// Returns the initialization options to pass to the specified language server.
fn language_server_initialization_options(
&mut self,
_language_server_id: &LanguageServerId,
_worktree: &Worktree,
) -> Result<Option<serde_json::Value>> {
Ok(None)
}
// [...]
}It's a normal Rust trait that defines a bunch of methods with default implementations that extensions can choose to implement. It's hand-written (you'll find out in a few moments why I make that distinction) and defines the outer-most layer of our extension API. From the perspective of an extension's author, this trait is all they have to interact with.
In the same file, there are more type definitions, which we've seen in the
fictional extension from above too. LanguageServerId, for example. That's
defined here, like this:
/// The ID of a language server.
#[derive(Debug, PartialEq, Eq, PartialOrd, Ord, Hash, Clone)]
pub struct LanguageServerId(String);Again: normal looking Rust.
But then, if you look around that file and try to jump to some definitions,
you'll notice that there are some definitions that are not hand-written —
they don't even show up in the file. Worktree, for example, which we also saw
above in our fictional code. Where is that defined?
And this is where the Rust rubber hits the Wasm road. Or where the Wasm rubber hits the Rust road. Or Wasm sky meets Rust sea. Or Rust style meets Wasm substance — you get the idea, we're close to figuring things out.
Because those types are defined at compile time! That's right. The types
spring forth from this little paragraph in the same
extension_api.rs file, at compile time:
// Excerpt from extension_api.rs
mod wit {
wit_bindgen::generate!({
skip: ["init-extension"],
path: "./wit/since_v0.2.0",
});
}Now what does this paragraph do? To answer that, we have to take a step back.
Marshall and Max explained to me that extensions are built on the WebAssembly Component Model, which I've never heard of but have since researched. There's a lot we could talk about here, but to keep us focused, I'm going to skip over quite a few details here and mention only what's essential for our exploration: the WebAssembly Component Model allows us to define interfaces — APIs — between Wasm modules and between Wasm modules and the host in which they're executed. It allows us to define types that can be shared between Wasm modules and their host.
To put it in practical terms: without the Wasm Component Model, if you want to interact with a Wasm module — say by running it in a Wasm host and passing data to it, or calling a function in it and taking data out — all data that crosses the Wasm module boundary has to be represented as integers or floats, basically. (And if you want a Wasm module to interact with more than your own tools, it's convention to use the C ABI.)
Integers and floats are cool, don't get me wrong, but so are strings and structs.
And the Wasm Component Model allows us to pass strings, structs, arrays — all
that fancy stuff — to Wasm modules and get them back again. It allows us to
pass something like struct Animal { name: String, age: u8 } or the Worktree
from above to a Wasm module.
It requires that the types you want to exchange with a Wasm module are defined
ahead of time, using an IDL (an Interface Definition Language) called WIT,
short for Wasm Interface Type.
So, taking a step forward again, and looking at that code above, let's see what these 4 lines do:
wit_bindgen::generate!({
skip: ["init-extension"],
path: "./wit/since_v0.2.0",
});The generate! macro comes from the wit_bindgen crate. It takes
in the files in the ./wit/since_v0.2.0 directory, which contain WIT
definitions, and turns them into Wasm compatible types in Rust.
Here's an excerpt from the ./wit/since_v0.2.0/extension.wit file:
package zed:extension;
world extension {
/// A command.
record command {
/// The command to execute.
command: string,
/// The arguments to pass to the command.
args: list<string>,
/// The environment variables to set for the command.
env: env-vars,
}
/// A Zed worktree.
resource worktree {
/// Returns the ID of the worktree.
id: func() -> u64;
/// Returns the root path of the worktree.
root-path: func() -> string;
/// Returns the textual contents of the specified file in the worktree.
read-text-file: func(path: string) -> result<string, string>;
/// Returns the path to the given binary name, if one is present on the `$PATH`.
which: func(binary-name: string) -> option<string>;
/// Returns the current shell environment.
shell-env: func() -> env-vars;
}
/// Returns the command used to start up the language server.
export language-server-command: func(language-server-id: string, worktree: borrow<worktree>) -> result<command, string>;
/// [... other definitions ...]
}A command record, a worktree resource, and a language-server-command
function. language-server-command — sounds familiar, right? That's because
the fictional extension I showed you above contained an implementation of this
method that's part of the zed::Extension API:
fn language_server_command(
&mut self,
language_server_id: &LanguageServerId,
worktree: &Worktree,
) -> Result<Command> {
// ...
}Now how does this fit together — the definitions in *.wit files and the Rust code?
To answer that, let's ignore the &Worktree and the Result<Command> types and
focus on &LanguageServerId, which is easier to understand, because it is, as
we saw above, only a newtype around a String:
pub struct LanguageServerId(String);And in the code I just showed you, in ./wit/since_v0.2.0/extension.wit file,
the language-server-id is also just a string:
export language-server-command: func(language-server-id: string, worktree: borrow<worktree>) -> result<command, string>;That means we have a String on the Rust side (wrapped in another type that
gets added at runtime) and a string in the WIT file.
But Wasm modules don't know about strings! They only know about numbers and pointers to numbers (which are, technically, also numbers — don't send me angry letters).
To bridge that gap, wit_bindgen::generate! turns WIT definitions into Rust
type definitions that are valid Rust code on one side (so we can work with them
when writing an extension), but Wasm-compatible, C-ABI-exporting code on the
other side.
We can use cargo expand to see how the wit_bindgen::generate!
macro does that at compile time. Here's what wit_bindgen generates for the
language-server-command function:
#[export_name = "language-server-command"]
unsafe extern "C" fn export_language_server_command(
arg0: *mut u8,
arg1: usize,
arg2: i32,
) -> *mut u8 {
self::_export_language_server_command_cabi::<Component>(arg0, arg1, arg2)
}
unsafe fn _export_language_server_command_cabi<T: Guest>(
arg0: *mut u8,
arg1: usize,
arg2: i32,
) -> *mut u8 {
let handle1;
let len0 = arg1;
let bytes0 = _rt::Vec::from_raw_parts(arg0.cast(), len0, len0);
let result2 = T::language_server_command(
_rt::string_lift(bytes0),
{
handle1 = Worktree::from_handle(arg2 as u32);
&handle1
},
);
// [... a lot more code that only deals with numbers ...]
}The best way to understand this is to read it from the inside out: right there, in
the middle of _export_language_server_command_cabi you can see it calling
T::language_server_command — that's the call into the Rust code of our extension!
But since the Wasm module has to expose a C ABI compatible function — that's
the extern "C" fn export_language_server_command — it can't use String and
&Workspace. C and Wasm modules don't know about those types.
The solution is to turn the types we have - String and &Worktree — into C
ABI compatible types that Wasm modules can understand. The String gets turned
into two arguments: arg0: *mut u8, pointing to the data of the string, and
arg1, the length of the string. The borrow<worktree>, a reference to a
Worktree, is a i32 — a pointer.
In other words: wit_bindgen::generate! generates C ABI and Wasm module
compatible wrapper code around our Rust code, based on the WIT definitions. That
happens for all types and functions inside the WIT files, at compile time. And
then, together with the code of the extension, it's all compiled down to Wasm.
At this point, let's pause and admit to ourselves that, yes, this is a bit of a mind-bender. Maybe more than a bit.
But, leaving aside different tools, macros, and ABIs, the short version is this:
- Zed wants to execute Zed extensions that are Wasm modules and that implement
the
zed::Extensiontrait defined in Rust in theextension_apicrate. - Zed extensions, written in Rust, can implement the
zed::Extensiontrait. - To turn an extension that implements this trait into a Wasm module that
adheres to the pre-defined interface, we use WIT and
wit_bindgen!to generate Rust code that, when compiled down to Wasm together with the rest of the extension, exposes a Wasm module compatible API that calls into our once-written-in-Rust extension code.
Even shorter: extensions are written in Rust against a pre-defined interface and compiled into Wasm modules that can then be executed in Zed.
Now, how does that part work, the executing in Zed?
Running Wasm in Zed
Here's the life of a Zed extension so far:
We've defined an extension API, we wrote an extension that implements this API, we compiled it into Wasm, we uploaded that Wasm module, we clicked on the "Install extension" button, we downloaded the archive that contains the Wasm code — what next?
First, the downloaded extension archive is
extracted,
the files it contained put into the right places, and the extension added to an
internal index. Then this method, called
extensions_updated,
goes through all extensions, removes uninstalled ones, cleans them up, reloads
installed ones, and adds newly-added extensions.
Towards the end of extensions_updated, there's this bit of code which loads
the Wasm binary code of each extension:
// Extract from extension_store.rs
let mut path = root_dir.clone();
path.extend([extension.manifest.clone().id.as_ref(), "extension.wasm"]);
let mut wasm_file = fs
.open_sync(&path)
.await
.context("failed to open wasm file")?;
let mut wasm_bytes = Vec::new();
wasm_file
.read_to_end(&mut wasm_bytes)
.context("failed to read wasm")?;
wasm_host
.load_extension(
wasm_bytes,
extension.manifest.clone().clone(),
cx.background_executor().clone(),
)
.await
.with_context(|| {
format!("failed to load wasm extension {}", extension.manifest.id)
})The magic is in that last statement, the wasm_host.load_extension
call. That call
loads the Wasm module into the Wasm runtime we use in Zed: Wasmtime.
Here's the code:
// Excerpt from `wasm_host.rs`
pub fn load_extension(
self: &Arc<Self>,
wasm_bytes: Vec<u8>,
manifest: Arc<ExtensionManifest>,
executor: BackgroundExecutor,
) -> Task<Result<WasmExtension>> {
let this = self.clone();
executor.clone().spawn(async move {
let zed_api_version = parse_wasm_extension_version(&manifest.id, &wasm_bytes)?;
let component = Component::from_binary(&this.engine, &wasm_bytes)
.context("failed to compile wasm component")?;
let mut store = wasmtime::Store::new(
&this.engine,
WasmState {
ctx: this.build_wasi_ctx(&manifest).await?,
manifest: manifest.clone(),
table: ResourceTable::new(),
host: this.clone(),
},
);
let mut extension = Extension::instantiate_async(
&mut store,
this.release_channel,
zed_api_version,
&component,
)
.await?;
extension
.call_init_extension(&mut store)
.await
.context("failed to initialize wasm extension")?;
let (tx, mut rx) = mpsc::unbounded::<ExtensionCall>();
executor
.spawn(async move {
while let Some(call) = rx.next().await {
(call)(&mut extension, &mut store).await;
}
})
.detach();
Ok(WasmExtension {
manifest,
tx,
zed_api_version,
})
})
}It creates a wasmtime::Component from the binary data in the *.wasm file,
initializes a wasmtime::Store, and instantiates our Extension type with the
Extension::instantiate_async call.
That last bit, the instantiation of the Extension type, that's where the
rubber— I mean, that's where the extension becomes alive in Zed.
If we jump into the code and try to follow what happens under the hood in instantiate_async we'd first see this:
// Excerpt from wasm_host/wit.rs
impl Extension {
pub async fn instantiate_async(
store: &mut Store<WasmState>,
release_channel: ReleaseChannel,
version: SemanticVersion,
component: &Component,
) -> Result<Self> {
// Note: The release channel can be used to stage a new version of the extension API.
let allow_latest_version = match release_channel {
ReleaseChannel::Dev | ReleaseChannel::Nightly => true,
ReleaseChannel::Stable | ReleaseChannel::Preview => false,
};
if allow_latest_version && version >= latest::MIN_VERSION {
let extension =
latest::Extension::instantiate_async(store, component, latest::linker())
.await
.context("failed to instantiate wasm extension")?;
Ok(Self::V020(extension))
} else {
// [... code that handles other versions ...]
}
}
}That instantiates the extension depending on which extension API it was compiled
against. I told you above to remember that the extension API version is stored
in the expanded extension.toml — here's where it comes back into play.
But if we then try to jump to the definition of
latest::Extension::instantiate_async, which is called right there in the
middle, to see what it does, we won't get far, because — surprise! — that
code is generated at compile time too! And it's also generated from WIT files
— the same WIT files we saw earlier, in fact.
If we open up crates/extension/src/wasm_host/wit/since_v0_2_0.rs (the
wasm_host in the file path tells us that we're now on the other side of the
extension/host divide) we see this macro call:
wasmtime::component::bindgen!({
async: true,
trappable_imports: true,
path: "../extension_api/wit/since_v0.2.0",
with: {
"worktree": ExtensionWorktree,
"key-value-store": ExtensionKeyValueStore,
"zed:extension/http-client/http-response-stream": ExtensionHttpResponseStream
},
});Just like we created Wasm module compatible Rust bindings on the extension side
with wit_bindgen::generate!, this call to wasmtime::component::bindgen! now
creates host-side code. It uses the same WIT files that we saw earlier, this
time passing in resources to map WIT types to Zed-application-code types.
I won't dive into all the details here. It's enough to know that this instantiates the extension Wasm module inside Wasmtime and makes the host-side of the API available.
(If you want to know more, I highly recommend you read through the documentation
for the Wasm Component Model, wit_bindgen, and wasmtime — there are some
nice examples to play around with.)
One thing I want to call out though, because it blew my mind. See that async: true parameter in that wasmtime::component::bindgen! call? This parameter
ensures that the methods the host calls when calling into the extension are
async (as in: async Rust) even though they look like non-async functions
from inside the extension! That means if the extension does blocking IO by
downloading a language server, for example, it's async code from the host
perspective and the host can do other things, concurrently, while the extension
waits for its download.
Now, back to where we were. Once the latest::Extension::instantiate_async call
returns and the extension is initialized — then it's actually time to start
using it in Zed.
Here, for example, is how the language_server_command method from our
fictional extension from above would be called inside Zed application-level code
once the extension is initialized:
// extension_lsp_adapter.rs, simplified
struct ExtensionLspAdapter {
extension: WasmExtension,
language_server_id: LanguageServerName,
config: LanguageServerConfig,
host: Arc<WasmHost>,
}
#[async_trait(?Send)]
impl LspAdapter for ExtensionLspAdapter {
fn get_language_server_command<'a>(
self: Arc<Self>,
delegate: Arc<dyn LspAdapterDelegate>,
_: LanguageServerBinaryOptions,
_: futures::lock::MutexGuard<'a, Option<LanguageServerBinary>>,
_: &'a mut AsyncAppContext,
) -> Pin<Box<dyn 'a + Future<Output = Result<LanguageServerBinary>>>> {
async move {
let command = self
.extension
.call({
let this = self.clone();
|extension, store| {
async move {
let resource = store.data_mut().table().push(delegate)?;
let command = extension
.call_language_server_command(
store,
&this.language_server_id,
&this.config,
resource,
)
.await?
.map_err(|e| anyhow!("{}", e))?;
anyhow::Ok(command)
}
.boxed()
}
})
.await?;
let path = self
.host
.path_from_extension(&self.extension.manifest.id, command.command.as_ref());
// ...
Ok(LanguageServerBinary {
path,
arguments: command.args.into_iter().map(|arg| arg.into()).collect(),
env: Some(command.env.into_iter().collect()),
})
}
.boxed_local()
}
// [...]
}You have to squint a little bit to not get distracted by a lot of syntactic and
type system stuff going on here, but if you focus on the indented code in the
middle, you'll see this call: extension.call_language_server_command()
That, in turn, ends up calling this:
pub async fn call_language_server_command(
&self,
store: &mut Store<WasmState>,
language_server_id: &LanguageServerName,
config: &LanguageServerConfig,
resource: Resource<Arc<dyn LspAdapterDelegate>>,
) -> Result<Result<Command, String>> {
match self {
Extension::V020(ext) => {
ext.call_language_server_command(store, &language_server_id.0, resource)
.await
}
// [...]
}
}Recognize it yet? That's the host-side of the extern "C" fn export_language_server_command we saw above!
It passes in the LanguageServerName — the String! — and the resource,
which is what's called the worktree in the WIT files and on the extension
side.
Thanks to the host-side bindgen! calls, this call will end up turning the Rust
types we have here into C ABI compatible types and then use the Wasm engine to
call the export_language_server_command function that was generated on the
extension side.
... and that's it! That's the life of an extension. From Rust code, to Wasm, into an archive, up through the clouds, back down into your Zed, into a Wasm engine, and called from the Zed application code.
Dizzy?
Right place, right time
I certainly ended up feeling dizzy when walking through the code with Marshall and Max. There's a lot of indirection going on, a lot of different libraries and tools and APIs.
But it works, which, frankly, surprised me. I've worked with Wasm before and I couldn't believe that all the pieces here — WIT, Wasmtime, wit_bindgen, Rust — all fit together. Whenever I had tried to do something with Wasm in the past, I ran into abandoned libraries, or this engine following that standard, but that language's toolchain not following it yet, or the standard having been abandoned.
This time, for Zed, it was different. In our conversation, I asked Max and Marshall specifically about what a marvel it is that all these libraries and tools fit together. Marshall said the following (edited and paraphrased, for clarity):
When we were first playing around with the extension API, we were initially going in a JavaScript direction. We were going to embed a JavaScript runtime, looking at things like Deno's V8 bindings.
But while Max and I were working on it, we realized we don't really want to write JavaScript. We want to write our extensions in Rust.
But when we first looked at using WebAssembly for our extensions, it wasn't mature enough. So the effort fizzled out.
Then, at the start of the year, Max said, hey, let's look at this again. And we found the WebAssembly Component Model and all this stuff had only recently become usable, in the past few months.
We were in the right place, at the right time. We built a proof of concept. We got
wasmtimeand the Wasm component model plugged in. And everything just sort of clicked together at that point.
Right place, right time, resulting in some undeniably powerful stuff.
Shout-out to the hackers
I want to end this issue of Zed Decoded with the story that Marshall told us at the end of our conversation and give a shout-out to a real hacker.
Right now our extension API only allows users to add new language support, themes, snippets, and slash commands. There's no support for modifying the UI to create new panels, or making arbitrary HTTP requests, or touching the file system how you want.
It's limited and, yes, we are aware that people want a more powerful API (no, really, we know, you don't have to tell us), but we haven't gotten around to it, because there's a ton to do. (While some of you were asking for a more powerful extension API, others were asking: "Linux when?")
And yet, it turns out, you can do more with the API if you try hard enough.
Jozef Steinhübl, a 15-year-old hacker that goes by the name xhyrom on GitHub, built a Zed Discord presence extension. No, we don't offer any Discord features in the extension API. Jozef still made it work by — get this — building a language server that communicates with Zed to figure out which files the user has open and then broadcasting that to Discord.
Where one person sees a limitation, another person sees the eye of a needle through which they can thread their ideas. Beautiful.