Add Calling Swift from Rust
This commit is contained in:
parent
3474c02b6c
commit
92d844c316
|
|
@ -0,0 +1,215 @@
|
|||
```
|
||||
title = "Calling Swift from Rust"
|
||||
tags = ["rust", "swift"]
|
||||
date = "2023-02-18 20:29:42 -0500"
|
||||
short_desc = ""
|
||||
slug = "rust-swift"
|
||||
```
|
||||
|
||||
From the person that brought you [calling Rust from Swift](/2022/swift-rust/) comes the thrilling[^1], action[^2]-packed sequel: calling Swift from Rust! For a [recent project](/2023/rewritten-in-rust/), I found myself needing to call into Swift from a Rust project (on both macOS and Linux) and so am documenting here in case you, too, are in this unenviable situation.
|
||||
|
||||
[^1]: "Thrilling" is here defined as "confounding".
|
||||
|
||||
[^2]: Herein, "action" refers to linker errors.
|
||||
|
||||
<!-- excerpt-end -->
|
||||
|
||||
There are unfortunately few options for parsing and syntax highlighting Swift from Rust code. There does exist a Swift grammar for Tree Sitter, which I use for the rest of the new version of my blog, but using it is incredibly slow—just parsing the highlight query took upwards of 5 seconds. What's more, it doesn't produce especially good highlighting results. Since Swift accounts for a substantial portion of what I write about on here, that wasn't tenable.
|
||||
|
||||
The best Swift highlighter I know of is John Sundell's [Splash](https://github.com/johnsundell/Splash) library. But, since it's written in Swift, using it from my new blog engine is rather more complicated than ideal.
|
||||
|
||||
There is an existing package called [swift-rs](https://github.com/Brendonovich/swift-rs), however it doesn't work for this use case. It relies in no small part on Objective-C and its runtime for transferring values across the FFI boundary. My blog is hosted on a Linux machine, where no such runtime is present, so just using that crate is a no-go.
|
||||
|
||||
First off is the Swift package. This can be fairly simple, since it just exposes a single function annotated with `@_cdecl`. The only complication is that, since the Swift function is exposed to C, its type signature must be expressible in C. That means no complex Swift types, like String. But since the goal of this is syntax highlighting, passing a string across the FFI boundary is integral. So, strings are passed as a pointer to the underlying byte buffer and a length.
|
||||
```swift
|
||||
@_cdecl("highlight_swift")
|
||||
public func highlight(codePtr: UnsafePointer<UInt8>, codeLen: UInt64, htmlLenPtr: UnsafeMutablePointer<UInt64>) -> UnsafeMutablePointer<UInt8> {
|
||||
}
|
||||
```
|
||||
|
||||
Reading the input is accomplished by turning the base pointer and length into a buffer pointer, turning that into a `Data`, and finally into a `String`. Unfortunately, there are no zero-copy initializers[^3], so this always copies its input. Being in a Rust mindset, I really wanted to get rid of this copy, but there doesn't seem to be an obvious way, and at the end of the day, it's not actually a problem.
|
||||
|
||||
[^3]: There is a [`bytesNoCopy`](https://developer.apple.com/documentation/swift/string/init(bytesnocopy:length:encoding:freewhendone:)) initializer, but it's deprecated and the documentation notes that Swift doesn't support zero-copy initialization.
|
||||
|
||||
```swift
|
||||
let buf = UnsafeBufferPointer(start: codePtr, count: Int(codeLen))
|
||||
let data = Data(buffer: buf)
|
||||
let code = String(data: data, encoding: .utf8)!
|
||||
```
|
||||
|
||||
You may notice that the code string length is being passed into the function as an unsigned 64-bit integer, and then being converted to an `Int`, which may not be capable of representing the value. But, since this is just a syntax highlighter for my blog, there's absolutely no chance of it ever being used to highlight a string longer than 2^63-1 bytes.
|
||||
|
||||
The actual highlighting I'll skip, you can refer to the documentation for Splash. Once that's done, though, the output needs to be sent back to Rust somehow. Again, since the function signature needs to be compatible with C, it returns a pointer to a byte buffer containing the UTF-8 encoded string. It also sets a length pointer provided by the caller to the length in bytes of the output.
|
||||
|
||||
```swift
|
||||
var html = // ...
|
||||
let outPtr = UnsafeMutableBufferPointer<UInt8>.allocate(capacity: html.utf8.count)
|
||||
_ = html.withUTF8 { buf in
|
||||
buf.copyBytes(to: outPtr, count: buf.count)
|
||||
}
|
||||
htmlLenPtr.pointee = UInt64(outPtr.count)
|
||||
return outPtr.baseAddress!
|
||||
```
|
||||
|
||||
Note that the `html` string is declared as variable, since `withUTF8` may mutate it if the backing storage is not already contiguous.
|
||||
|
||||
The Swift code allocates a buffer of the appropriate length, and copies the data into it. By itself, this would leak, but the Rust code on the other side of the FFI boundary will take ownership of it and then deallocate it as usual when the string is dropped.
|
||||
|
||||
Invoking the Swift function on the Rust is is fairly straightforward. First the external function needs to be declared:
|
||||
|
||||
```rust
|
||||
extern "C" {
|
||||
fn highlight_swift(code_ptr: *const u8, code_len: u64, html_len_ptr: *mut u64) -> *mut u8;
|
||||
}
|
||||
```
|
||||
|
||||
Then, there's a little wrapper function that provides a more Rust-y interface, rather than the actual client having to deal with raw pointers:
|
||||
|
||||
```rust
|
||||
pub fn highlight(code: &str) -> String {
|
||||
unsafe {
|
||||
let mut html_len: u64 = 0;
|
||||
let html_ptr = highlight_swift(code.as_ptr(), code.len() as u64, &mut html_len);
|
||||
String::from_raw_parts(html_ptr, html_len as usize, html_len as usize)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
[`String::from_raw_parts`](https://doc.rust-lang.org/std/string/struct.String.html#method.from_raw_parts) takes the base pointer, the length, and the buffer capacity and produces an owned `String` that uses that buffer as its storage, with the given length and capacity. This does require that the buffer be managed by the same allocator as Rust's global one, but here everything is using the system `malloc`, so it's safe.
|
||||
|
||||
Next, comes actually building this thing. First, the Swift package product needs to be changed to be a static library:
|
||||
|
||||
```swift
|
||||
let package = Package(
|
||||
// ...
|
||||
products: [
|
||||
.library(
|
||||
name: "highlight-swift",
|
||||
type: .static,
|
||||
targets: ["highlight-swift"]
|
||||
)
|
||||
],
|
||||
// ...
|
||||
)
|
||||
```
|
||||
|
||||
Then, comes a whole bunch of stuff in the `build.rs` script of the Rust wrapper crate. Before I get into it, I want to note that much of this is based off the work of the [swift-rs](https://github.com/Brendonovich/swift-rs) project.
|
||||
|
||||
First comes a bunch of stuff to get the macOS build working. This part is cribbed from swift-rs, albeit simplified to only do exactly what I need. It needs to emit instructions for the Rust compiler to link against the Swift standard library as well as compile the Swift package and link against it too.
|
||||
|
||||
```rust
|
||||
fn main() {
|
||||
link_swift();
|
||||
link_swift_package("highlight-swift", "./highlight-swift/");
|
||||
}
|
||||
|
||||
fn link_swift() {
|
||||
let swift_target_info = get_swift_target_info();
|
||||
|
||||
swift_target_info
|
||||
.paths
|
||||
.runtime_library_paths
|
||||
.iter()
|
||||
.for_each(|path| {
|
||||
println!("cargo:rustc-link-search=native={}", path);
|
||||
});
|
||||
}
|
||||
|
||||
fn link_swift_package(package_name: &str, package_root: &str) {
|
||||
let profile = env::var("PROFILE").unwrap();
|
||||
|
||||
if !Command::new("swift")
|
||||
.args(&["build", "-c", &profile])
|
||||
.current_dir(package_root)
|
||||
.status()
|
||||
.unwrap()
|
||||
.success()
|
||||
{
|
||||
panic!("Failed to compile swift package {}", package_name);
|
||||
}
|
||||
|
||||
let swift_target_info = get_swift_target_info();
|
||||
|
||||
println!("cargo:rustc-link-search=native={}.build/{}/{}", package_root, swift_target_info.unversioned_triple, profile);
|
||||
println!("cargo:rustc-link-lib=static={}", package_name);
|
||||
}
|
||||
```
|
||||
|
||||
This relies on another supporting function from swift-rs, `get_swift_target_info`, which parses the output of `swift -print-target-info` to get information about the current target and location of the Swift stdlib. Note that this also requires `serde` and `serde_json` to be added to the `[build-dependencies]` section of `Cargo.toml`.
|
||||
|
||||
```rust
|
||||
fn get_swift_target_info() -> SwiftTarget {
|
||||
let swift_target_info_str = Command::new("swift")
|
||||
.args(&["-print-target-info"])
|
||||
.output()
|
||||
.unwrap()
|
||||
.stdout;
|
||||
serde_json::from_slice(&swift_target_info_str).unwrap()
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct SwiftTarget {
|
||||
target: SwiftTargetInfo,
|
||||
paths: SwiftPaths,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct SwiftTargetInfo {
|
||||
unversioned_triple: String,
|
||||
#[serde(rename = "librariesRequireRPath")]
|
||||
libraries_require_rpath: bool,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct SwiftPaths {
|
||||
runtime_library_paths: Vec<String>,
|
||||
runtime_resource_path: String,
|
||||
}
|
||||
```
|
||||
|
||||
This is enough to get everything to compile, link, and run on macOS. But unsurprisingly, running on Linux won't yet work. The macOS dynamic linker, dyld, is doing a lot of heavy lifting: at runtime it will dynamically link in the Swift standard library and make sure the Swift runtime is initialized. But the same is not the case on Linux, so the build script also needs to tell Rust which libraries to link against.
|
||||
|
||||
First, the Swift standard library paths need to be included in the Rust target's rpath, so that the dynamic libraries there will be located at runtime.
|
||||
|
||||
```rust
|
||||
fn main() {
|
||||
// ...
|
||||
let target = get_swift_target_info();
|
||||
if target.target.unversioned_triple.contains("linux") {
|
||||
target.paths.runtime_library_paths.iter().for_each(|path| {
|
||||
println!("cargo:rustc-link-arg=-Wl,-rpath={}", path);
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Then come the instructions to link against the various components of the Swift standard library. Not all of these are likely necessary, but rather than try to figure out which were, I opted to just link against all of of the `.so`s in the `/usr/lib/swift/linux` directory inside the Swift install.
|
||||
|
||||
```rust
|
||||
fn main() {
|
||||
// ...
|
||||
if target.target.unversioned_triple.contains("linux") {
|
||||
// ...
|
||||
println!("cargo:rustc-link-lib=dylib=BlocksRuntime");
|
||||
println!("cargo:rustc-link-lib=dylib=dispatch");
|
||||
// etc.
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Finally, the `swiftrt.o` object file, which actually initializes the runtime (needed for just about any non-trivial Swift code).
|
||||
|
||||
```rust
|
||||
fn main() {
|
||||
// ...
|
||||
if target.target.unversioned_triple.contains("linux") {
|
||||
// ...
|
||||
let arch = env::var("CARGO_CFG_TARGET_ARCH").unwrawp();
|
||||
println!("cargo:rustc-link-arg={}/linux/{}/swiftrt.o", &target.paths.runtime_resource_path, arch);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
I'm doing this by just passing the object file as a rustc link argument, since the linker will interpret it properly, but I'm not sure if there's a better way of telling rustc I want this object file included. Unfortunately, this method means that `swiftrt.o` isn't included when this crate is compiled into another one (and nor is the rpath linker flag used by the outer binary). So, in my actual blog crate, there's some [very similar code](https://git.shadowfacts.net/shadowfacts/v6/src/commit/c25100692be02b8293125450c34e8963c643f0c9/build.rs).
|
||||
|
||||
And, at long last, that is enough to run the program on Linux, with the Rust code calling into Swift and getting the result back as expected. If you want to see the entire project, it can be found [on my Gitea](https://git.shadowfacts.net/shadowfacts/splash-rs/).
|
||||
Loading…
Reference in New Issue