WIP: Use xtask as build system (#2012)

* xtask: Implement a new build system

xtask is a cargo alias that is used to extend the cargo build system
with custom commands. For an introduction to xtask, see here:
https://github.com/matklad/cargo-xtask/

The idea is that instead of writing makefiles, xtask requires no
additional dependencies except `cargo` and `rustc`, which must be
available to build the project anyway.

This commit provides a basic implementation of the `build` and `test`
subcommands.

* xtask/deps: Add 'which'

* xtask/test: Handle error when cargo not found

* xtask/flags: Add more commands

to perform different useful tasks. Includes:

- clippy
- format
- "make" (composite)
- "install" (composite)

Also add more options to `build` to selectively compile plugins or leave
them out entirely.

* xtask/main: Return error when cargo not found

* xtask/build: Add more subtasks

- `wasm_opt_plugins` and
- `manpage`

that perform other build commands. Add thorough documentation on what
each of these does and also handle the new `build` cli flags
appropriately.

* xtask/clippy: Add job to run clippy

* xtask/format: Add job to run rustfmt

* xtask/pipeline: Add composite commands

that perform multiple atomic xtask commands sequentially in a pipeline
sort of fashion.

* xtask/deps: Pin dependencies

* xtask/main: Integrate new jobs

and add documentation.

* xtask: Implement 'dist'

which performs an 'install' and copies the resulting zellij binary along
with some other assets to a `target/dist` folder.

* cargo: Update xflags version

* xtask: Measure task time, update tty title

* xtask: Update various tasks

* xtask: wasm-opt plugins in release builds

automatically.

* xtask/build: Copy debug plugins to assets folder

* xtask: Add 'run' subcommand

* xtask: Add arbitrary args to test and run

* xtask: Rearrange CLI commands in help

* xtask: Add deprecation notice

* docs: Replace `cargo make` with `xtask`

* github: Use `xtask` in workflows.

* xtask: Add support for CI commands

* xtask: Streamline error handling

* github: Use new xtask commands in CI

* xtask: Add 'publish' job

* xtask/publish: Add retry when publish fails

* xtask: Apply rustfmt

* xtask: Refine 'make' deprecation warning

* xtask: add task to build manpage

* contributing: Fix e2e commands

* xtask/run: Add missing `--`

to pass all arguments following `xtask run` directly to the zellij
binary being run.

* xtask: Stay in invocation dir

and make all tasks that need it change to the project root dir
themselves.

* xtask/run: Add `--data-dir` flag

which will allow very quick iterations when not changing the plugins
between builds.

* xtask/ci: Install dependencies without asking

* utils: Allow including plugins from target folder

* utils/assets: Reduce asset map complexity

* utils/consts: Update asset map docs

* xtask: Fix plugin includes

* xtask/test: Build plugins first

because the zellij binary needs to include the plugins.

* xtask/test: Fix formatting

* xtask: Add notice on how to disable it
This commit is contained in:
har7an 2022-12-17 13:27:18 +00:00 committed by GitHub
parent 6e93e8ffce
commit d1f50150f6
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
22 changed files with 1301 additions and 40 deletions

View file

@ -1 +1,6 @@
parallel-compiler = true parallel-compiler = true
[alias]
xtask = "run --package xtask --"
x = "xtask"
make = "xtask deprecated"

View file

@ -41,13 +41,11 @@ jobs:
run: sudo apt-get install -y --no-install-recommends musl-tools run: sudo apt-get install -y --no-install-recommends musl-tools
- name: Add musl target - name: Add musl target
run: rustup target add x86_64-unknown-linux-musl run: rustup target add x86_64-unknown-linux-musl
- name: Install cargo-make
run: nix profile install nixpkgs#cargo-make
- name: Install wasm-opt - name: Install wasm-opt
run: sudo apt-get install -y --no-install-recommends binaryen run: sudo apt-get install -y --no-install-recommends binaryen
#run: cargo install --debug cargo-make #run: cargo install --debug cargo-make
- name: Build asset - name: Build asset
run: cargo make build-e2e run: cargo xtask ci e2e --build
# we copy this manually into the target folder instead of mounting it because # we copy this manually into the target folder instead of mounting it because
# github actions creates the service first, and if it has a mount that is part # github actions creates the service first, and if it has a mount that is part
# of your yet unchecked out code, you cannot checkout the code after the mount # of your yet unchecked out code, you cannot checkout the code after the mount
@ -60,4 +58,4 @@ jobs:
with: with:
args: docker restart ssh args: docker restart ssh
- name: Test - name: Test
run: cargo make e2e-test run: cargo xtask ci e2e --test

View file

@ -59,9 +59,6 @@ jobs:
- name: Add WASM target - name: Add WASM target
run: rustup target add wasm32-wasi run: rustup target add wasm32-wasi
- name: Install cargo-make
run: cargo install --debug cargo-make
- name: Install musl-tools - name: Install musl-tools
if: matrix.os == 'ubuntu-latest' if: matrix.os == 'ubuntu-latest'
run: sudo apt-get install -y --no-install-recommends musl-tools run: sudo apt-get install -y --no-install-recommends musl-tools
@ -83,7 +80,7 @@ jobs:
EOF EOF
- name: Build release binary - name: Build release binary
run: cargo make ci-build-release ${{ matrix.target }} run: cargo xtask ci cross ${{ matrix.target }}
# this breaks on aarch64 and this if conditional isn't working for some reason: TODO: investigate # this breaks on aarch64 and this if conditional isn't working for some reason: TODO: investigate
#- name: Strip release binary #- name: Strip release binary

View file

@ -32,12 +32,10 @@ jobs:
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }} key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
- name: Add WASM target - name: Add WASM target
run: rustup target add wasm32-wasi run: rustup target add wasm32-wasi
- name: Install cargo-make
run: test -x "${HOME}/.cargo/bin/cargo-make" || cargo install --debug cargo-make
- name: Build - name: Build
run: cargo make build run: cargo xtask build
- name: Test - name: Test
run: cargo make test run: cargo xtask test
format: format:
name: Check Formatting name: Check Formatting
@ -53,10 +51,8 @@ jobs:
~/.cargo/git ~/.cargo/git
target target
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }} key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
- name: Install cargo-make
run: test -x "${HOME}/.cargo/bin/cargo-make" || cargo install --debug cargo-make
- name: Check Format - name: Check Format
run: cargo make check-format run: cargo xtask format --check
clippy: clippy:
name: Check Clippy Lints name: Check Clippy Lints
@ -72,7 +68,5 @@ jobs:
~/.cargo/git ~/.cargo/git
target target
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }} key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
- name: Install cargo-make
run: test -x "${HOME}/.cargo/bin/cargo-make" || cargo install --debug cargo-make
- name: Check clippy lints - name: Check clippy lints
run: cargo make clippy run: cargo xtask clippy

View file

@ -6,6 +6,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
## [Unreleased] ## [Unreleased]
* fix: show visual error when unable to split panes vertically/horizontally (https://github.com/zellij-org/zellij/pull/2025) * fix: show visual error when unable to split panes vertically/horizontally (https://github.com/zellij-org/zellij/pull/2025)
* build: Use `xtask` as build system (https://github.com/zellij-org/zellij/pull/2012)
## [0.34.4] - 2022-12-13 ## [0.34.4] - 2022-12-13

View file

@ -11,8 +11,9 @@ Before contributing please read our [Code of Conduct](CODE_OF_CONDUCT.md) which
all contributors are expected to adhere to. all contributors are expected to adhere to.
## Building ## Building
To build Zellij, we're using cargo-make you can install it by running `cargo
install --locked --force cargo-make`. To build Zellij, we're using cargo xtask. This is a standalone package shipped
inside the repository, so you don't have to install additional dependencies.
To edit our manpage, the mandown crate (`cargo install --locked To edit our manpage, the mandown crate (`cargo install --locked
mandown`) is used and the work is done on a markdown file in docs/MANPAGE.md. mandown`) is used and the work is done on a markdown file in docs/MANPAGE.md.
@ -21,25 +22,27 @@ Here are some of the commands currently supported by the build system:
```sh ```sh
# Format code, build, then run tests and clippy # Format code, build, then run tests and clippy
cargo make cargo xtask
# You can also perform these actions individually # You can also perform these actions individually
cargo make format cargo xtask format
cargo make build cargo xtask build
cargo make test cargo xtask test
# Run Zellij (optionally with additional arguments) # Run Zellij (optionally with additional arguments)
cargo make run cargo xtask run
cargo make run -l strider cargo xtask run -l strider
# Run Clippy (potentially with additional options) # Run Clippy
cargo make clippy cargo xtask clippy
cargo make clippy -W clippy::pedantic
# Install Zellij to some directory # Install Zellij to some directory
cargo make install /path/of/zellij/binary cargo xtask install /path/of/zellij/binary
# Publish the zellij and zellij-tile crates # Publish the zellij and zellij-tile crates
cargo make publish cargo xtask publish
# Update manpage # Update manpage
cargo make manpage cargo xtask manpage
``` ```
You can see a list of all commands (with supported arguments) with `cargo xtask
--help`. For convenience, `xtask` may be shortened to `x`: `cargo x build` etc.
To run `install` or `publish`, you'll need the package `binaryen` in the To run `install` or `publish`, you'll need the package `binaryen` in the
version `wasm-opt --version` > 97, for it's command `wasm-opt`. version `wasm-opt --version` > 97, for it's command `wasm-opt`.
@ -78,8 +81,8 @@ To run these tests locally, you'll need to have either `docker` or `podman` and
Once you do, in the repository root: Once you do, in the repository root:
1. `docker-compose up -d` will start up the docker container 1. `docker-compose up -d` will start up the docker container
2. `cargo make build-e2e` will build the generic linux executable of Zellij in the target folder, which is shared with the container 2. `cargo xtask ci e2e --build` will build the generic linux executable of Zellij in the target folder, which is shared with the container
3. `cargo make e2e-test` will run the tests 3. `cargo xtask ci e2e --test` will run the tests
To re-run the tests after you've changed something in the code base, be sure to repeat steps 2 and 3. To re-run the tests after you've changed something in the code base, be sure to repeat steps 2 and 3.

51
Cargo.lock generated
View file

@ -3067,6 +3067,15 @@ version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cda74da7e1a664f795bb1f8a87ec406fb89a02522cf6e50620d016add6dbbf5c" checksum = "cda74da7e1a664f795bb1f8a87ec406fb89a02522cf6e50620d016add6dbbf5c"
[[package]]
name = "toml"
version = "0.5.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1333c76748e868a4d9d1017b5ab53171dfd095f70c712fdb4653a406547f598f"
dependencies = [
"serde",
]
[[package]] [[package]]
name = "tracing" name = "tracing"
version = "0.1.35" version = "0.1.35"
@ -3878,6 +3887,48 @@ version = "0.36.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c811ca4a8c853ef420abd8592ba53ddbbac90410fab6903b3e79972a631f7680" checksum = "c811ca4a8c853ef420abd8592ba53ddbbac90410fab6903b3e79972a631f7680"
[[package]]
name = "xflags"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c4554b580522d0ca238369c16b8f6ce34524d61dafe7244993754bbd05f2c2ea"
dependencies = [
"xflags-macros",
]
[[package]]
name = "xflags-macros"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f58e7b3ca8977093aae6b87b6a7730216fc4c53a6530bab5c43a783cd810c1a8"
[[package]]
name = "xshell"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6d47097dc5c85234b1e41851b3422dd6d19b3befdd35b4ae5ce386724aeca981"
dependencies = [
"xshell-macros",
]
[[package]]
name = "xshell-macros"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "88301b56c26dd9bf5c43d858538f82d6f3f7764767defbc5d34e59459901c41a"
[[package]]
name = "xtask"
version = "0.1.0"
dependencies = [
"anyhow",
"lazy_static",
"toml",
"which",
"xflags",
"xshell",
]
[[package]] [[package]]
name = "yaml-rust" name = "yaml-rust"
version = "0.4.5" version = "0.4.5"

View file

@ -40,6 +40,7 @@ members = [
"zellij-utils", "zellij-utils",
"zellij-tile", "zellij-tile",
"zellij-tile-utils", "zellij-tile-utils",
"xtask",
".", ".",
] ]
@ -69,6 +70,6 @@ pkg-fmt = "tgz"
[features] [features]
# See remarks in zellij_utils/Cargo.toml # See remarks in zellij_utils/Cargo.toml
default = [ "zellij-utils/asset_map" ] default = [ "zellij-utils/plugins_from_target" ]
disable_automatic_asset_installation = [ "zellij-utils/disable_automatic_asset_installation" ] disable_automatic_asset_installation = [ "zellij-utils/disable_automatic_asset_installation" ]
unstable = [ "zellij-client/unstable", "zellij-utils/unstable" ] unstable = [ "zellij-client/unstable", "zellij-utils/unstable" ]

View file

@ -80,9 +80,8 @@ To get started, you can:
## How do I start a development environment? ## How do I start a development environment?
* Clone the project * Clone the project
* Install cargo-make with `cargo install --locked --force cargo-make` * In the project folder, for debug builds run: `cargo xtask run`
* In the project folder, for debug builds run: `cargo make run` * To run all tests: `cargo xtask test`
* To run all tests: `cargo make test`
For more build commands, see [CONTRIBUTING.md](CONTRIBUTING.md). For more build commands, see [CONTRIBUTING.md](CONTRIBUTING.md).

13
xtask/Cargo.toml Normal file
View file

@ -0,0 +1,13 @@
[package]
name = "xtask"
version = "0.1.0"
edition = "2021"
[dependencies]
anyhow = "1.0"
lazy_static = "1.4"
xshell = "= 0.2.2"
xflags = "0.3.1"
which = "4.2"
toml = "0.5"

167
xtask/src/build.rs Normal file
View file

@ -0,0 +1,167 @@
//! Subcommands for building.
//!
//! Currently has the following functions:
//!
//! - [`build`]: Builds general cargo projects (i.e. zellij components) with `cargo build`
//! - [`wasm_opt_plugin`]: Calls `wasm-opt` on all plugins
//! - [`manpage`]: Builds the manpage with `mandown`
use crate::flags;
use anyhow::Context;
use std::path::{Path, PathBuf};
use xshell::{cmd, Shell};
/// Build members of the zellij workspace.
///
/// Build behavior is controlled by the [`flags`](flags::Build). Calls some variation of `cargo
/// build` under the hood.
pub fn build(sh: &Shell, flags: flags::Build) -> anyhow::Result<()> {
let _pd = sh.push_dir(crate::project_root());
let cargo = crate::cargo()?;
if flags.no_plugins && flags.plugins_only {
eprintln!("Cannot use both '--no-plugins' and '--plugins-only'");
std::process::exit(1);
}
for subcrate in crate::WORKSPACE_MEMBERS.iter() {
let err_context = || format!("failed to build '{subcrate}'");
if subcrate.contains("plugins") {
if flags.no_plugins {
continue;
}
} else {
if flags.plugins_only {
continue;
}
}
let _pd = sh.push_dir(Path::new(subcrate));
// Tell the user where we are now
println!();
let msg = format!(">> Building '{subcrate}'");
crate::status(&msg);
println!("{}", msg);
let mut base_cmd = cmd!(sh, "{cargo} build");
if flags.release {
base_cmd = base_cmd.arg("--release");
}
base_cmd.run().with_context(err_context)?;
if subcrate.contains("plugins") {
let (_, plugin_name) = subcrate
.rsplit_once('/')
.context("Cannot determine plugin name from '{subcrate}'")?;
if flags.release {
// Perform wasm-opt on plugin
wasm_opt_plugin(sh, plugin_name).with_context(err_context)?;
}
}
}
Ok(())
}
/// Call `wasm-opt` on all plugins.
///
/// Plugins are discovered automatically by scanning the contents of `target/wasm32-wasi/release`
/// for filenames ending with `.wasm`. For this to work the plugins must be built beforehand.
// TODO: Should this panic if there is no plugin found? What should we do when only some plugins
// have been built before?
pub fn wasm_opt_plugin(sh: &Shell, plugin_name: &str) -> anyhow::Result<()> {
let err_context = || format!("failed to run 'wasm-opt' on plugin '{plugin_name}'");
let wasm_opt = wasm_opt(sh).with_context(err_context)?;
let asset_dir = crate::project_root()
.join("zellij-utils")
.join("assets")
.join("plugins");
sh.create_dir(&asset_dir).with_context(err_context)?;
let _pd = sh.push_dir(asset_dir);
let plugin = PathBuf::from(
std::env::var_os("CARGO_TARGET_DIR")
.unwrap_or(crate::project_root().join("target").into_os_string()),
)
.join("wasm32-wasi")
.join("release")
.join(plugin_name)
.with_extension("wasm");
if !plugin.is_file() {
return Err(anyhow::anyhow!("No plugin found at '{}'", plugin.display()))
.with_context(err_context);
}
let name = match plugin.file_name().with_context(err_context)?.to_str() {
Some(name) => name,
None => {
return Err(anyhow::anyhow!(
"couldn't read filename containing invalid unicode"
))
.with_context(err_context)
},
};
// This is a plugin we want to optimize
println!();
let msg = format!(">> Optimizing plugin '{name}'");
crate::status(&msg);
println!("{}", msg);
let input = plugin.as_path();
cmd!(sh, "{wasm_opt} -O {input} -o {name}")
.run()
.with_context(err_context)?;
Ok(())
}
/// Get the path to a `wasm-opt` executable.
///
/// If the executable isn't found, an error is returned instead.
// TODO: Offer the user to install latest wasm-opt on path?
fn wasm_opt(_sh: &Shell) -> anyhow::Result<PathBuf> {
match which::which("wasm-opt") {
Ok(path) => Ok(path),
Err(e) => {
println!("!! 'wasm-opt' wasn't found but is needed for this build step.");
println!("!! Please install it from here: https://github.com/WebAssembly/binaryen");
Err(e).context("couldn't find 'wasm-opt' executable")
},
}
}
/// Build the manpage with `mandown`.
// mkdir -p ${root_dir}/assets/man
// mandown ${root_dir}/docs/MANPAGE.md 1 > ${root_dir}/assets/man/zellij.1
pub fn manpage(sh: &Shell) -> anyhow::Result<()> {
let err_context = "failed to generate manpage";
let mandown = mandown(sh).context(err_context)?;
let project_root = crate::project_root();
let asset_dir = &project_root.join("assets").join("man");
sh.create_dir(&asset_dir).context(err_context)?;
let _pd = sh.push_dir(asset_dir);
cmd!(sh, "{mandown} {project_root}/docs/MANPAGE.md 1")
.read()
.and_then(|text| sh.write_file("zellij.1", text))
.context(err_context)
}
/// Get the path to a `mandown` executable.
///
/// If the executable isn't found, an error is returned instead.
fn mandown(_sh: &Shell) -> anyhow::Result<PathBuf> {
match which::which("mandown") {
Ok(path) => Ok(path),
Err(e) => {
eprintln!("!! 'mandown' wasn't found but is needed for this build step.");
eprintln!("!! Please install it with: `cargo install mandown`");
Err(e).context("Couldn't find 'mandown' executable")
},
}
}

172
xtask/src/ci.rs Normal file
View file

@ -0,0 +1,172 @@
//! Tasks related to zellij CI
use crate::{
build,
flags::{self, CiCmd, Cross, E2e},
};
use anyhow::Context;
use std::{ffi::OsString, path::PathBuf};
use xshell::{cmd, Shell};
pub fn main(sh: &Shell, flags: flags::Ci) -> anyhow::Result<()> {
let err_context = "failed to run CI task";
match flags.subcommand {
CiCmd::E2e(E2e {
build: false,
test: false,
..
}) => Err(anyhow::anyhow!(
"either '--build' or '--test' must be provided!"
)),
CiCmd::E2e(E2e {
build: true,
test: true,
..
}) => Err(anyhow::anyhow!(
"flags '--build' and '--test' are mutually exclusive!"
)),
CiCmd::E2e(E2e {
build: true,
test: false,
..
}) => e2e_build(sh),
CiCmd::E2e(E2e {
build: false,
test: true,
args,
}) => e2e_test(sh, args),
CiCmd::Cross(Cross { triple }) => cross_compile(sh, &triple),
}
.context(err_context)
}
fn e2e_build(sh: &Shell) -> anyhow::Result<()> {
let err_context = "failed to build E2E binary";
build::build(
sh,
flags::Build {
release: true,
no_plugins: false,
plugins_only: true,
},
)
.context(err_context)?;
// Copy plugins to e2e data-dir
let project_root = crate::project_root();
let plugin_dir = project_root
.join("zellij-utils")
.join("assets")
.join("plugins");
let data_dir = project_root.join("target").join("e2e-data");
let plugins: Vec<_> = std::fs::read_dir(plugin_dir)
.context(err_context)?
.filter_map(|dir_entry| {
if let Ok(entry) = dir_entry {
entry
.file_name()
.to_string_lossy()
.ends_with(".wasm")
.then_some(entry.path())
} else {
None
}
})
.collect();
sh.remove_path(&data_dir)
.and_then(|_| sh.create_dir(&data_dir))
.and_then(|_| sh.create_dir(&data_dir.join("plugins")))
.context(err_context)?;
for plugin in plugins {
sh.copy_file(plugin, data_dir.join("plugins"))
.context(err_context)?;
}
let _pd = sh.push_dir(project_root);
crate::cargo()
.and_then(|cargo| {
cmd!(
sh,
"{cargo} build --verbose --release --target x86_64-unknown-linux-musl"
)
.run()
.map_err(anyhow::Error::new)
})
.context(err_context)
}
fn e2e_test(sh: &Shell, args: Vec<OsString>) -> anyhow::Result<()> {
let err_context = "failed to run E2E tests";
let _pd = sh.push_dir(crate::project_root());
e2e_build(sh).context(err_context)?;
// Build debug plugins for test binary
build::build(
sh,
flags::Build {
release: false,
no_plugins: false,
plugins_only: true,
},
)
.context(err_context)?;
crate::cargo()
.and_then(|cargo| {
cmd!(sh, "{cargo} test -- --ignored --nocapture --test-threads 1")
.args(args)
.run()
.map_err(anyhow::Error::new)
})
.context(err_context)
}
fn cross_compile(sh: &Shell, target: &OsString) -> anyhow::Result<()> {
let err_context = || format!("failed to cross-compile for {target:?}");
crate::cargo()
.and_then(|cargo| {
cmd!(sh, "{cargo} install mandown").run()?;
Ok(cargo)
})
.and_then(|cargo| {
cmd!(sh, "{cargo} install cross")
.run()
.map_err(anyhow::Error::new)
})
.with_context(err_context)?;
build::build(
sh,
flags::Build {
release: true,
no_plugins: false,
plugins_only: true,
},
)
.and_then(|_| build::manpage(sh))
.with_context(err_context)?;
cross()
.and_then(|cross| {
cmd!(sh, "{cross} build --verbose --release --target {target}")
.run()
.map_err(anyhow::Error::new)
})
.with_context(err_context)
}
fn cross() -> anyhow::Result<PathBuf> {
match which::which("cross") {
Ok(path) => Ok(path),
Err(e) => {
eprintln!("!! 'cross' wasn't found but is needed for this build step.");
eprintln!("!! Please install it with: `cargo install cross`");
Err(e).context("couldn't find 'cross' executable")
},
}
}

43
xtask/src/clippy.rs Normal file
View file

@ -0,0 +1,43 @@
//! Handle running `cargo clippy` on the sources.
use crate::{build, flags};
use anyhow::Context;
use std::path::{Path, PathBuf};
use xshell::{cmd, Shell};
pub fn clippy(sh: &Shell, _flags: flags::Clippy) -> anyhow::Result<()> {
let _pd = sh.push_dir(crate::project_root());
build::build(
sh,
flags::Build {
release: false,
no_plugins: false,
plugins_only: true,
},
)
.context("failed to run task 'clippy'")?;
let cargo = check_clippy()
.and_then(|_| crate::cargo())
.context("failed to run task 'clippy'")?;
for subcrate in crate::WORKSPACE_MEMBERS.iter() {
let _pd = sh.push_dir(Path::new(subcrate));
// Tell the user where we are now
println!();
let msg = format!(">> Running clippy on '{subcrate}'");
crate::status(&msg);
println!("{}", msg);
cmd!(sh, "{cargo} clippy --all-targets --all-features")
.run()
.with_context(|| format!("failed to run task 'clippy' on '{subcrate}'"))?;
}
Ok(())
}
fn check_clippy() -> anyhow::Result<PathBuf> {
which::which("cargo-clippy").context(
"Couldn't find 'clippy' executable. Please install it with `rustup component add clippy`",
)
}

1
xtask/src/dist.rs Normal file
View file

@ -0,0 +1 @@

211
xtask/src/flags.rs Normal file
View file

@ -0,0 +1,211 @@
//! CLI flags for `cargo xtask`
use std::ffi::OsString;
use std::path::PathBuf;
xflags::xflags! {
src "./src/flags.rs"
/// Custom build commands for zellij
cmd xtask {
/// Deprecation warning. Compatibility to transition from `cargo make`.
cmd deprecated {
repeated args: OsString
}
/// Tasks for the CI
cmd ci {
/// end-to-end tests
cmd e2e {
/// Build E2E binary of zellij
optional --build
/// Run the E2E tests
optional --test
/// Additional arguments for `--test`
repeated args: OsString
}
/// Perform cross-compiled release builds
cmd cross {
/// Target-triple to compile the application for
required triple: OsString
}
}
/// Build the manpage
cmd manpage {}
/// Publish zellij and all the sub-crates
cmd publish {
/// Perform a dry-run (don't push/publish anything)
optional --dry-run
}
/// Package zellij for distribution (result found in ./target/dist)
cmd dist {}
/// Run `cargo clippy` on all crates
cmd clippy {}
/// Sequentially call: format, build, test, clippy
cmd make {
/// Build in release mode without debug symbols
optional -r, --release
/// Clean project before building
optional -c, --clean
}
/// Generate a runnable `zellij` executable with plugins bundled
cmd install {
required destination: PathBuf
}
/// Run debug version of zellij
cmd run {
/// Take plugins from here, skip building plugins. Passed to zellij verbatim
optional --data-dir path: PathBuf
/// Arguments to pass after `cargo run --`
repeated args: OsString
}
/// Run `cargo fmt` on all crates
cmd format {
/// Run `cargo fmt` in check mode
optional --check
}
/// Run application tests
cmd test {
/// Arguments to pass after `cargo test --`
repeated args: OsString
}
/// Build the application and all plugins
cmd build {
/// Build in release mode without debug symbols
optional -r, --release
/// Build only the plugins
optional -p, --plugins-only
/// Build everything except the plugins
optional --no-plugins
}
}
}
// generated start
// The following code is generated by `xflags` macro.
// Run `env UPDATE_XFLAGS=1 cargo build` to regenerate.
#[derive(Debug)]
pub struct Xtask {
pub subcommand: XtaskCmd,
}
#[derive(Debug)]
pub enum XtaskCmd {
Deprecated(Deprecated),
Ci(Ci),
Manpage(Manpage),
Publish(Publish),
Dist(Dist),
Clippy(Clippy),
Make(Make),
Install(Install),
Run(Run),
Format(Format),
Test(Test),
Build(Build),
}
#[derive(Debug)]
pub struct Deprecated {
pub args: Vec<OsString>,
}
#[derive(Debug)]
pub struct Ci {
pub subcommand: CiCmd,
}
#[derive(Debug)]
pub enum CiCmd {
E2e(E2e),
Cross(Cross),
}
#[derive(Debug)]
pub struct E2e {
pub args: Vec<OsString>,
pub build: bool,
pub test: bool,
}
#[derive(Debug)]
pub struct Cross {
pub triple: OsString,
}
#[derive(Debug)]
pub struct Manpage;
#[derive(Debug)]
pub struct Publish {
pub dry_run: bool,
}
#[derive(Debug)]
pub struct Dist;
#[derive(Debug)]
pub struct Clippy;
#[derive(Debug)]
pub struct Make {
pub release: bool,
pub clean: bool,
}
#[derive(Debug)]
pub struct Install {
pub destination: PathBuf,
}
#[derive(Debug)]
pub struct Run {
pub args: Vec<OsString>,
pub data_dir: Option<PathBuf>,
}
#[derive(Debug)]
pub struct Format {
pub check: bool,
}
#[derive(Debug)]
pub struct Test {
pub args: Vec<OsString>,
}
#[derive(Debug)]
pub struct Build {
pub release: bool,
pub plugins_only: bool,
pub no_plugins: bool,
}
impl Xtask {
#[allow(dead_code)]
pub fn from_env_or_exit() -> Self {
Self::from_env_or_exit_()
}
#[allow(dead_code)]
pub fn from_env() -> xflags::Result<Self> {
Self::from_env_()
}
#[allow(dead_code)]
pub fn from_vec(args: Vec<std::ffi::OsString>) -> xflags::Result<Self> {
Self::from_vec_(args)
}
}
// generated end

36
xtask/src/format.rs Normal file
View file

@ -0,0 +1,36 @@
//! Handle running `cargo fmt` on the sources.
use crate::flags;
use anyhow::Context;
use std::path::{Path, PathBuf};
use xshell::{cmd, Shell};
pub fn format(sh: &Shell, flags: flags::Format) -> anyhow::Result<()> {
let _pd = sh.push_dir(crate::project_root());
let cargo = check_rustfmt()
.and_then(|_| crate::cargo())
.context("failed to run task 'format'")?;
for subcrate in crate::WORKSPACE_MEMBERS.iter() {
let _pd = sh.push_dir(Path::new(subcrate));
// Tell the user where we are now
println!();
let msg = format!(">> Formatting '{subcrate}'");
crate::status(&msg);
println!("{}", msg);
let mut cmd = cmd!(sh, "{cargo} fmt");
if flags.check {
cmd = cmd.arg("--check");
}
cmd.run()
.with_context(|| format!("Failed to format '{subcrate}'"))?;
}
Ok(())
}
fn check_rustfmt() -> anyhow::Result<PathBuf> {
which::which("rustfmt").context(
"Couldn't find 'rustfmt' executable. Please install it with `cargo install rustfmt`",
)
}

152
xtask/src/main.rs Normal file
View file

@ -0,0 +1,152 @@
//! See <https://github.com/matklad/cargo-xtask/>.
//!
//! This binary defines various auxiliary build commands, which are not expressible with just
//! `cargo`. Notably, it provides tests via `cargo test -p xtask` for code generation and `cargo
//! xtask install` for installation of rust-analyzer server and client.
//!
//! This binary is integrated into the `cargo` command line by using an alias in `.cargo/config`.
// Current default "flow":
// - format-flow: `cargo fmt`
// - format-toml-conditioned-flow: ??
// - build: `cargo build`
// - test: `cargo test`
// - clippy: `cargo clippy --all-targets --all-features -- --deny warnings $@`
//
// # Install flow:
// - build-plugins-release: `cargo build --release ...`
// - wasm-opt-plugins: `wasm-opt ...`
// - build-release: `cargo build --release`
// - install-mandown: `cargo install mandown`
// - manpage: |
// mkdir -p ${root_dir}/assets/man
// mandown ${root_dir}/docs/MANPAGE.md 1 > ${root_dir}/assets/man/zellij.1
// - install: `cp target/release/zellij "$1"`
//
// # Release flow:
// - workspace: cargo make --profile development -- release
//
// # Publish flow:
// - update-default-config:
// - build-plugins-release: `cargo build --release ...`
// - wasm-opt-plugins: `wasm-opt ...`
// - release-commit:
// - commit-all: `git commit -aem "chore(release): v${CRATE_VERSION}"`
// - tag-release: `git tag --annotate --message "Version ${CRATE_VERSION}"
// "v${CRATE_VERSION}"`
// - `git push --atomic origin main "v${CRATE_VERSION}"`
// - publish-zellij: `cargo publish [tile, client, server, utils, tile-utils, zellij]`
mod build;
mod ci;
mod clippy;
mod dist;
mod flags;
mod format;
mod pipelines;
mod test;
use anyhow::Context;
use std::{
env,
path::{Path, PathBuf},
time::Instant,
};
use xshell::Shell;
lazy_static::lazy_static! {
pub static ref WORKSPACE_MEMBERS: Vec<&'static str> = vec![
"default-plugins/compact-bar",
"default-plugins/status-bar",
"default-plugins/strider",
"default-plugins/tab-bar",
"zellij-utils",
"zellij-tile-utils",
"zellij-tile",
"zellij-client",
"zellij-server",
".",
];
}
fn main() -> anyhow::Result<()> {
let shell = &Shell::new()?;
let flags = flags::Xtask::from_env()?;
let now = Instant::now();
match flags.subcommand {
flags::XtaskCmd::Deprecated(_flags) => deprecation_notice(),
flags::XtaskCmd::Dist(flags) => pipelines::dist(shell, flags),
flags::XtaskCmd::Build(flags) => build::build(shell, flags),
flags::XtaskCmd::Clippy(flags) => clippy::clippy(shell, flags),
flags::XtaskCmd::Format(flags) => format::format(shell, flags),
flags::XtaskCmd::Test(flags) => test::test(shell, flags),
flags::XtaskCmd::Manpage(_flags) => build::manpage(shell),
// Pipelines
// These are composite commands, made up of multiple "stages" defined above.
flags::XtaskCmd::Make(flags) => pipelines::make(shell, flags),
flags::XtaskCmd::Install(flags) => pipelines::install(shell, flags),
flags::XtaskCmd::Run(flags) => pipelines::run(shell, flags),
flags::XtaskCmd::Ci(flags) => ci::main(shell, flags),
flags::XtaskCmd::Publish(flags) => pipelines::publish(shell, flags),
}?;
let elapsed = now.elapsed().as_secs();
status(&format!("xtask (done after {} s)", elapsed));
println!("\n\n>> Command took {} s", elapsed);
Ok(())
}
fn project_root() -> PathBuf {
Path::new(
&env::var("CARGO_MANIFEST_DIR").unwrap_or_else(|_| env!("CARGO_MANIFEST_DIR").to_owned()),
)
.ancestors()
.nth(1)
.unwrap()
.to_path_buf()
}
pub fn cargo() -> anyhow::Result<PathBuf> {
std::env::var_os("CARGO")
.map_or_else(|| which::which("cargo"), |exe| Ok(PathBuf::from(exe)))
.context("Couldn't find 'cargo' executable")
}
// Set terminal title to 'msg'
pub fn status(msg: &str) {
print!("\u{1b}]0;{}\u{07}", msg);
}
fn deprecation_notice() -> anyhow::Result<()> {
Err(anyhow::anyhow!(
" !!! cargo make has been deprecated by zellij !!!
Our build system is now `cargo xtask`. Don't worry, you won't have to install
anything!
- To get an overview of the new build tasks, run `cargo xtask --help`
- Quick compatibility table:
| cargo make task | cargo xtask equivalent |
| ------------------------------- | ----------------------------- |
| make | xtask |
| make format | xtask format |
| make build | xtask build |
| make test | xtask test |
| make run | xtask run |
| make run -l strider | xtask run -- -l strider |
| make clippy | xtask clippy |
| make clippy -W clippy::pedantic | N/A |
| make install /path/to/binary | xtask install /path/to/binary |
| make publish | xtask publish |
| make manpage | xtask manpage |
In order to disable xtask during the transitioning period: Delete/comment the
`[alias]` section in `.cargo/config.toml` and use `cargo make` as before.
If you're unhappy with `xtask` and decide to disable it, please tell us why so
we can discuss this before making it final for the next release. Thank you!
"
))
}

336
xtask/src/pipelines.rs Normal file
View file

@ -0,0 +1,336 @@
//! Composite pipelines for the build system.
//!
//! Defines multiple "pipelines" that run specific individual steps in sequence.
use crate::flags;
use crate::{build, clippy, format, test};
use anyhow::Context;
use xshell::{cmd, Shell};
/// Perform a default build.
///
/// Runs the following steps in sequence:
///
/// - format
/// - build
/// - test
/// - clippy
pub fn make(sh: &Shell, flags: flags::Make) -> anyhow::Result<()> {
let err_context = || format!("failed to run pipeline 'make' with args {flags:?}");
if flags.clean {
crate::cargo()
.and_then(|cargo| cmd!(sh, "{cargo} clean").run().map_err(anyhow::Error::new))
.with_context(err_context)?;
}
format::format(sh, flags::Format { check: false })
.and_then(|_| {
build::build(
sh,
flags::Build {
release: flags.release,
no_plugins: false,
plugins_only: false,
},
)
})
.and_then(|_| test::test(sh, flags::Test { args: vec![] }))
.and_then(|_| clippy::clippy(sh, flags::Clippy {}))
.with_context(err_context)
}
/// Generate a runnable executable.
///
/// Runs the following steps in sequence:
///
/// - [`build`](build::build) (release, plugins only)
/// - [`wasm_opt_plugins`](build::wasm_opt_plugins)
/// - [`build`](build::build) (release, without plugins)
/// - [`manpage`](build::manpage)
/// - Copy the executable to [target file](flags::Install::destination)
pub fn install(sh: &Shell, flags: flags::Install) -> anyhow::Result<()> {
let err_context = || format!("failed to run pipeline 'install' with args {flags:?}");
// Build and optimize plugins
build::build(
sh,
flags::Build {
release: true,
no_plugins: false,
plugins_only: true,
},
)
.and_then(|_| {
// Build the main executable
build::build(
sh,
flags::Build {
release: true,
no_plugins: true,
plugins_only: false,
},
)
})
.and_then(|_| {
// Generate man page
build::manpage(sh)
})
.with_context(err_context)?;
// Copy binary to destination
let destination = if flags.destination.is_absolute() {
flags.destination.clone()
} else {
std::env::current_dir()
.context("Can't determine current working directory")?
.join(&flags.destination)
};
sh.change_dir(crate::project_root());
sh.copy_file("target/release/zellij", &destination)
.with_context(err_context)
}
/// Run zellij debug build.
pub fn run(sh: &Shell, flags: flags::Run) -> anyhow::Result<()> {
let err_context = || format!("failed to run pipeline 'run' with args {flags:?}");
if let Some(ref data_dir) = flags.data_dir {
let data_dir = sh.current_dir().join(data_dir);
crate::cargo()
.and_then(|cargo| {
cmd!(sh, "{cargo} run")
.args(["--package", "zellij"])
.arg("--no-default-features")
.args(["--features", "disable_automatic_asset_installation"])
.args(["--", "--data-dir", &format!("{}", data_dir.display())])
.run()
.map_err(anyhow::Error::new)
})
.with_context(err_context)
} else {
build::build(
sh,
flags::Build {
release: false,
no_plugins: false,
plugins_only: true,
},
)
.and_then(|_| crate::cargo())
.and_then(|cargo| {
cmd!(sh, "{cargo} run --")
.args(&flags.args)
.run()
.map_err(anyhow::Error::new)
})
.with_context(err_context)
}
}
/// Bundle all distributable content to `target/dist`.
///
/// This includes the optimized zellij executable from the [`install`] pipeline, the man page, the
/// `.desktop` file and the application logo.
pub fn dist(sh: &Shell, _flags: flags::Dist) -> anyhow::Result<()> {
let err_context = || format!("failed to run pipeline 'dist'");
sh.change_dir(crate::project_root());
if sh.path_exists("target/dist") {
sh.remove_path("target/dist").with_context(err_context)?;
}
sh.create_dir("target/dist")
.map_err(anyhow::Error::new)
.and_then(|_| {
install(
sh,
flags::Install {
destination: crate::project_root().join("./target/dist/zellij"),
},
)
})
.with_context(err_context)?;
sh.create_dir("target/dist/man")
.and_then(|_| sh.copy_file("assets/man/zellij.1", "target/dist/man/zellij.1"))
.and_then(|_| sh.copy_file("assets/zellij.desktop", "target/dist/zellij.desktop"))
.and_then(|_| sh.copy_file("assets/logo.png", "target/dist/logo.png"))
.with_context(err_context)
}
/// Make a zellij release and publish all crates.
pub fn publish(sh: &Shell, flags: flags::Publish) -> anyhow::Result<()> {
let err_context = "failed to publish zellij";
sh.change_dir(crate::project_root());
let dry_run = if flags.dry_run {
Some("--dry-run")
} else {
None
};
let cargo = crate::cargo().context(err_context)?;
let project_dir = crate::project_root();
let manifest = sh
.read_file(project_dir.join("Cargo.toml"))
.context(err_context)?
.parse::<toml::Value>()
.context(err_context)?;
// Version of the core crate
let version = manifest
.get("package")
.and_then(|package| package["version"].as_str())
.context(err_context)?;
let mut skip_build = false;
if cmd!(sh, "git tag -l")
.read()
.context(err_context)?
.contains(version)
{
println!();
println!("Git tag 'v{version}' is already present.");
println!("If this is a mistake, delete it with: git tag -d 'v{version}'");
println!("Skip build phase and continue to publish? [y/n]");
let stdin = std::io::stdin();
loop {
let mut buffer = String::new();
stdin.read_line(&mut buffer).context(err_context)?;
match buffer.trim_end() {
"y" | "Y" => {
skip_build = true;
break;
},
"n" | "N" => {
skip_build = false;
break;
},
_ => {
println!(" --> Unknown input '{buffer}', ignoring...");
println!();
println!("Skip build phase and continue to publish? [y/n]");
},
}
}
}
if !skip_build {
// Clean project
cmd!(sh, "{cargo} clean").run().context(err_context)?;
// Build plugins
build::build(
sh,
flags::Build {
release: true,
no_plugins: false,
plugins_only: true,
},
)
.context(err_context)?;
// Update default config
sh.copy_file(
project_dir
.join("zellij-utils")
.join("assets")
.join("config")
.join("default.kdl"),
project_dir.join("example").join("default.kdl"),
)
.context(err_context)?;
// Commit changes
cmd!(sh, "git commit -aem")
.arg(format!("chore(release): v{}", version))
.run()
.context(err_context)?;
// Tag release
cmd!(sh, "git tag --annotate --message")
.arg(format!("Version {}", version))
.arg(format!("v{}", version))
.run()
.context(err_context)?;
}
let closure = || -> anyhow::Result<()> {
// Push commit and tag
if flags.dry_run {
println!("Skipping push due to dry-run");
} else {
cmd!(sh, "git push --atomic origin main v{version}")
.run()
.context(err_context)?;
}
// Publish all the crates
for member in crate::WORKSPACE_MEMBERS.iter() {
if member.contains("plugin") {
continue;
}
let _pd = sh.push_dir(project_dir.join(member));
loop {
if let Err(err) = cmd!(sh, "{cargo} publish {dry_run...}")
.run()
.context(err_context)
{
println!();
println!("Publishing crate '{member}' failed with error:");
println!("{:?}", err);
println!();
println!("Retry? [y/n]");
let stdin = std::io::stdin();
let mut buffer = String::new();
let retry: bool;
loop {
stdin.read_line(&mut buffer).context(err_context)?;
match buffer.trim_end() {
"y" | "Y" => {
retry = true;
break;
},
"n" | "N" => {
retry = false;
break;
},
_ => {
println!(" --> Unknown input '{buffer}', ignoring...");
println!();
println!("Retry? [y/n]");
},
}
}
if retry {
continue;
} else {
println!("Aborting publish for crate '{member}'");
return Err::<(), _>(err);
}
} else {
println!("Waiting for crates.io to catch up...");
std::thread::sleep(std::time::Duration::from_secs(15));
break;
}
}
}
Ok(())
};
// We run this in a closure so that a failure in any of the commands doesn't abort the whole
// program. When dry-running we need to undo the release commit first!
let result = closure();
if flags.dry_run {
cmd!(sh, "git reset --hard HEAD~1")
.run()
.context(err_context)?;
}
result
}

66
xtask/src/test.rs Normal file
View file

@ -0,0 +1,66 @@
use crate::{build, flags};
use anyhow::{anyhow, Context};
use std::path::Path;
use xshell::{cmd, Shell};
pub fn test(sh: &Shell, flags: flags::Test) -> anyhow::Result<()> {
let err_context = "failed to run task 'test'";
let _pdo = sh.push_dir(crate::project_root());
let cargo = crate::cargo().context(err_context)?;
let host_triple = host_target_triple(sh).context(err_context)?;
build::build(
sh,
flags::Build {
release: false,
no_plugins: false,
plugins_only: true,
},
)
.context(err_context)?;
for subcrate in crate::WORKSPACE_MEMBERS.iter() {
let _pd = sh.push_dir(Path::new(subcrate));
// Tell the user where we are now
println!("");
let msg = format!(">> Testing '{}'", subcrate);
crate::status(&msg);
println!("{}", msg);
cmd!(sh, "{cargo} test --target {host_triple} --")
.args(&flags.args)
.run()
.with_context(|| format!("Failed to run tests for '{}'", subcrate))?;
}
Ok(())
}
// Determine the target triple of the host. We explicitly run all tests against the host
// architecture so we can test the plugins, too (they default to wasm32-wasi otherwise).
pub fn host_target_triple(sh: &Shell) -> anyhow::Result<String> {
let rustc_ver = cmd!(sh, "rustc -vV")
.read()
.context("Failed to determine host triple")?;
let maybe_triple = rustc_ver
.lines()
.filter_map(|line| {
if !line.starts_with("host") {
return None;
}
if let Some((_, triple)) = line.split_once(": ") {
return Some(triple.to_string());
} else {
return None;
}
})
.collect::<Vec<String>>();
match maybe_triple.len() {
0 => Err(anyhow!("rustc didn't output the 'host' triple")),
1 => Ok(maybe_triple.into_iter().next().unwrap()),
_ => Err(anyhow!(
"rustc provided multiple host triples: {:?}",
maybe_triple
)),
}
}

View file

@ -98,7 +98,7 @@ If you're a user:
If you're a developer: If you're a developer:
Please run zellij with updated plugins. The easiest way to achieve this Please run zellij with updated plugins. The easiest way to achieve this
is to build zellij with `cargo make install`. Also refer to the docs: is to build zellij with `cargo xtask install`. Also refer to the docs:
https://github.com/zellij-org/zellij/blob/main/CONTRIBUTING.md#building https://github.com/zellij-org/zellij/blob/main/CONTRIBUTING.md#building
", ",
first_line, first_line,

View file

@ -57,4 +57,4 @@ insta = { version = "1.6.0", features = ["backtrace"] }
# - builtin plugins MUST be available from whatever is configured as `PLUGIN_DIR` # - builtin plugins MUST be available from whatever is configured as `PLUGIN_DIR`
disable_automatic_asset_installation = [] disable_automatic_asset_installation = []
unstable = [] unstable = []
asset_map = [] plugins_from_target = []

View file

@ -45,16 +45,31 @@ mod not_wasm {
use std::path::PathBuf; use std::path::PathBuf;
// Convenience macro to add plugins to the asset map (see `ASSET_MAP`) // Convenience macro to add plugins to the asset map (see `ASSET_MAP`)
//
// Plugins are taken from:
//
// - `zellij-utils/assets/plugins`: When building in release mode OR when the
// `plugins_from_target` feature IS NOT set
// - `zellij-utils/../target/wasm32-wasi/debug`: When building in debug mode AND the
// `plugins_from_target` feature IS set
macro_rules! add_plugin { macro_rules! add_plugin {
($assets:expr, $plugin:literal) => { ($assets:expr, $plugin:literal) => {
$assets.insert( $assets.insert(
PathBuf::from("plugins").join($plugin), PathBuf::from("plugins").join($plugin),
#[cfg(any(not(feature = "plugins_from_target"), not(debug_assertions)))]
include_bytes!(concat!( include_bytes!(concat!(
env!("CARGO_MANIFEST_DIR"), env!("CARGO_MANIFEST_DIR"),
"/assets/plugins/", "/assets/plugins/",
$plugin $plugin
)) ))
.to_vec(), .to_vec(),
#[cfg(all(feature = "plugins_from_target", debug_assertions))]
include_bytes!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../target/wasm32-wasi/debug/",
$plugin
))
.to_vec(),
); );
}; };
} }