tgvashworth

My team maintains a number of increasingly complex CLI tools, mostly written in Bash, though some is Scala in order to share code with other applications.

We’re looking at migrating these to a more testable, modern language.

Python is the de-facto language for this at Twitter, but I also wanted to explore how Rust would fit the bill. These are my notes.

Requirements

What are we already doing with Bash?

• Figure out git root
• Parse args & print help
• Computed default values for arguments
• Shell out to other executables (including jars, pants and other CLI tools) and pipe their output to files
• Handle errors in sub-processes
• Log to stderr
• Error messages
• pwd
• pushd/popd
• Manipulate paths
• URL encode
• Read from stdin
• Open URLs (shell out to open)
• Proxy args onto other tools
• cp
• mkdir -p
• command -v
• arc
• GET/POST URLs (compressed)
• jq or JSON parse
• sleep
• git — has to be CLI to use source_git (Twitter’s git)
• Subcommands

Crates

These crates might be useful:

• Arg-parser — clap
• Serde — serde
• vitiral/path_abs: Absolute serializable path types and associated methods.
• killercup/assert_cli: Test CLI Applications in Rust.
• vitiral/termstyle: create and test the style and formatting of text in your terminal applications
• colin-kiegel/rust-pretty-assertions: Overwrite assert_eq! with a drop-in replacement, adding a colorful diff.
• brson/stdx: The missing batteries of Rust
• vitiral/stdcli: (pre-alpha) rust meta library for cli applications, which contains the following (and more)…
  • dtolnay/indoc: Indented document literals for Rust
  • mitsuhiko/console: A rust console and terminal abstraction
  • mitsuhiko/dialoguer: Rust utility library for nice command line prompts and similar things
  • mitsuhiko/indicatif: A command line progress reporting library for Rust
• Stebalien/tempfile: Temporary file library for rust
• daboross/fern: Simple, efficient logging for Rust

I got lots of this from @vitiral’s great Rust2018 And The Great Cli Awakening post.

Rust has work-in-progress support for Thrift but this may not be compatible with scrooge. Perhaps Scrooge could be extended to provide Rust support. Thrift support isn’t a hard requirement yet.

We won’t actually be pushing Rust too hard: mostly the existing tools wrangle paths and shell out to other executables. For the most part, the code will be very imperative and will handle errors & exit immediately.

Pants currently has no support for rust although parts of its core are being rewritten in the language. There is appetite at Twitter for using Rust, but only a small community who already have experience.

The biggest con of the idea is requiring the GraphQL team to learn Rust when there is Python within the team already, and the risk of reverting this decision in the future as the team changes or if Rust moves to a negative standing on our tech radar.