SwiftShell

A Swift OS X Framework for running shell commands.

Introduction

SwiftShell is an OS X Framework for running shell commands from Swift. It also aims to make writing shell scripts easier for those who already know Swift, and easier to read for everyone. Version 1 has been out for a while and has been updated for Swift 2.0. Version 2 is under development to take advantage of the new stuff in Swift 2 and fix several shortcomings.

SwiftShell 1

SwiftShell 1 uses the |> operator as a mix between pipe forward from functional programming and | from shell commands. So commands can be piped together:

run("echo piped to the next command") |> run("wc -w") |>> standardoutput

And combined with functions:

var i = 1
standardinput.lines() |> map {line in "line \(i++): \(line)"} |>> standardoutput

Problems

Although I am quite pleased with how the project turned out, it has some shortcomings:

  • It was written before Swift 2.0, so it doesn’t have any of the new goodies like error handling and protocol extensions.
  • There are far too many global variables.
  • It relies too much on prior knowledge of bash shell scripting. Like the use of $ for running shell commands and returning the output as a String.
  • Getting standard error from a command as opposed to standard output requires some nasty bash redirection trickery.
  • There’s no way to run a command asynchronously.
  • ...or check if it completed successfully.
  • Using the |> operator requires freestanding curried functions and does not play well with the Swift 2.0 standard library, where pretty much everything is a method on a protocol or type.

SwiftShell 2

Version 2 of SwiftShell seeks to fix these problems and be more in the spirit of Swift 2, which I believe to be error handling, method chaining and being explicit.

Context

A problem with shell scripting and SwiftShell 1 is that the information about the environment in which the script is running is spread over many global variables. All of that is now gathered here:

public protocol ShellContextType {
    var encoding: NSStringEncoding {get set}
    var env: [String: String] {get set}
    var stdin: ReadableStream {get set}
    var stdout: WriteableStream {get set}
    var stderror: WriteableStream {get set}
    var currentdirectory: String {get set}
}

There’s just one global value, main.. It implements this protocol and also holds the path to the script, and its arguments. All properties are mutable so you can set e.g. main.stdout to a file or your own stream . Very useful for testing.

There is also a ShellContext struct implementing the protocol, so you can have several contexts simultaneously. This allows you to set up the environment for a group of shell commands without affecting the environment for the entire script.

Commands

Everything that implements ShellContextType can also run shell commands. There are 3 different ways of doing that, the first 2 have identical counterparts in bash shell scripting.

All the different methods for running commands have global versions, which just forward the call to ‘main’. This is just so you don’t have to write main. every time you want to run a command.

runAndPrint

The same as a one line shell command in a shell script.

try runAndPrint(bash: "cmd1 arg | cmd2 arg") 

This will run the command and print any resulting error output and standard output to the context’s stderror and stdout, and if the command returns with a non-zero exit code it will throw a ShellError.

The name runAndPrint may seem a bit cumbersome, but it explains exactly what it does. SwiftShell never prints anything without explicitly being told to.

run

Similar to $(cmd) in bash, this just returns the output from the command as a string:

print("Today's date in UTC is " + run("date", "-u"))

runAsync

Start running a command and return with an AsyncShellTask, before the command has finished.

public struct AsyncShellTask {
    let stdout: ReadableStream
    let stderror: ReadableStream

    func finish() throws -> AsyncShellTask
}

You can then process standard output and standard error further, and optionally wait until it’s finished and handle any errors:

let command = runAsync("cmd", "-n", 245)
// do something with command.stderror and command.stdout
do {
    try command.finish()
} catch {
    // deal with errors. or not.
}

Note that if you read all of command.stderror or command.stdout it will wait for the command to finish running. You can still call finish() to check for errors.

Command parameters

The various run* functions take 2 different types of parameters:

(executable: String, _ args: Any …)

If the path to the executable is without any /, SwiftShell will try to find the full path using the which shell command.

The array of arguments can contain any type, since everything is convertible to strings in Swift. If it contains any arrays it will be flattened so only the elements will be used, not the arrays themselves.

run("echo", "We", "are", 4, "arguments")
// echo We are 4 arguments

let array = ["But", "we", "are"]
run("echo", array, array.count + 2, "arguments")
// echo But we are 5 arguments

(bash bashcommand: String)

These are the commands you normally use in the Terminal. You can use pipes and redirection and all that good stuff. Support for other shell interpreters can easily be added.

Standard input

To use something else than main.stdin as standard input for a command you can customise a context:

let context = ShellContext()
context.stdin = open("file.txt")

Streams can also run commands using themselves as standard input:

runAsync("cmd1").stdout.run("cmd2") 

Laziness

An important aspect of shell commands is that when you pipe the output of one command into another command they both run simultaneously. So if you’re filtering the lines of a large text file you don’t need to keep the entire file in memory at once:

cat largefile.txt | egrep "complex regexp"

SwiftShell achieves the same thing using runAsync and lazy sequences:

let interestinglines = runAsync("cat", "largefile.txt").stdout.lines
    .filter { line in /* much nicer Swift code */ } .array

The Swift version is more explicit, which I like. It specifically says it is taking standard output, splitting it into lines, and filtering on each line.

Note that if you don’t put the .array at the end or otherwise consume the sequence right away the entire file may still be kept in memory at once because the command will keep writing the contents to stdout but nothing will be reading from it.

Challenges

Most of the code in SwiftShell is pretty simple, as NSTask and NSFileHandle does all the heavy lifting. The main problem has been in designing the API, and (as always with me) naming things.

Run, command, run!

Naming things well is often the hardest part of creating a framework, at least for me. Take the previous name for in-line commands, $(cmd arg1 arg2) in bash and $("cmd arg1 arg2") in SwiftShell 1; it only makes sense if you’re already familiar with bash, if that. So I wanted a more descriptive name, which you would think shouldn’t be too hard since anything is more descriptive than $. But coming up with the new and fairly obvious name “run” took a very long time. Once again simplicity proved to be surprisingly elusive.

Loosing context

When creating a custom context and running commands in a method chain, that context is lost as soon as you use a method defined outside of SwiftShell:

let mycontext = ShellContext() 
...
mycontext.runAsync("cmd").stdout.lines
    .filter { ... }.join("") 
    .run("cmd2") 

The 2nd command will be run using the “main” context because the filter and join methods don’t know what a context is. I think if someone writes the above code they will want to run both commands using the “mycontext” context. You could add a "withContext(context: ShellContextType) -> Streamable" method to Streamable to regain the context, but either way it’s not very intuitive.

Standard Library going its own way

Swift has 2 built-in functions which use standard input and standard output; readLine and print. These understandably do not adhere to any customisations made to main.stdin or main.stdout. So if anyone uses print instead of main.stdout.write (and who can blame them) the text may not end up where intended. Hopefully this can be fixed by using freopen in the getter for main.stdout.

Portability and future proofing

Currently all frameworks, including the SwiftShell framework itself, must be located in either “~/Library/Frameworks”, “/Library/Frameworks” or a folder mentioned in the $SWIFTSHELL_FRAMEWORK_PATH environment variable to be able to be imported in a SwiftShell script. It would be nice (or rather: Awesome!) to be able to specify the location and version number of a framework and have it automatically downloaded and built:

#!/usr/bin/env swiftshell --swiftv 2.0

/* Carthage
    github "nori0620/SwiftFilePath" ~> 0.0.5
*/

import SwiftShell
import SwiftFilePath 

Include the Swift version in the first line and all the information needed to run a script will be contained in that script.

Conclusion

So that is SwiftShell. I hope it can make automation on the Mac simpler and safer.

For future updates about the project see http://blog.nottoobadsoftware.com/category/swiftshell/ , and of course the repository at https://github.com/kareman/SwiftShell .