MattDiephouse

Reviewer of PRs.

Maintainer for Carthage, ReactiveCocoa, ReactiveSwift, PersistDB.

Ex-GitHub, ex-Apple.

I'm passionate about writing great software.

Most Recent

  1. » Types as Proof
  2. » Value-Oriented Programming
  3. » Eliminating Impossible States with Never
  4. » Better, Faster, Cheaper
  5. » When Not to Use an Enum

All Articles

Types as Proof

08 Nov 2018

The Swift standard library has a collection called Repeated of an Element that’s repeated count times. Here’s a copy of its basic interface:

public struct Repeated<Element> {
  public let count: Int
  public let repeatedValue: Element

  internal init(_repeating repeatedValue: Element, count: Int) {
    precondition(count >= 0, "Repetition count should be non-negative")
    self.count = count
    self.repeatedValue = repeatedValue
  }
}

// Why a function instead of making a public `init`? I have no idea. 🤷🏻‍♂️
public func repeatElement<T>(_ element: T, count n: Int) -> Repeated<T> {
  return Repeated(_repeating: element, count: n)
}

You can’t repeat something a negative number of times; that doesn’t make any sense. So if you try to create a Repeated with a negative count, it will helpfully assert with a precondition.

Assertions like these can be very helpful. When assumptions are violated, unexpected things will happen: the program might exhibit incorrect, but hard to notice, behavior; or it might crash or deadlock without any clues about the cause.

Assertions help by (1) identifying which assumption was violated (Repetition count should be non-negative) and (2) isolating where the mistake occurred. Asserting early and often can be especially helpful: knowing the time and location of a mistake can be the most helpful information, but also be the most costly.

But no matter how many assertions you add, no matter how specific, no matter how helpful, you’re still left with runtime errors. You’ve made assumptions, and the inevitable mistakes can be hard to catch—even with assertions. Instead of assuming, let’s show our work and prove our logic.

Proof is provided via types.

In the case of Repeated, that’s easy. Negative numbers aren’t valid, so we can use a UInt to restrict the input to values that aren’t negative. Then the compiler can check your work and provide a compiler error if you’ve made a mistake:

error: negative integer '-5' overflows when stored into unsigned type 'UInt'
dump(repeatElement("A", count: -5))
                                ^

The Swift documentation recommends that you not use UInt this way. That’s because UInt isn’t an Int that’s limited to non-negative numbers: the upper half of UInt’s values can’t be represented by an Int. That makes conversions failable in both directions, complicating conversions between the two.

We could design our own type instead, providing a failable initializer that takes an Int and returns nil if the number is negative:

struct NonNegativeInt {
  var value: Int

  init?(_ int: Int) {
    guard int >= 0 else { return nil }
    value = int
  }
}

That makes it easy to convert back to an Int—every NonNegativeInt can be represented as an Int—reducing some of the friction.

Instead of the precondition from Repeated, NonNegativeInt has a failable init. Repeated could also have returned nil instead of using a precondition, but the assertion works better. Return nil when you don’t want to assume. Repeated should assume that you’d give it a non-negative count. NonNegativeInt shouldn’t assume that an incoming Int is non-negative.

This works well in practice because that Int comes from somewhere. It’s best to catch invalid values at the edges of your program, where they originate, and not at the point of use. If this number came from user input, we’d want to provide an error when the user tried to input a negative number. If it came from a JSON API, we’d want to fail during decoding because the server response wasn’t valid.

But I would make an ExpressibleByIntegerLiteral conformance use a precondition. You should assume that a programmer won’t use a negative number when assigning a literal (and they should have some coverage that will easily provide an error if they do).

extension NonNegativeInt: ExpressibleByIntegerLiteral {
    init(integerLiteral value: Int) {
        precondition(value >= 0, "NonNegativeInt must not be negative")
        self.init(value)!
    }
}

That’s not to say that you should model numbers this way (but I’m not saying that you shouldn’t either). This is meant as a familiar example to demonstrate a broader principle: preconditions can be replaced by types.

To apply this generally, it’s helpful to understand the Curry-Howard correspondence. Briefly stated, a correspondence exists between computer programs and logic: types correspond to propositions; functions to inferences; values to proofs; product types to conjunctions; sum types to disjunctions. You can use this correspondence to extend your reasoning from one to the other.

In propositional logic, you might say P → Q, meaning “if P then Q” (an inference with premise P and conclusion Q). If you know P (a proposition), then you can use P → Q to know Q (also a proposition). The process of applying P to P → Q to obtain Q is a proof.

According to the Curry-Howard correspondence, P → Q corresponds to a function (P) -> Q. If you have an instance of P, you can use (P) -> Q to get an instance of Q. You need a value of type P to execute the function, just as you need to know that P is true to make use of P → Q.

There’s an important aspect of this that I’ve only recently understood, thanks to The Little Typer: while types correspond to propositions, values correspond to proofs. If you have a value of a given type, you’ve proven the corresponding proposition. If a type is uninhabited, then it can never be proven. (Never → Q will never prove Q and (Never) -> Q can never be used to get an instance of Q.)

Assertions are assumptions because they’re not stated as premises. Repeated says Element ∧ Int → Repeated (if you have an Element and an Int, then you can get a Repeated). But that’s not correct: You can’t get a Repeated from any Int; it’s assumed that Int will be non-negative. Element ∧ UInt → Repeated fixes the premise, removing the need to assume.

Assertions will alert us when assumptions are invalid. But instead of assuming, we can use a type to create a premise that will let us prove our logic. Compilers can use this to catch our mistakes, eliminating runtime errors.

It’s not always pragmatic to do this, as we’ve seen with UInt. But we have a general principle. Let’s consider a more practical application of this idea.

Consider an iOS app with an in-app purchase to unlock additional features, such as in-app search. Ordinarily you might represent that using a Bool:

// Returns `false` if the in-app purchase doesn't exist or is invalid
func hasPremiumFeatures() -> Bool { … }

Before you launch any premium features, you need to remember to check that flag. Unfortunately, forgetting is easy—especially when you consider the many ways a feature might be launched (buttons, deep links, shortcuts, Siri actions, etc.). If you mistakenly allow basic users to access premium features via any of these methods, then you’ve undermined your business model.

You can add an assertion to catch these mistakes. But now you’ve replaced errant behavior with a crash—one that might not tell you the source of the bug.

final class PremiumViewController: UIViewController {
  init(user: User) {
    precondition(hasPremiumFeatures())
    …
  }
}

Instead, we can replace this boolean flag with a type that represents the presence of premium features. This type is proof of the in-app purchase. It doesn’t need to carry any information; it only needs to limit your ability to create a value of that type.

struct HasPremiumFeatures {
  // Returns `nil` if the in-app purchase doesn't exist or is invalid
  // You might want to pass in data here. Or maybe you want to decode a JSON
  // model but replace a boolean property with a type like this. If its `init`
  // is marked `private` so that it can only be created via JSON decoding, then
  // you have a way to provide proof.
  init?() { … }
}

Now your view controller can list HasPremiumFeatures as an input. It doesn’t need to do anything with it. But requiring a HasPremiumFeatures adds a premise that states that you can only create this view controller if you have premium features.

final class PremiumViewController: UIViewController {
  // Add a property to ensure that any added `init`s also require this
  let hasPremiumFeatures: HasPremiumFeatures

  init(hasPremiumFeatures: HasPremiumFeatures) {
    self.hasPremiumFeatures = hasPremiumFeatures
    …
  }
}

This should feel somewhat familiar: having a T? and another type that requires a T is a common occurrence in languages like Swift. This is exactly why Optional types are important: requiring a non-null T replaces runtime errors with compile-time errors by replacing an assumption with proof. But we can use this more generally to move other assumptions into the type system.

Many premises can’t be expressed with types in Swift because the type system is limited.1 But there’s a lot that you can prove in Swift this way—even it it's just a single boolean like we've done here. Next time you write a precondition, stop and consider whether you can use a type instead.


  1. e.g. if you support multiple users, all of whom may or may not be premium, you can’t create a type to prove that a specific user has premium features. Once you’ve created a HasPremiumFeatures value, you could use it for any user. Adding a property to HasPremiumFeatures won’t help at compile time, because it’s not part of the type. But dependently typed languages like Idris let you create a type that includes a value, which would let you prove that. 

Value-Oriented Programming

29 Aug 2018

At WWDC 2015, in a very influential session titled Protocol-Oriented Programming in Swift, Dave Abrahams explained how Swift’s protocols can be used to overcome some shortcomings of classes. He suggested this rule: “Don’t start with a class. Start with a protocol”.

To illustrate the point, Dave described a protocol-oriented approach to a primitive drawing app. The example worked from a few of primitive shapes:

protocol Drawable {}

struct Polygon: Drawable {
  var corners: [CGPoint] = []
}

struct Circle: Drawable {
  var center: CGPoint
  var radius: CGFloat
}

struct Diagram: Drawable {
  var elements: [Drawable] = []
}

These are value types. That eliminates many of the problems of an object-oriented approach:

  1. Instances aren’t shared implicitly

    The reference semantics of objects add complexity when passing objects around. Changing a property of an object in one place can affect other code that has access to that object. Concurrency requires locking, which adds tons of complexity.

  2. No problems from inheritance

    Reusing code via inheritance is fragile. Inheritance also couples interfaces to implementations, which makes reuse more difficult. This is its own topic, but even OO programmers will tell you to prefer “composition over inheritance”.

  3. No imprecise type relationships

    With subclasses, it’s difficult to precisely identify types. e.g. with NSObject.isEqual(), you must be careful to only compare against compatible types. Protocols work with generics to precisely identify types.

To handle the actual drawing, a Renderer protocol was added that describes the primitive drawing operations:

protocol Renderer {
  func move(to p: CGPoint)
  func line(to p: CGPoint)
  func arc(at center: CGPoint, radius: CGFloat, startAngle: CGFloat, endAngle: CGFloat)
}

Each type could then draw with a Renderer.

protocol Drawable {
  func draw(_ renderer: Renderer)
}

extension Polygon : Drawable {
  func draw(_ renderer: Renderer) {
    renderer.move(to: corners.last!)
    for p in corners {
      renderer.line(to: p)
    }
  }
}

extension Circle : Drawable {
  func draw(renderer: Renderer) {
    renderer.arc(at: center, radius: radius, startAngle: 0.0, endAngle: twoPi)
  }
}

extension Diagram : Drawable {
  func draw(renderer: Renderer) {
    for f in elements {
      f.draw(renderer)
    }
  }
}

This made it possible to define different renderers that worked easily with the given types. A main selling point was the ability to define a test renderer, which let you verify drawing by comparing strings:

struct TestRenderer : Renderer {
  func move(to p: CGPoint) { print("moveTo(\(p.x), \(p.y))") }
  func line(to p: CGPoint) { print("lineTo(\(p.x), \(p.y))") }
  func arc(at center: CGPoint, radius: CGFloat, startAngle: CGFloat, endAngle: CGFloat) {
      print("arcAt(\(center), radius: \(radius),"
        + " startAngle: \(startAngle), endAngle: \(endAngle))")
  }
}

But you could also easily extend platform-specific types to make them work as renderers:

extension CGContext : Renderer {
  // CGContext already has `move(to: CGPoint)`

  func line(to p: CGPoint) {
    addLine(to: p)
  }

  func arc(at center: CGPoint, radius: CGFloat, startAngle: CGFloat, endAngle: CGFloat) {
    addArc(
      center: center,
      radius: radius,
      startAngle: startAngle,
      endAngle: endAngle,
      clockwise: true
    )
  }
}

Lastly, Dave showed that you can extended the protocol to provide conveniences:

extension Renderer {
  func circle(at center: CGPoint, radius: CGFloat) {
    arc(at: center, radius: radius, startAngle: 0, endAngle: twoPi)
  }
}

I think that approach is pretty compelling. It’s much more testable. It also allows us to interpret the data differently by providing separate renderers. And value types neatly sidestep a number of problems that an object-oriented version would have.

But I think there’s a better way to write this code.

Despite the improvements, logic and side effects are still tightly coupled in the protocol-oriented version. Polygon.draw does 2 things: it converts the polygon into a number of lines and then renders those lines. So when it comes time to test the logic, we need to use TestRenderer—which, despite what the WWDC talk implies, is a mock.

extension Polygon : Drawable {
  func draw(_ renderer: Renderer) {
    renderer.move(to: corners.last!)
    for p in corners {
      renderer.line(to: p)
    }
  }
}

We can separate logic and effects here by turning them into separate steps. Instead of the Renderer protocol, with move, line, and arc, let’s declare value types that represent the underlying operations.

enum Path: Hashable {
  struct Arc: Hashable {
    var center: CGPoint
    var radius: CGFloat
    var startAngle: CGFloat
    var endAngle: CGFloat
  }

  struct Line: Hashable {
    var start: CGPoint
    var end: CGPoint
  }

  // Replacing `arc(at: CGPoint, radius: CGFloat, startAngle: CGFloat, endAngle: CGFloat)`
  case arc(Arc)
  // Replacing `move(to: CGPoint)` and `line(to: CGPoint)`
  case line(Line)
}

Now, instead of calling those methods, Drawables can return a set of Paths that are used to draw them:

protocol Drawable {
  var paths: Set<Path> { get }
}

extension Polygon : Drawable {
  var paths: Set<Path> {
    return Set(zip(corners, corners.dropFirst() + corners.prefix(1))
      .map(Path.Line.init)
      .map(Path.line))
  }
}

extension Circle : Drawable {
  var paths: Set<Path> {
    return [.arc(Path.Arc(center: center, radius: radius, startAngle: 0.0, endAngle: twoPi))]
  }
}

extension Diagram : Drawable {
  var paths: Set<Path> {
    return elements
      .map { $0.paths }
      .reduce(into: Set()) { $0.formUnion($1) }
  }
}

And now CGContext to be extended to draw those paths:

extension CGContext {
    func draw(_ arc: Path.Arc) {
        addArc(
            center: arc.center,
            radius: arc.radius,
            startAngle: arc.startAngle,
            endAngle: arc.endAngle,
            clockwise: true
        )
    }

    func draw(_ line: Path.Line) {
        move(to: line.start)
        addLine(to: line.end)
    }

    func draw(_ paths: Set<Path>) {
        for path in paths {
            switch path {
            case let .arc(arc):
                draw(arc)
            case let .line(line):
                draw(line)
            }
        }
    }
}

And we can add our convenience method for creating circles:

extension Path {
  static func circle(at center: CGPoint, radius: CGFloat) -> Path {
    return .arc(Path.Arc(center: center, radius: radius, startAngle: 0, endAngle: twoPi))
  }
}

This works just the same as before and requires roughly the same amount of code. But we’ve introduced a boundary that lets us separate two parts of the system. That boundary lets us:

  1. Test without a mock

    We don’t need TestRenderer anymore. We can verify that a Drawable will be drawn correctly testing the values return from its paths property. Path is Equatable, so this is a simple test.

    let polygon = Polygon(corners: [(x: 0, y: 0), (x: 6, y: 0), (x: 3, y: 6)])
    let paths: Set<Path> = [
      .line(Line(from: (x: 0, y: 0), to: (x: 6, y: 0))),
      .line(Line(from: (x: 6, y: 0), to: (x: 3, y: 6))),
      .line(Line(from: (x: 3, y: 6), to: (x: 0, y: 0))),
    ]
    XCTAssertEqual(polygon.paths, paths)
    
  2. Insert more steps

    With the value-oriented approach, we can take our Set<Path> and transform it directly. Say you wanted to flip the result horizontally. You calculate the size and then return a new Set<Path> with flipped coordinates.

    In the protocol-oriented approach, it would be somewhat difficult to transform our drawing steps. To flip horizontally, you need to know the final width. Since that width isn’t known ahead of time, you’d need to write a Renderer that (1) saved all the calls to move, line, and arc and then (2) pass it another Render to render the flipped result.

    (This theoretical renderer is creating the same boundary we created with the value-oriented approach. Step 1 corresponds to our .paths method; step 2 corresponds to draw(Set<Paths>).)

  3. Easily inspect data while debugging

    Say you have a complex Diagram that isn’t drawing correctly. You drop into the debugger and find where the Diagram is drawn. How do you find the problem?

    If you’re using the protocol-oriented approach, you’ll need to create a TestRenderer (if it’s available outside your tests) or you’ll need to use a real renderer and actually render somewhere. Inspecting that data will be difficult.

    But if you’re using the value-oriented approach, you only need to call paths to inspect this information. Debuggers can display values much more easily than effects.

The boundary adds another semantic layer, which opens up additional possibilities for testing, transformation, and inspection.

I’ve used this approach on a number of projects and found it immensely helpful. Even with a simple example like the one given here, values have a number of benefits. But those benefits become much more obvious and helpful when working in larger, more complex systems.

If you’d like to see a real world example, check out PersistDB, the Swift persistence library I’ve been working on. The public API presents Querys, Predicates, and Expressions. These are reduced to SQL.Querys, SQL.Predicates, and SQL.Expressions. And each of those is reduced to a SQL, a value representing some actual SQL.

Eliminating Impossible States with Never

18 Jul 2018

Swift has a very useful type called Never in the standard library. It’s defined very simply:

public enum Never {}

That might seem strange. What kind of enum has no cases?

This is an example of an uninhabited type—a type that doesn’t have any valid values. There’s no way to create a Never value. That’s a useful property for proving things in the type system. Uninhabited types let us make things impossible.

When it was introduced in SE-0102, Never was framed as a useful return value for functions. Before then, Swift functions that never returned, like fatalError, were marked with a @noreturn attribute. By declaring fatalError to return Never instead, you’ve proven that it never returns. Since there’s no way to create a Never, there’s no way to return one—so you’ve proven that the function can never return.

public func fatalError(
  _ message: @autoclosure () -> String = String(),
  file: StaticString = #file, line: UInt = #line
) -> Never {
  …
}

This isn’t the only place that uninhabited types like Never are useful.

Types help us by restricting the number of possible values. The clearest example of this may be the Result type:

enum Result<Value, Error> {
  case success(Value)
  case failure(Error)
}

Without Result, asynchronous, failable APIs look like this:

func run(completion: @escaping (Value?, Error?) -> ()) { … }

This API is susceptible to bugs because two impossible states can be represented by the types. There should never be (1) both a value and an error or (2) neither a value nor an error. But since it’s possible to represent those in the types, the run function and its caller can both make that mistake.

run { value, error in
    switch (value, error) {
    case let (value?, nil):
        handleSuccess(value)
    case let (nil, error?):
        handleError(error)
    case (_?, _?):
        fatalError("Can't have both a value and an error")
    case (nil, nil):
        fatalError("Can't have neither a value nor an error")
    }
}

Result eliminates those impossible states. By making the impossible states impossible, potential bugs are eliminated.

run { result in
  // With `Result`, no `fatalError`s are necessary.
  // We've made those impossible states impossible.
  switch result {
  case let .success(value):
    handleSuccess(value)
  case let .failure(error):
    handleError(error)
  }
}

Uninhabited types let us go a step further. We can mark cases of an enum as impossible, essentially erasing them from the type:

// This can't fail because you can't create a `Never` to use
// for a `.failure`
// This is basically equivalent to declaring `value` as an `Int`
let value: Result<Int, Never> = …

switch value {
  case let .success(value):
    …
  // A failure case isn't required because it's impossible
  // This is actual working Swift code
}

// This always fails because you can't create a `Never` to use
// for a `.success`
// This is basically equivalent to declaring `error` as an `Error`
let error: Result<Never, Error> = …

switch error {
  // A success case isn't required because it's impossible
  case let .failure(error):
    …
}

That doesn’t make much sense if you’re just declaring local variables with Result: you could just declare your Int or Error directly. But it becomes important once you start using associatedtypes, generics, or a type that adds effects (e.g. an FRP observable).

Take this Elm-inspired description of a UI, e.g.:

// Something approximating The Elm Architecture
// `View`s have `Message`s that can be sent back to the `Component`
protocol Component {
  associatedtype Message
  mutating func update(_: Message) -> Command?
  nonmutating func render() -> View<Message>
}

// You can create a `Button<String>`,
// but you can't create a `Button<Never>`
// because you couldn't set its action
struct Button<Message> {
  var action: Message
  var title: String
}

// You can create a `View<String>.button`,
// but you can't create a `View<Never>.button`
// because you can't create a `Button<Never>`
enum View<Message> {
  case button(Button<Message>)
  case text(String)
}

Using Never, we can prove that some views aren’t interactive, i.e. that they contain no buttons. A View<Never> can’t contain a Button, or any other type of view with an action, because those views can’t be created. That means we can restrict portions of our view hierarchy to non-interactive views.

Let’s say, for instance, that we’ve created a document viewer. We want to render documents to views, but those views should always be static—users shouldn’t be allowed to create buttons. We can enforce that by rendering documents to a View<Never>:

// Our spec says that documents should never contain interactive
// elements. Returning `View<Never>` here proves this in the type
// system. It's impossible to accidentally return a `Button` now.
func render(_ document: Document) -> View<Never> { … }

We may also want to export documents to HTML. We decide that writing a second renderer would be difficult, but it’s not so bad to convert each View to an HTML type. Ordinarily, you might write this generically over the message type:

func export<Message>(_ view: View<Message>) -> HTML {
  switch view {
  case .button:
    // `fatalError`, like `!`, means you haven't
    // proven something in the type system
    fatalError("Documents can’t contain buttons.")
  case let .text(text):
    return HTML.text(text)
  }
}

Unfortunately, while we know that we’re exporting documents and that documents never contain buttons, we haven’t proven that, so the compiler doesn’t know. We have an impossible state that can be represented in the types, so we have to resort to fatalError.

We can prove that we’ll never try to export a button by using Never again:

func export(_ view: View<Never>) -> HTML {
  switch view {
  case let .text(text):
    return HTML.text(text)
  }
}

By using Never, we can completely get rid of the case .button! The Swift compiler knows that you can never create a View<Never>.button, so you’re not required to include that case in your switch. Now someone can’t accidentally call export on an interactive view.

One problem remains: we’ll want to use our rendered document, a View<Never>, inside a View<Message>. This can be done by using another switch statement like the one above above:

extension View where Message == Never {
  func promote<M>(to _: M.self) -> View<M> {
    switch self {
    case let .text(text)
      return .text(text)
    }
  }
}

We only need to unwrap and rewrap our elements to be in the right View type. And we don’t need to include a case button.

Now we can fully render our UI:

func render(_ document: Document) -> View<Never> { … }

extension View where Message == Never {
  func promote<M>(to _: M.self) -> View<M> { … }
}

func render(_ state: State) -> View<Message> {
  return View.stack([
    render(state.document).promote(to: Message.self),
    .button("Export", Message.export),
  ])
}

Never let us eliminate impossible states:

  1. render(_: Document) can’t return interactive elements.
  2. export(_: View<Never>) doesn’t need to handle interactive documents.
  3. You can’t accidentally call export with an interactive view

This same approach can also be used with a protocol’s associatedtype—fulfilling the protocol, but eliminating impossibilities within specific types that conform to it.

Types are useful because they restrict possible values. Never can help us by eliminating some of those values, restricting the possible values even further. Proving that they’re impossible in the type system gives us confidence in the code that we write.

📣 Swift Compiler Blog

13 Jun 2018

I’m hoping to start making regular contributions to the Swift compiler. There’s a lot to learn, so I want to write about the process. Hopefully this will spread knowledge and document what it looks like to climb the learning curve.

Since I want to write about this regularly, and because I expect it to be more casual, probably less accurate, and of less long-term interest, I’ve decided not to put this on my main blog. Instead, it will be its own thing.

This Swift compiler blog will live here. If you’d like to use RSS, there’s a feed available. I’ll also be tweeting about this on Twitter.

Better, Faster, Cheaper

31 May 2018

Good, Fast, Cheap. Pick two. –unknown

It’s a familiar adage to most developers. Software will be buggy, lack features, or be expensive to develop.

But this is just a project management tool.

Known as the project management triangle (or a few other names), this framework gives project managers a way to approach quality, scope, and cost. Each attribute corresponds to something a project manager controls.

  • Good: When can it ship?
  • Fast: What’s the scope?
  • Cheap: What’s the budget?

Adjusting the ship date, scope, or budget will necessarily affect at least one of the others. This is incredibly useful. Adjustments are often necessary; it helps to know what will be needed to compensate.

But too often this is used to accept the current quality, scope, and cost of software. The project management triangle doesn’t actually say anything about quality, scope, or cost—it’s about the relationship between them. It says that you can always trade between quality, scope, and cost.

Engineers work from a different set of constraints.

From an engineering perspective, quality, scope, and cost are byproducts. Engineers fix bugs and write features. Based on the engineering output, project managers allocate quality, scope, and cost. Those project decisions are byproducts of the engineering work.

Software can be better, faster, and cheaper—if engineers can write it more quickly and/or with fewer defects. If development improves substantially in speed or quality, then project managers can reallocate the savings to improve quality, scope, and/or cost. In other words, software development must improve if we want improved software.

Thankfully, software development improves constantly. And while individual changes are usually not substantial, the cumulative effect is. It’s much easier to develop most applications today than it was even a few years ago. But because software quality is acceptable from the project management perspective, the savings are most often used to increase scope and decrease cost.

Still, if you reflect on today’s best practices, you can hopefully see how they’ve improved development by making it faster or reducing defects.

  • Automated testing makes code easier to refactor, prevents defects, and saves time by reducing the need for manual testing (by engineers).

  • Type safety makes code easier to refactor, prevents defects, and helps engineers understand the system.

  • Iterative development approaches help engineers prioritize the most important work and eliminate time wasted on unnecessary requirements.

  • Functional architectures like React or Elm increase type safety, testability, and reproducibility.

  • Libraries like Scientist provide confidence for large changes by proving the results before they’re visible to users.

  • Faster hardware can lower compile times, reducing wait times and context switching during engineering work.

  • Platform improvements hopefully increase productivity by adding new APIs and fixing old bugs.

  • Screencasts provide valuable insights into improved techniques, which should improve the development process.

There are many more improvements—far too many to list.

Software is improved by improving the training, tooling, or processes that drive its development. Better frameworks, architectures, compilers, debuggers, package managers, data modeling, etc. work together to improve software overall. That’s the only way out of the mess of modern software. Training, tooling, and processes must improve to make development faster with fewer defects. Then the savings can be redistributed to improve quality, scope, and cost.

That doesn’t mean that software can only be improved by people who work full-time on training, tooling, or processes.

We all have a responsibility to push our individual teams forward. Don’t accept your current quality, scope, and cost. Question how you can change your development to produce features more quickly and with fewer defects.

  • Invest in additional training: screencasts, conferences, books. That might mean allocating budget or time. It’s hard to improve your software development without learning about possible improvements.

  • Establish an architecture for your application that makes adding features easier and writing bugs harder. This probably means adding more distinct and well-defined horizontal or vertical layers.

  • Create a design language system. Reusing UI means writing less code and debugging less code. It also improves communication between design and engineering.

  • Automate where possible. Always question whether a computer can do the work you’re doing manually. Building the app for release yourself? Automate it. Deploying manually? Automate it. Wasting time formatting your code? Automate it.

  • Use tools that catch bugs. This is especially true for automated tools. Thread debuggers, memory debuggers, static analyzers, coverage tools, hotspot analysis.

  • Find ways to make debugging easier. Maybe you need to add debugDescriptions to some of your Swift classes. Maybe you need to refactor to make code easier to understand. Maybe you need better testing environments or test data.

  • Make testing easy. Ideally it’s easier to verify code with a unit test than by manually running the software. This usually means providing good fixtures or data generation, adding custom assertions, and/or refactoring code to separate logic from side effects.

Most importantly, approach development as something that can improve. Listen to feedback, reflect on your performance, and question how you can do better. Don’t be satisfied with the current state of the industry or of your team.

We all need to continually strive for better ways to write software. That’s the only way we’ll get better software.

When Not to Use an Enum

07 Dec 2017

Earlier this week, John Sundell wrote a nice article about building an enum-based analytics system in Swift. He included many fine suggestions, but I believe he’s wrong about one point: that enums are the right choice.

John’s design met all of his goals, but left some gnarly computed properties that switch over self:

extension AnalyticsEvent {
    var name: String {
        switch self {
        case .loginScreenViewed, .loginAttempted,
             .loginSucceeded, .messageListViewed:
            return String(describing: self)
        case .loginFailed:
            return "loginFailed"
        case .messageSelected:
            return "messageSelected"
        case .messageDeleted:
            return "messageDeleted"
        }
    }

    var metadata: [String : String] { … }
}

With just 7 event types, these properties are already getting large. Most apps have many more analytics events, so this would quickly become unwieldy in a real world app.

This also exhibits poor locality. If you want to look up the name and the metadata for an event, or change them for some reason, you need to search through two switch statements. It’d be better if we could specify the name and the metadata together.

We can achieve John’s goals in another way that improves on both points.

Examine the interface for AnalyticsEvent:

enum AnalyticsEvent {
    case loginFailed(reason: LoginFailureReason)
    case loginSucceeded
    …

    var name: String { … }
    var metadata: [String : String] { … }
}

This interface is used for 3 things:

  1. To create an event
  2. To get the name for the event
  3. To get any metadata for the event

Notably missing from this list is any switching. This violates my design rule for enums: Enums are for switching.1

If you’re not switching over an enum, then you’re using it for its properties. We have a better tool for value types with properties: structs.2 So let’s try that:

struct AnalyticsEvent {
    var name: String
    var metadata: [String : String]

    private init(name: String, metadata: [String: String] = [:]) {
        self.name = name
        self.metadata = metadata
    }

    static func loginFailed(reason: LoginFailureReason) -> AnalyticsEvent {
        return AnalyticsEvent(
            name: "loginFailed"
            metadata: ["reason" : String(describing: reason)]
        )
    }
    static let loginSucceeded = AnalyticsEvent(name: "loginSucceeded")
    …
}

The struct version is identical in usage to John’s enum version. Events can be created by name and asked for their name and metadata. By using a private init, arbitrary events can’t be created.

// Enum case with associated values?
// Or struct with a static function?
// You can't tell from this line of code. :)
analytics.log(.messageDeleted(index: index, read: message.read))

But where the enum required two large computed properties, the struct has a simple init and its data is localized to a small func or let. (And as of today, it’s much easier to make the struct version Equatable/Hashable.)

The boilerplate to create the loginFailed function above might seem a little annoying or verbose, but it’s no more verbose than adding cases to two different switch statements.

Next time you reach for an enum, ask yourself whether you’re going to switch over it. If you're not, you're probably better off with a struct instead.


  1. Really I should say pattern matching here to include if case let … = …

  2. Using a sum type as a product type, or vice-versa, is generally not a good idea. 

A Smarter Package Manager

06 Sep 2017

Software evolves. When incompatible changes are made to the libraries you depend on, finding a set of compatible versions (resolving dependencies) can be very difficult.

That’s why we use package managers.

To resolve dependencies, a package manager needs a list of available versions and a method for determining compatibility. It’s become standard practice to determine compatibility with semantic versioning.

Semantic versioning codifies rules about version numbers so that they describe compatibility. Each version is tagged major.minor.patch. Breaking (incompatible) changes increment the major version, compatible additions increment minor, and bug fixes increment patch. The version numbers carry meaning, hence semantic versioning.

If libraries list compatible versions for their dependencies, then a package manager can find a compatible set of all dependencies automatically. (Or not find, if no compatible set exists for those dependencies and versions.)

This is a dramatic improvement over manual dependency resolution, but it’s far from perfect:

  1. Semantic versioning doesn’t specify whether you’re declaring source compatibility or binary compatibility. For many languages, that’s okay. But it’s a real problem for languages with type inference and overloading like Swift’s.1

  2. It can be hard to know what changes are actually breaking. (This is true regardless of whether you’re aiming for source compatibility or binary compatibility.)

  3. Even the smallest breaking change requires a new major version—even if it’s an API that no one uses. And when you release a new major version, then any libraries that depend on yours need to explicitly upgrade to the new version.

  4. The pain of new major versions encourages large releases. Large releases require more management from maintainers and make upgrading harder for users.

  5. It’s easy for a library to accidentally depend on a newer version of a dependency. If A is declared to be compatible with B 1.0, but developed against B 1.1, it’s easy to pick up an addition from 1.1 without updating the version requirement.

These are fallout from using version numbers as a proxy for compatibility. What really matters is whether the compiler considers two versions compatible.

Knowing what the compiler will find to be compatible can be very nuanced, requiring intimate knowledge of the compiler. If you want to always get the right answer, the compiler needs to be used to make the decision.2 Otherwise maintainers will make mistakes and the whole scheme falls down.3

Elm’s elm-package does this. Version numbers are determined automatically based on changes to the API. This eliminates the potential for mistakes, but doesn’t improve granularity: a new major release still isn’t compatible if the portions of the API that you’re using don’t change.

The reality is that version numbers don’t communicate much about compatibility. If we want to do better, we need to actually calculate compatibility instead of choosing such a conservative approximation. But we can do that! Compilers already know how.

A smarter package manager would require:

  1. A list of the public APIs in a library at any version.

  2. The actual APIs that were used from each dependency.

  3. A minimum version for each dependency. This would be used to build and calculate the APIs that were used from each dependency. But it would also serve as a way to guarantee the availability of bugfixes in dependencies.

By capturing the information that the compiler uses when building, compatibility can be determined. If one of the used APIs is no longer there, the version is incompatible. But a breaking change to an unused API would be compatible! 4

This scheme would improve on the problems of semantic versioning:

  1. Maintainers could release breaking changes as needed without a major release.

    Only clients who depended on that API would need to update. This should let breaking changes to be released more frequently with less disruption.

  2. Updating could be done in smaller steps.

    Instead of updating to the next major version, which had potentially grouped together a large number of breaking changes, a package manager could update one breaking (for you) release at a time.

  3. The package manager could guarantee compatibility.

    Maintainers wouldn’t need to worry about mistakes. The package manager could generate—and thus verify—all the compatibility requirements.

  4. Version numbers could be repurposed.

    They could be designed for humans instead of machines. We might stick with a familiar major.minor.patch template, or maybe we’d just a year.month.date.patch to communicate the passage of time, or something else. But we could choose whatever was most helpful for us.

Semantic versioning knows nothing about the content it versions. This leaves humans to do the machine’s work when computers should be doing ours. We’re not good at this sort of work, which leads to mistakes and painful workflows.

Let’s build tools that are designed for humans—that are easy to use and prevent mistakes. Smart tools make developers more productive and development more accessible. Our compilers and IDEs are getting smarter. I think it’s time for our package managers to get smarter too.


  1. e.g., changing class A { } to typealias A = B; class B { } is source-compatible but binary-incompatible in Swift. 

  2. As the Perl community says, “Only perl can parse Perl”. 

  3. I’ve made a few of these mistakes myself over the past couple years. Even if you’re looking for them, they can be hard to catch. 

  4. This is most likely an oversimplification. But I’m pretty sure the idea itself is valid. 

Type-Driven Development with Swift

23 May 2017

I recently started reading Type-Driven Development with Idris, by Edwin Brady. It’s been enlightening to learn about Idris’s dependent types; I highly recommend it. I've found the book to be very approachable.

In Type-Driven Development, as presented in the book, you focus on the types of your functions before implementing them. After the types compile, you go back and write the actual code. This gives you a proven foundation for your code. (Proven because types are proofs.)

Idris does this with holes, which fill in for an expression that you haven't written yet.

module Main

main : IO ()
main = putStrLn ?greeting

?greeting is a stand-in for the value that’s going to be printed. Idris will report the type of the hole. In this case, ?greeting is a String.

Swift doesn't have holes (maybe it should??), but I’ve found myself using a very similar technique with fatalError. I can figure out my types and then fill in the implementations. This is particularly useful when I’m unsure how to represent a concept in the type system.

enum Result<T, E: Error> {
    case success(T)
    case error(E)

    func andThen<U>(_ action: (T) -> Result<U, E>) -> Result<U, E> {
      // I can work out the types before I write this
      fatalError()
    }
}

let x = Result  // Oops! `T` and `E` can't be inferred.
    .success(3) // I can figure this out before implementing `andThen`.
    .andThen { value in
        return .success("foo")
    }

This works at the top level of your program. But it can also work as you implement a given function:

extension Result {
    func andThen<U>(_ action: (T) -> Result<U, E>) -> Result<U, E> {
        switch self {
        case let .success(value):
            return action(value)
        case .failure:
            // I can come back to this after finishing the `.success` case
            fatalError()
        }
    }
}

This has been a very handy tool for me in the past month. Type inference can be hard to predict, and this lets me work through an API without spending the time implement it.

Logic Programming in Swift

21 Dec 2016

Today I open-sourced Logician, a Swift logic programming library. Since I suspect most people don’t know much about logic programming, I wanted to explain a bit about logic programming, what it’s good for, and how it works.

Why Logic Programming?

Most programs tell a computer exactly what to do, step by step:

Set sum to 0; set i to 0; check that i is less than array.count; add array[i] to sum; repeat.

This is imperative programming.

Declarative programming tells the computer what something is without describing all the individual steps:

sum is the reduction of array that starts with 0 and adds each element to the total.

Logic programming is a form of declarative programming that describes the solution instead of the calculation. Based on that description, the computer solves the problem. Describing a solution to a problem can be a far easier than devising a method to solve it, which is why logic programming can be appealing.

Example Applications

I find most logic programming examples to be uninteresting and uninspiring. Usually they mention artificial intelligence and involve some sort of fact database. Since I can’t imagine ever needing to solve a problem like that, I had never been very interested in logic programming.

But there are other problems that can be solved with logic programming. Here are a few examples that I find more interesting and the constraints that describe the solution:

  • N Queens (Logician implementation)

    1. Each queen is on a different row
    2. Each queen is on a different column
    3. Each queen is on a different forward diagonal
    4. Each queen is on a different backward diagonal
  • Sudoku (Logician implementation)

    1. All numbers are 1 through 9
    2. Each row has exactly one of each number
    3. Each column has exactly one of each number
    4. Each subgrid has exactly one of each number
  • Map Coloring

    1. Each region is one of n colors
    2. Each region is different from all of its neighbors
  • Dependency Resolver

    1. Each project is one of a set number of versions
    2. Each dependency can have one of a set of pre-defined constraints

Most of us don’t solve these sorts of problems in our day to day work, but these problems do need to be solved occasionally. And while I can envision solving them with logic programming, solving them imperatively or functionally would be quite a bit of work.

How Logic Programming Works

In logic programming, constraints establish relationships between variables (unknowns) and values (knowns). To solve a problem, you express the constraints and ask the solver for valid solutions.

This is not unlike algebra: x * 2 + y * 2 == 16 can be thought of as a constraint. Solving the problem establishes a value for all variables. In this case, valid solutions include (x = 1, y = 7), (x = 2, y = 6), etc.

Constraints are stored in a data structure that we'll call State. Finding solutions requires two different operations on the state:

  • Reification: Resolving the non-variable value of a variable

    If a == b and b == 7, the reified value of a is 7, even though it might be stored as a == b. If a == b is the only constraint, then a and b have no reified value.

  • Unification: Combining two sets of constraints, erroring if they're inconsistent

    a == 5 and b == 6 unify, but a == 5 and a == 7 are inconsistent—a can't have two different values. The state signals that a constraint has been violated by throwing or returning an error.

There are a number of ways that you can implement unification and reification. One simple way is to use a dictionary of variables to variables/values.

/// An unknown value used in a logic problem.
/// The identity of the object serves as the identity of the variable.
class Variable: Hashable {
    static func ==(lhs: Variable, rhs: Variable) -> Bool {
        return lhs === rhs
    }

    var hashValue: Int {
        return ObjectIdentifier(self).hashValue
    }
}

class State {
    /// An assignment of a value/variable to a variable. Since the
    /// value can be either a variable or a value, an enum is used.
    enum Term {
        case variable(Variable)
        case value(Int)
    }

    /// The assignments of variables/values to variables. This is just
    /// one possible approach.
    var substitutions: [Variable: Term]

    /// Look up the actual value of a variable--following intermediate
    /// variables until the actual value is found.
    func reify(_ variable: Variable) -> Int? {
        switch substitutions[variable] {
            case let .variable(variable)?:
                return reify(variable)
            case let .value(value)?:
                return value
            case .none:
                return nil
        }
    }

    /// Unify two variables, erroring if they have inconsistent values.
    mutating func unify(_ a: Variable, _ b: Variable) throws {
        if let a = reify(a), let b = reify(b), a != b {
            throw Inconsistent
        }
        substitutions[a] = Term.variable(b)
    }

    /// Unify a variable and a value, erroring if the variable has an
    /// existing, inconsistent value.
    mutating func unify(_ a: Variable, _ b: Int) throws {
        if let a = reify(a), a != b {
            throw Inconsistent
        }
        substitutions[a] = Term.value(b)
    }
}

A solver uses unification and reification to find solutions to logic problems. This is a complex subject, with room for lots of optimization. But a simple approach, modeled by μKanren, is to represent constraints with goals—blocks that build an infinite lazy stream.

/// Something that generates values and can be iterated over.
class Generator<Value>: IteratorProtocol {
    func next() -> Value?
}

/// A block that takes a state and produces a generator (stream) of
/// possible states.
typealias Goal = (State) -> Generator<State>

If a goal returns an empty stream, it yields no solutions; if it returns a stream with multiple values, there are multiple viable solutions.

Creating a goal is easy.

func goal(_ block: @escaping (State) throws -> (State)) -> Goal {
    return { state in
        do {
            let state = try block(state)
            return Generator(values: [ state ])
        } catch {
            return Generator()
        }
    }
}

func == (variable: Variable, value: Int) -> Goal {
    // A goal that will return either 0 or 1 values.
    return goal { try $0.unifying(variable, value) }
}

Goals handle the unification, but we still need to reify the solutions. To do so, we use a solve function that introduces a new variable, solves for a goal, and then reifies that variable.

extension Generator {
    /// Apply a function to every value created by the generator,
    /// resulting in a new generator that returns the the new
    /// non-`nil` values.
    func flatMap<New>(_ transform: (Value) -> New?) -> Generator<New>
}

func solve<Value>(_ block: (Variable) -> Goal) -> Generator<Int> {
    let variable = Variable()
    let goal = block(variable)
    let initial = State()
    return goal(initial).flatMap { return $0.reify(variable) }
}

That's all it takes. (But you probably want a few more constraints—at least && and ||.) This is a simplified version of Logician's current solver, but it highlights the core concepts.

Logician has many more features, including different value types (not just Ints), inequality constraints, mapping over variables, constraints on properties of a variable, etc. It’s still very naïve; but I hope that it can develop into a sophisticated logic solver.

Read More

If you’d like to learn more about logic programming, the Logician README has a Learn More section with links to some great resources.

Taking on Technical Debt

23 Jun 2014

The Metaphor

In this metaphor, doing things the quick and dirty way sets us up with a technical debt, which is similar to a financial debt. Like a financial debt, the technical debt incurs interest payments, which come in the form of the extra effort that we have to do in future development.
—Martin Fowler

When programmers talk about technical debt, it's overwhelmingly negative. We talk about accruing technical debt and paying down technical debt. This reflects the reality of software development: most debt accrues unintentionally as bugs are fixed and requirements change.

But there's more to it than that. To continue the metaphor, we talk about the interest from technical debt, but not the capital. We focus entirely on the long-term downsides of unintentional technical debt without considering the short-term opportunities that intentional debt can provide.

Why Technical Debt

The metaphor also explains why it may be sensible to do the quick and dirty approach. Just as a business incurs some debt to take advantage of a market opportunity developers may incur technical debt to hit an important deadline.
—Martin Fowler

Martin Fowler suggests that technical debt can be used to hit deadlines, but I think that's an incomplete picture.

Software development is fundamentally a process of change: either from nothing to something, or from one thing to another. Developing iteratively and incrementally helps manage this process. But often the real difficulty is knowing (1) what the end product should be and (2) how to transform what you have into what you want.

Building Knowledge

If you don't know what to implement, you lack knowledge about the system, problem space, or algorithm. Programming is often the most efficient way to learn; trying to solve a problem will introduce you to all its pieces and challenges.

But I often resist building a solution because I don't know how to build a good solution. I need to be okay with building something that's flawed. Instead of worrying about the good solution, I need to worry about my understanding of the problem.

The difference between that initial version and the one I'm happy with is technical debt that I've taken on. It has let me learn what I need to build a better solution. This leaves me with some debt to pay down, but iterating tends to be faster than designing everything upfront.

Technical debt lets you build your knowledge to arrive at a good solution.

Decomposing Change

As the size of a change increases, the effort required to complete and test it increases dramatically. Build failures and broken pieces multiply the work required to arrive at a working state.

Breaking down a change into smaller pieces helps you develop efficiently. But even within a well-designed system, changes don't always decompose well. New requirements can span multiple components, or even the whole system, leaving you unable break down the change into discrete, well-designed pieces.

By focusing on one part of the system, and adding cruft to the other parts where necessary, you can make small, functional changes. Faking or working around an interface insulates the rest of system, but leaves you with technical debt to clean up.

Technical debt lets you break down work into manageable pieces.

The Power of Capital

Just as a monetary loan can help businesses accelerate their plans, technical debt can speed up the development of new features and software. I believe this is true on both the small and large scale, within a pull request or across a release. It's important to write good code, but don't be afraid to take on technical debt to help you get there.

I've often felt stuck because I didn't know how to write something well. Rather than getting stuck trying to write great code initially, I should be happy to start with bad code and make it into great code. Make it work, make it right, and then make it fast.