Adopting Value Types: Networking

4 December 2019

Value types can make code easier to understand, test, and reuse.¹ But it can be difficult to start using value types: they require a shift in thinking, frameworks don’t always cooperate, and sample code often doesn’t illustrate real-world scenarios.

So let’s walk through a concrete, real-world example: developing a client for the lovely Deck of Cards JSON API. This API exposes endpoints to shuffle decks of cards and draw cards from them.²

Each deck of cards has an identifier, which we’ll represent with a Deck type:

/// A value that uniquely identifies a Deck.
struct Deck: Equatable {
  var id: String
}

// The string representation of a deck is its ID.
extension Deck: CustomStringConvertible {
  var description: String {
    return id
  }
}

// A deck can be decoded from a JSON string.
extension Deck: Decodable {
  init(from decoder: Decoder) throws {
    id = try String(from: decoder)
  }
}

A deck is made up of cards:

/// A card in a deck of cards.
struct Card: Decodable, Equatable {
  enum Suit: String, Decodable, Equatable {
    case clubs = "CLUBS"
    case diamonds = "DIAMONDS"
    case hearts = "HEARTS"
    case spades = "SPADES"
  }
  enum Value: String, Decodable, Equatable {
    case one = "1"
    case two = "2"
    case three = "3"
    case four = "4"
    case five = "5"
    case six = "6"
    case seven = "7"
    case eight = "8"
    case nine = "9"
    case ten = "10"
    case jack = "JACK"
    case queen = "QUEEN"
    case king = "KING"
    case ace = "ACE"
  }
  var suit: Suit
  var value: Value
}

We will be working with 2 endpoints:

GET /api/deck/new/shuffle/

Create a new deck, shuffle it, and return its ID.

/// The top-level response from the API that creates a deck.
struct DeckInfo: Decodable, Equatable {
  enum CodingKeys: String, CodingKey {
    case deck = "deck_id"
  }
  var deck: Deck
}

GET /api/deck/{deck id}/draw/?count={count}

Draw and return cards from the top of a deck.

/// Information returned when drawing cards.
struct Draw: Decodable, Equatable {
  var remaining: Int
  var cards: [Card]
}

Let’s also define a few fixtures—example JSON payloads—that we can use for tests:

/// A JSON fixture used in tests. (Implementation omitted for brevity)
struct Fixture {
  var string: String
  var data: Data
}

extension Fixture {
  /// A fixture for the `/deck/new/shuffle/` API.
  static let shuffle: Fixture = …

  /// A fixture for the `/deck/{id}/draw/` API.
  static let draw: Fixture = …
}

With that out of the way, we’re ready to implement the client code for the API.

Clients for JSON APIs often have an interface like this:

class Client {
  /// Shuffle a new deck and return its ID.
  func shuffle(_ completion: @escaping (Result<DeckInfo, Error>) -> Void)

  /// Draw and return cards from the top of a deck.
  func draw(
    count: Int,
    from deck: Deck,
    completion: @escaping (Result<Draw, Error>) -> Void
  )
}

You might use promises, Rx, ReactiveSwift, or Combine instead of a callback, but the idea is roughly the same: a class defines a client for the API and methods correspond to the different endpoints.

Most often, I see implementations that look something like this:

class Client {
  /// Shuffle a new deck and return its ID.
  func shuffle(_ completion: @escaping (Result<DeckInfo, Error>) -> Void) {
    fetch(method: .get, endpoint: "new/shuffle/", completion)
  }

  /// Draw and return cards from the top of a deck.
  func draw(
    count: Int,
    from deck: Deck,
    completion: @escaping (Result<Draw, Error>) -> Void
  ) {
    fetch(
      method: .get,
      endpoint: "\(deck)/draw/",
      parameters: [("count", "\(count)")],
      completion
    )
  }

  /// Fetch any endpoint on the API.
  internal func fetch<Value: Decodable>(
    method: Method,
    endpoint: String, 
    parameters: [(String, String)] = [],
    _ completion: @escaping (Result<Value, Error>) -> Void
  ) {
    let url = Client.BASE_URL.appendingPathComponent(endpoint)
    var components = URLComponents(url: url, resolvingAgainstBaseURL: false)!
    if !parameters.isEmpty {
      components.queryItems = parameters.map { pair in
        URLQueryItem(name: pair.0, value: pair.1)
      }
    }
    var request = URLRequest(url: components.url!)
    request.httpMethod = method.rawValue
    fetch(request, completion)
  }

  /// Fetch any URL that returns a JSON response.
  internal func fetch<Value: Decodable>(
    _ request: URLRequest,
    _ completion: @escaping (Result<Value, Error>) -> Void
  ) {
    URLSession
      .shared
      .dataTask(with: request) { data, response, error in
        if let error = error {
          completion(.failure(error))
        } else {
          completion(.init { try JSONDecoder().decode(Value.self, from: data!) })
        }
      }
      .resume()
  }
}

Each endpoint method calls into a helper method that will call any endpoint. That method calls one that takes any URL request. It’s highly imperative: each method corresponds to a command and is defined with a more general command.

I dislike this style:

It doesn’t offer much abstraction. In order to really understand how the API call is made, I need to understand 3 methods simultaneously.
It requires mocks to test anything.
To share code between methods, we need to add more methods to the call stack.

Let’s write some tests. For each endpoint, we want to verify that (1) we call the endpoint correctly and (2) we parse the response correctly.

To verify that we’ve called the endpoint correctly, we need to capture the URL we’re requesting. To do that, we need to actually call the method. We’ll need to use HTTP mocks for this. A number of open-source libraries provide this functionality, based on URLProtocol, but for now let’s assume a simple version:

/// Support for mocking HTTP requests.
final class HTTPMocks: URLProtocol {
  /// A dictionary from URLs to the results that should be returned from their
  /// requests.
  static var mocked: [URL: Result<Data, Error>]
  /// An array of all incoming URL requests.
  static var sent: [URLRequest]

  /// Start mocking requests.
  class func start()
  /// Stop mocking requests.
  class func stop() 
}

Then we can write tests that call our endpoints and verify that the sent URL requests have the correct values:

func testShuffleMakesCorrectRequest() {
  // Make the request and wait for it to complete
  let expectation = Expectation("request")
  client.shuffle() { _ in
    expectation.fulfill()
  }
  expectation.wait(timeout: 1)

  // Get the request that was sent
  let actual = HTTPMocks.sent.last
  // Make our assertions
  XCTAssertEqual(actual?.httpMethod, "GET")
  XCTAssertEqual(actual?.url, URL(string: "https://deckofcardsapi.com/api/deck/new/shuffle/")!)
}

This test works—it’ll catch failures—but it’s not ideal: there’s a lot of setup for what’s a pretty simple assertion. And if the test fails, we may be stuck debugging the mocking layer—trying to figure out why we didn’t get the value we expected.

Testing that we parse responses correctly is even more complex:

func testShuffleParsesResponseCorrectly() {
  let url = URL(string: "https://deckofcardsapi.com/api/deck/new/shuffle/")!
  HTTPMocks.mocked[url] = .success(Fixture.shuffle.data)

  let expectation = Expectation("request")
  var actual: DeckInfo?
  client.shuffle() { result in
    if case let .success(deck) = result {
      actual = deck
    }
    expectation.fulfill()
  }
  expectation.wait(timeout: 1)

  let expected = DeckInfo(deck: Deck(id: "gy7aqxz5nmpd"))
  XCTAssertEqual(actual, expected)
}

Again, this works as a test. But we have all the same shortcomings as our request test: lots of setup and the complexity of the mocking layer. And this test has an additional source of brittleness: it requires the URL for the request. If the location of the endpoint changes, this test will fail when the thing it’s testing (decoding) hasn’t broken!

Let’s start applying value types.

The code already uses some value types and transforms between them. But that transformation is tightly coupled to the actual network call.

shuffle()

Transform semantic values (none in this case; the deck and count for draw) into an API-specific request. Then call fetch to make that request.
fetch(method: Method, endpoint: String, parameters: [String: String])

Transform an API-specific request into a generic URL request. Then call fetch to make that request.
fetch(_ request: URLRequest)

Make the actual request.

The transformations are what we’re trying to test: that’s where the logic is. But we need to use mocks because we’ve tied those transformations to the actual request. So let’s separate them out.

Right now, our API-specific requests are represented by a tuple of values: the method, endpoint, and parameters. Tuples have some limitations that can be frustrating to work with, so let’s add a new Request type that wraps up the API-specific request used by the intermediate fetch method.

struct Request {
  var method: Method
  var endpoint: String
  var parameters: [(String, String)]
}

class Client {
  /// Shuffle a new deck and return its ID.
  func shuffle(_ completion: @escaping (Result<DeckInfo, Error>) -> Void) {
    let request = Request(
      method: .get,
      endpoint: "new/shuffle/",
      parameters: []
    )
    fetch(request, completion)
  }

  /// Fetch any endpoint on the API.
  internal func fetch<Value: Decodable>(
    _ request: Request
    _ completion: @escaping (Result<Value, Error>) -> Void
  ) {
    let url = Client.BASE_URL.appendingPathComponent(request.endpoint)
    var components = URLComponents(url: url, resolvingAgainstBaseURL: false)!
    if !request.parameters.isEmpty {
      components.queryItems = request.parameters.map { pair in
        URLQueryItem(name: pair.0, value: pair.1)
      }
    }
    var request = URLRequest(url: components.url!)
    request.httpMethod = request.method.rawValue
    fetch(request, completion)
  }
}

With that in place, we can extract the transformations into separate methods: one to transform semantic values into Requests and another to transform Requests into URLRequests.

class Client {
  // Transform endpoint-specific values into an endpoint-agnostic request
  func shuffle() -> Request {
    return Request(
      method: .get,
      endpoint: "new/shuffle/",
      parameters: []
    )
  }

  /// Shuffle a new deck and return its ID.
  func shuffle(_ completion: @escaping (Result<DeckInfo, Error>) -> Void) {
    fetch(shuffle(), completion)
  }

  // Transform an endpoint-agnostic request into a more general URL request
  func urlRequest(for request: Request) -> URLRequest {
    let url = Client.BASE_URL.appendingPathComponent(request.endpoint)
    var components = URLComponents(url: url, resolvingAgainstBaseURL: false)!
    if !request.parameters.isEmpty {
      components.queryItems = request.parameters.map { pair in
        URLQueryItem(name: pair.0, value: pair.1)
      }
    }
    var urlRequest = URLRequest(url: components.url!)
    urlRequest.httpMethod = request.method.rawValue
    return urlRequest
  }

  /// Fetch any endpoint on the API.
  internal func fetch<Value: Decodable>(
    _ request: Request
    _ completion: @escaping (Result<Value, Error>) -> Void
  ) {
    fetch(urlRequest(for: request), completion)
  }
}

Now our tests for the requests can use these internal methods instead of using mocks:

func testShuffleMakesCorrectRequest() {
  let actual = client.urlRequest(for: client.shuffle())
  XCTAssertEqual(actual.httpMethod, "GET")
  XCTAssertEqual(actual.url, URL(string: "https://deckofcardsapi.com/api/deck/new/shuffle/")!)
}

That’s a big improvement over what we had before:

func testShuffleMakesCorrectRequest() {
  // Make the request and wait for it to complete
  let expectation = Expectation("request")
  client.shuffle() { _ in
    expectation.fulfill()
  }
  expectation.wait(timeout: 1)

  // Get the request that was sent
  let actual = HTTPMocks.sent.last
  // Make our assertions
  XCTAssertEqual(actual?.httpMethod, "GET")
  XCTAssertEqual(actual?.url, URL(string: "https://deckofcardsapi.com/api/deck/new/shuffle/")!)
}

We got rid of the expectation and timeout, the mocks, and even the optional return value. The test is shorter, faster, more reliable, easier to debug, and easier to read.

But we can do even better. We’ve identified 2 transformations. We’re currently testing them together, but we can simplify our tests by testing them separately:

func testShuffleMakesCorrectRequest() {
  let actual = client.shuffle()
  let expected = Request(method: .get, endpoint: "new/shuffle/", parameters: [])
  XCTAssertEqual(actual, expected)
}

func testRequestToURLRequest() {
  let request = Client.Request(
    method: .get,
    endpoint: "foo",
    parameters: [("bar", "baz")]
  )
  let urlRequest = client.urlRequest(for: request)
  XCTAssertEqual(urlRequest.httpMethod, "GET")
  XCTAssertEqual(urlRequest.url, URL(string: "https://deckofcardsapi.com/api/deck/foo?bar=baz"))
}

By recognizing this semantic layer and formalizing it into a Request type, we can write our assertions in the language that we’d use to describe the expected behavior. It also lets us think in the terms of this semantic layer. The test is shorter, clearer, and less brittle (it won’t break if we change how we create URLRequests or even if we abandon URLRequest).

The code in urlRequest(for: Request) didn’t have any direct tests in our initial version (it was indirectly tested through the endpoint methods). Since the code was buried inside an imperative method, it didn’t stand out as something that could or should be tested. And testing this code would have required more mocks, but now it doesn’t need them.

The intermediate fetch method no longer provides much value:

func fetch<Value: Decodable>(
  _ request: Request,
  _ completion: @escaping (Result<Value, Error>) -> Void
) {
  fetch(urlRequest(for: request), completion)
}

So we can get rid of it and move the Request-to-URLRequest conversion into the final fetch method:

func fetch<Value: Decodable>(
  _ request: Request,
  _ completion: @escaping (Result<Value, Error>) -> Void
) {
  URLSession
    .shared
    .dataTask(with: urlRequest(for: request)) { data, response, error in
      if let error = error {
        completion(.failure(error))
      } else {
        completion(.init { try JSONDecoder().decode(Value.self, from: data!) })
      }
    }
    .resume()
}

We’re starting to benefit from abstraction. It’s easier to reason about the methods in isolation now that we’re using a meaningful value as the boundary between them.

We’ve cleaned up the requests quite a bit, but responses are still a bit of a mess. Let’s see what we can do.

JSON decoding is transformation from a Data into a Result<Value, Error>—it’s a pure function. Each request is tied to a specific decoding: shuffle() returns a DeckInfo and draw() returns a Draw. We can move that decoding into Request:

struct Request<Value: Decodable> {
  var method: Method
  var endpoint: String
  var parameters: [(String, String)]

  func decode(_ data: Data) -> Result<Value, Error> {
    return Result { try JSONDecoder().decode(Value.self, from: data) }
  }
}

Then our Request-generating methods can specify the response type:

func shuffle() -> Request<DeckInfo> {
  return Request(method: .get, endpoint: "new/shuffle/", parameters: [])
}

func draw(count: Int, from deck: Deck) -> Request<Draw> {
  return Request(
    method: .get,
    endpoint: "\(deck)/draw/",
    parameters: [("count", "\(count)")]
  )
}

Now we can clean up the response tests, just like we did with the request tests:

func testShuffleParsesResponseCorrectly() throws {
  let actual = try client
    .shuffle()
    .decode(Fixture.shuffle.data)
    .get()

  let expected = DeckInfo(deck: Deck(id: "gy7aqxz5nmpd"))
  XCTAssertEqual(actual, expected)
}

This is another big improvement over what we had before:

func testShuffleParsesResponseCorrectly() {
  let url = URL(string: "https://deckofcardsapi.com/api/deck/new/shuffle/")!
  HTTPMocks.mocked[url] = .success(Fixture.shuffle.data)

  let expectation = Expectation("request")
  var actual: DeckInfo?
  client.shuffle() { result in
    if case let .success(deck) = result {
      actual = deck
    }
    expectation.fulfill()
  }
  expectation.wait(timeout: 1)

  let expected = DeckInfo(deck: Deck(id: "gy7aqxz5nmpd"))
  XCTAssertEqual(actual, expected)
}

We got rid of the coupling to the URL, the expectation, the mock, and the timeout. We’ve made the test shorter, clearer, faster, easier to debug, and more reliable.

Now we need to update fetch to use this new method:

func fetch<Value>(
  _ request: Request<Value>,
  _ completion: @escaping (Result<Value, Error>) -> Void
) {
  URLSession
    .shared
    .dataTask(with: urlRequest(for: request)) { data, response, error in
      if let error = error {
        completion(.failure(error))
      } else {
        completion(request.decode(data!))
      }
    }
    .resume()
}

Our pursuit of value types has led us to a very straightforward implementation of this method.

At this point, we’ve adopted value types internally. We haven’t changed the public API of Client, but we got everything that we hoped to get from value types:

Testability is improved
The code is easier to understand
There's more potential for code reuse

You can stop there and gain a lot of benefit.

But if you’re open to a more radical approach and to changes to the public API, you can keep going.

The public methods in the client no longer do very much:

/// Shuffle a new deck and return its ID.
func shuffle(_ completion: @escaping (Result<DeckInfo, Error>) -> Void) {
  fetch(shuffle(), completion)
}

/// Draw and return cards from the top of a deck.
func draw(
  count: Int,
  from deck: Deck,
  completion: @escaping (Result<Draw, Error>) -> Void
) {
  fetch(draw(count: count, from: deck), completion)
}

If you expose the 3 methods that these 2 methods call, then you can get rid of them entirely and have a public API like this:

class Client {
  /// Shuffle a new deck and return its ID.
  func shuffle() -> Request<DeckInfo>

  /// Draw and return cards from the top of a deck.
  func draw(count: Int, from deck: Deck) -> Request<Draw>

  /// Fetch a `Request`.
  func fetch<Value>(
    _ request: Request<Value>,
    _ completion: @escaping (Result<Value, Error>) -> Void
  )
}

But the endpoint-specific methods here no longer require a Client: their implementations only generate Requests. So we can take this opportunity to reorganize them a bit (and guarantee their independence in the process):

extension Deck {
  /// Shuffle a new deck and return its ID.
  static func shuffle() -> Request<DeckInfo>
}

extension Deck {
  /// Draw and return cards from the top of the deck.
  func draw(count: Int) -> Request<Draw>
}

class Client {
  /// Fetch a `Request`.
  func fetch<Value>(
    _ request: Request<Value>,
    _ completion: @escaping (Result<Value, Error>) -> Void
  )
}

This is the easiest and most minor change to make, but also the most radical.

The primary benefit of this step is to make it easier to adopt value types across different sections of your code. A method on Client can only be passed as a block. But a Request can be passed around, introspected, and possibly transformed. If you want to use value types everywhere, then using value types in your public APIs will make that easier.

You can get the benefit of value types without changing your public API. By using them internally, you can improve testability, comprehension, and reuse. This will lead to some different patterns than you may be used to, but the results are worth it.

Download the code.

I’ve written about Value-Oriented Programming. Gary Bernhardt makes a compelling case for using value types at the boundaries of a system in his Boundaries talk (which you should absolutely watch). ↩
Okay, so the API itself may not be very real-world. But for our purposes, the content doesn’t matter. Deck of Cards is simple, free, open, and illustrates the use of a JSON API. ↩