Campaigns design

Why are campaigns designed the way they are?

Principles

• Declarative API (not imperative). You declare your intent, such as "lint files in all repositories with a package.json file". The campaign figures out how to achieve your desired state. The external state (of repositories, changesets, code hosts, access tokens, etc.) can change at any time, and temporary errors frequently occur when reading and writing to code hosts. These factors would make an imperative API very cumbersome because each API client would need to handle the complexity of the distributed system.
• Define a campaign in a file (not some online API). The source of truth of a campaign's definition is a file that can be stored in version control, reviewed in code review, and re-applied by CI. This is in the same spirit as IaaC (infrastructure as code; e.g., storing your Terraform/Kubernetes/etc. files in Git). We prefer this approach over a (worse) alternative where you define a campaign in a UI with a bunch of text fields, checkboxes, buttons, etc., and need to write a custom API client to import/export the campaign definition.
• Shareable and portable. You can share your campaign specs, and it's easy for other people to use them. A campaign spec expresses an intent that's high-level enough to (usually) not be specific to your own particular repositories. You declare and inject configuration and secrets to customize it instead of hard-coding those values.
• Large-scale. You can run campaigns across 10,000s of repositories. It might take a while to compute and push everything, and the current implementation might cap out lower than that, but the fundamental design scales well.
• Accommodates a variety of code hosts and review/merge processes. Specifically, we don't to limit campaigns to only working for GitHub pull requests. (See current support list.)

Comparison to other distributed systems

Kubernetes is a distributed system with an API that many people are familiar with. Campaigns is also a distributed system. All APIs for distributed systems need to handle a similar set of concerns around robustness, consistency, etc. Here's a comparison showing how these concerns are handled for a Kubernetes Deployment and a Sourcegraph campaign. In some cases, we've found Kubernetes to be a good source of inspiration for the campaigns API, but resembling Kubernetes is not an explicit goal.

# File: foo.Deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  # Evaluate this to enumerate instances of...
  replicas: 2
  # ...this template.
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

# File: hello-world.campaign.yaml
name: hello-world
description: Add Hello World to READMEs
# Evaluate this to enumerate instances of...
on:
  - repositoriesMatchingQuery: file:README.md
steps:

run: echo Hello | tee -a $(find -name '*.md')
container: alpine:3


# ...this template.
changesetTemplate:
  title: Hello World
  body: My first campaign!
  branch: hello-world
  commit:
    message: Append Hello to .md files
  published: false

These docs explain more about Kubernetes' design:

• Kubernetes Object Management
• Kubernetes Controllers
  • Desired versus current state
• Kubernetes Architecture
• Kubernetes General Configuration Tips
• Kubernetes Design and Development Explained