Documentation

What Team adds Team

• A shared workspace, hosted by Breakpatch with nothing to set up, or in your company's own Google Firebase project (Host it yourself): your team's apps, tests, suites and runs in one place. A test is Only you until you add it to the team suite. Then it's In team suite.
• Members and roles: member, admin, a runner account and a CI account.
• Version history. Every save is kept. Runs show which version they tested, and you can restore any version.
• Fixed automatically. When a button has moved, the AI assistant finds it during the run and carries on. You accept the new position in the report.
• Why did this fail? In the report, the AI assistant says in plain words what changed, for example "The Save button now reads “Save changes”", with the likely cause and what to do.
• Schedules for suites, on any days and time.
• The local runner: one Mac that runs suites for the whole team.
• Run requests from CI or any other tool, and result messages after every suite run, to Slack, Microsoft Teams or any web address.
• Create issue in GitHub, Linear or Jira from a failed step, with the steps, the screenshot and a link to the report.
• breakpatch-ci: run the workspace's suites on your CI machines, with the results in the run history.

Prices, seats and support for Team and Business are set when they go on sale, on the pricing page.

Workspaces hosted by Breakpatch aren't open yet either. Until they are, a workspace lives in your company's own Firebase project, set up as in Create a workspace.

Solo Team

Coming later. Solo isn't on sale yet. To hear when it is, email support@breakpatch.dev.

Solo is planned for one person who wants Team's automation for themselves, on their own tests folder. This is how it works in the app today. Its price isn't set.

• Your tests stay where they are. Solo works on your tests folder, or a folder you keep in Git. There's nothing to set up in the cloud.
• A workspace is optional. For run requests, the local runner and version history, connect a workspace of your own (Create a workspace). It stays yours alone: Solo has no members or invites.
• The licence. With your tests folder open, go to Settings → Licence → Enter Solo licence key, and give the email you bought Solo with and the key (it looks like BP-XXXX-XXXX-XXXX-XXXX). The key stays in this Mac's Keychain. It works on one Mac at a time. To use it on another Mac, press Release this Mac, or free it on your licence page at account.breakpatch.dev, then enter the key on the new Mac. A Team or Business key there says "This is a Team licence key. Team licences are used in a workspace: connect or create one, then enter the key there."
• Schedules on your Mac. Set a suite's schedule and where its result message goes, as in Team. The suite runs on this Mac while Breakpatch is open, one test at a time: if you're running a test yourself, it waits for it. If a time came while Breakpatch was closed or the Mac was asleep, the suite runs once when you open the folder again (or the Mac wakes), not once for every time it missed, and only if that time was in the last 24 hours. Its run history says so, for example "This 8:00 run was missed, so it ran when Breakpatch opened at 9:14." When it finishes, you get a notification if Breakpatch is in the background (Settings → Notifications says which). The result message's address stays in this Mac's Keychain, never in the folder, so it doesn't end up in Git. The failed step's screenshot needs a workspace, so it isn't sent from a folder. A saved secret a scheduled test types needs Runner can use turned on in Settings → Saved secrets.
• One machine licence, for breakpatch-ci or a runner. Your Mac and the machine licence each run one test at a time. Solo has no extra machine licences. In CI, set BREAKPATCH_LICENCE_KEY to your Solo key and leave BREAKPATCH_WORKSPACE out to run test files from your folder (--test FILE.json); a second job on the same machine waits for the first.
• Your own workspace, if you want one. After you connect it, Settings → Licence offers Use Solo here: it's the same seat, for the same email, so there's no key to type. The workspace has no Members page and no invite link.
• One Solo per company, for companies without Team. If someone at your company already has Solo, the pricing page offers Team instead. If your company already uses Team or Business (an admin or anyone with a seat has an address at your company), ask its admin for a seat there: Solo isn't sold for it. Your company is your email's domain, with its subdomains: eng.acme.co.uk and acme.co.uk are one company. A personal address (such as Gmail or iCloud) counts on its own.
• Moving to Team keeps your tests. Email support@breakpatch.dev to switch your plan, then follow Upgrading to Team: connect or create the workspace, press Use Solo here (or enter the Team key if you have it already), and move your tests. Suites keep their schedules; set where their results go again in each suite. Then, in Settings → Licence, press Replace key and enter the Team key. The Solo seat on this Mac is given back.

Upgrading to Team Team

There's nothing to install: the Breakpatch you got with the install command already has Team inside, and a licence turns it on. (A copy you build yourself from the open-source repository is Community only.) The AI assistant, the test browser, your settings and your saved secrets stay as they are, and Breakpatch opens your tests folder like before. Once Team is on sale, go to Settings → Upgrade to Team (or Upgrade to Team on Home); until then, a tests folder doesn't offer it:

1. Connect a workspace: sign in to one hosted by Breakpatch, create one of your own, or open an invite link from your team. Then sign in.
2. The licence. A hosted workspace needs no key: Breakpatch sets up everyone's seat. If you created a workspace of your own, enter the licence key once in Settings → Licence (it looks like BP-XXXX-XXXX-XXXX-XXXX). If you joined with an invite link, the link sets up your licence and there's no key to enter. See Licences and seats.
3. Move your tests. Choose the tests folder. Breakpatch shows what it will copy first, for example "1 app, 12 tests, 3 shared steps and 2 suites will be copied.", and you can rename apps, or merge them with apps the workspace already has. It then copies the apps, tests, shared steps, suites and the last run of each test, unless that run is over 90 days old, into the workspace.
4. Check, then move to the Trash. Breakpatch reads everything back from the workspace and compares it with the folder. Only when it all matches, and you confirm, does it move the folder's breakpatch.json, apps and suites to the Trash, so there's one place to edit your tests. To undo it, drag them from the Trash back into the folder.

Good to know:

• Changed your mind? The connect screen always has a way back at the top: to where you came from, or Back to your tests folder, also after Switch workspace.
• Only Breakpatch's files move. Everything else in the folder stays as it is. If the folder is in a Git repository, the removed tests show as deleted files to commit, and earlier versions stay in the repository's history.
• If the move stops half way (the Mac restarts, say), Breakpatch picks it up where it left off the next time you open Upgrade to Team.
• A report of the move is kept in Breakpatch's data folder on this Mac: what was copied, what went to the Trash, and anything left out.
• Each test starts at version 1 in the workspace, because the folder keeps only the latest version.
• Runs over 90 days old aren't copied, because the workspace keeps runs for 90 days (see Run history is kept 90 days). Breakpatch says so before it copies, and the report lists them. They go to the Trash with the folder.
• Saved secrets travel as names in the steps. Their values stay in this Mac's Keychain; teammates add their own in Settings → Saved secrets.
• Files that can't be read stop the move until you fix them in the folder, so nothing unread goes to the Trash.
• Already in the workspace? Anything that's already there is left out. If a test there differs from the folder's (a teammate copied an older version, say), Breakpatch names it, and you can keep the workspace's version.
• A folder saved by a newer Breakpatch can't be moved until you update.

Hosted by Breakpatch Team

Coming later. Until then, keep your team's tests in your own Firebase project: see Host it yourself.

The easy way to a Team or Business workspace: Breakpatch keeps it for you, in Europe, with nothing to set up. Everyone signs in with their work email and a code; there are no passwords. It's the same Breakpatch as a workspace of your own, with the same tests, suites, runs and roles.

Sign in

Connect a team workspace → Hosted by Breakpatch. Type your work email and choose Email me a code. Enter the 6 digits from the email (it comes from signin@breakpatch.dev and works for 10 minutes). Breakpatch then lists your workspaces and your invitations: click a workspace to open it, or Join one you're invited to.

This Mac stays signed in to your Breakpatch account, so next time the list opens straight away. Use another address signs it out. If a workspace asks you to sign in again, it shows Open the workspace with the address you're signed in as: choose Continue, or get a new code.

Create the hosted workspace

A licence's admin does this once, after getting Team or Business. Sign in as above with the address the licence has for its admin, then choose Create a hosted workspace, give it a name, choose the country your company is in and press Create workspace. You're its first admin, and the licence's other admins are invited. Your data is stored in Europe. Hosting for companies in the EU and the EEA isn't open yet: those countries say (coming later) in the list, and choosing one offers Host it yourself instead.

Creating it turns on encryption: the workspace's key is made on your Mac, and Breakpatch never has it. Straight away, Breakpatch shows the workspace's recovery code: save it (Save recovery kit (PDF) or Copy code) and type back the 4 characters it asks for. The window stays open until you do, or until you choose Skip, I'll make a new one later and confirm. Without the code, losing every Mac that has the key means losing the tests.

There's no licence key to enter: Breakpatch gives everyone in the workspace their seat, the runner Mac too. Runs are kept for 90 days, or 365 with a licence that includes the longer history, then deleted; the Create screen says which.

Invite people

Settings → Members → Invite people: type their email addresses, choose Member or Admin, and press Send invitations. Each gets an email. They open Breakpatch, choose Hosted by Breakpatch, sign in with that address and choose Join.

• Invitations last 14 days. Waiting to join lists them; × takes one back.
• Anyone at @yourcompany.com can join lets people with an address at your own company's domain join without an invitation. It's never offered for public addresses such as Gmail or Outlook.
• A workspace holds its seats plus 5 people.
• Remove from workspace stops someone opening it at once. To let them back, invite them again.

A runner Mac and CI

A runner Mac and CI sign in with a token instead of an email. An admin makes one in Settings → Local runner:

• Set up a runner Mac: name it, press Make the token and copy it. Runner tokens start with BPM1-. It's shown only this once. On the runner Mac choose Connect a team workspace → Hosted by Breakpatch → This is a runner Mac, and paste it. That Mac becomes the workspace's local runner, and its runs show under its name. The runner's checklist in Settings → Local runner has the same Set up a runner Mac button.
• Add a CI token: for breakpatch-ci. CI tokens start with BPC1-; one made earlier starts with BPM1- and keeps working. The window lists what CI needs, each with its own Copy. In CI, set BREAKPATCH_WORKSPACE=hosted:<workspace>, and the secrets BREAKPATCH_WORKSPACE_TOKEN (the token), BREAKPATCH_MACHINE_KEY (the workspace's machine key) and BREAKPATCH_LICENCE_KEY (your licence key, for the machine licence). There's no .bpworkspace file or CI account: breakpatch-ci run --suite smoke-7f3a signs in with the token.
• The wrong one in the wrong place is refused before anything is sent, and the message says which it got and which goes there. A CI token pasted on the runner Mac says "That's a CI token, for breakpatch-ci. This Mac needs a runner token, which starts with BPM1-". A runner token or the runner's machine pass in BREAKPATCH_WORKSPACE_TOKEN says that breakpatch-ci needs a CI token, which starts with BPC1-. A token typed as a licence key says it isn't one. The runner's machine pass (also BPM1-…, but longer) is never typed: Breakpatch gives it to the runner.
• The trash button next to a token stops it working at once. A workspace holds its machine licences plus 2 tokens.

Plan, region and usage

Settings → Workspace shows the plan and seats, where the workspace is kept, how long runs are kept, and how many tests and how much storage it uses against its allowance (counted once a day). From 80 % the storage bar says Nearly full; when it's full, runs keep working and are saved, but new tests, new versions and screenshots wait until there's room. To add storage, write to support@breakpatch.dev, or delete tests and apps you no longer need. The emails and the account page say the same.

Download export (admins) makes a zip of the workspace's tests, versions, runs and suites and opens the download in your browser; the link works for 7 days. The account page has the same Download export. When the licence ends, the workspace becomes read-only for 30 days, so you can still open and export your tests, and then it's deleted: Settings → Workspace shows the date first. Renewing makes it active again. Remove from this Mac signs you out here; the workspace and its tests stay.

Host it yourself Team

Keep the workspace in your company's own Google Firebase project instead: for teams with an IT person or developer, or who want the data in their own cloud account. You set it up once; then teammates join with a link. In Breakpatch: Connect a team workspace → Host it yourself.

Create a workspace

Do this once for your organisation. You need owner access to a Firebase project. Use the project your product already signs in with, so everyone keeps their usual login. A new project works too: step 5 turns on sign-in and adds your account.

In Breakpatch: Connect a team workspace → Host it yourself → Setting up for your team? → Create a workspace. It has six steps, numbered as in the app. Each has Open Firebase console (step 4, Open Google Cloud console), which opens the right page.

1. Create the web app and paste its config. Project settings → General → Your apps → Add app → Web. Name it Breakpatch, leave Firebase Hosting off and click Register app. Ignore the code Firebase shows and click Continue to console. Then, under Web apps, pick Breakpatch. In SDK setup and configuration, choose Config, copy it and press Paste in Breakpatch (it pastes straight away). Then fill in Who can sign in: just the part after @, for example example.com.
2. Choose the database. Breakpatch asks which one to use:
   • This project's (default) database: best if the Firebase project is only for Breakpatch, and it's the one Firebase's free tier covers. Open Firestore Database. If the page shows Create database, click it, keep Standard edition and the ID (default), pick a location near your team (you can't change it later), keep Production mode and click Create. If the project has a (default) database already, there's nothing to do.
   • A separate database, if the project also runs your app, so test data stays apart. Open Firestore Database, open the database menu at the top of the page (the one that shows (default)) and choose Add database (a project with no database yet shows Create database instead). Keep Standard edition, enter the database ID breakpatch (copy it from the app), pick the same location as your main database, keep Production mode and click Create.
3. Paste the rules. In Breakpatch press Copy rules. In Firestore pick the database you chose in step 2 ((default) or breakpatch), open the Rules tab, replace everything with the rules you copied and click Publish.
4. Delete runs after 90 days automatically (recommended, saves money). You can skip this step and do it later. Breakpatch opens the Time-to-live (TTL) page of your database in the Google Cloud console: sign in with the same Google account as for Firebase. Click Create policy three times, each with the timestamp field expiresAt: for the collection groups runs, suiteRuns and runNotes (what's added to a run later: the issue made from it and the AI assistant's explanations). Firestore takes a few minutes to turn them on, then deletes them 90 days after they were saved (see Run history is kept 90 days). Or with the Google Cloud CLI, as the step shows them: gcloud firestore fields ttls update expiresAt --collection-group=runs --enable-ttl --database=breakpatch --project=<your project> (with your database's ID: --database='(default)' for the default one; the step shows the commands filled in), and the same with --collection-group=suiteRuns and --collection-group=runNotes. Press I've added the policies, or Skip this step: the steps at the top then show it as skipped, not done. If you skip it, an admin's Breakpatch deletes old runs instead, once a day, which costs a few reads more.
5. Turn on sign-in and add yourself. Open Authentication (click Get started if it's new). In Sign-in method choose Email/Password, turn on Enable and click Save. Then in Users click Add user and add yourself, with an address in the Who can sign in domain. Add your teammates the same way, now or later. If your team already signs in to this project with email and password, just check it's on. Breakpatch never creates accounts.
6. Connect. Check the summary. Breakpatch checks the connection by itself (see below); fix anything it names first. Name the workspace, add a logo if you like and press Create workspace. Then sign in with the account you added. If you haven't added it yet, the sign-in screen says where, with a link to Authentication → Users in the Firebase console.

People sign in with their work email and a password. The first time, Breakpatch sends them a link to confirm their email address.

Whoever connects the workspace becomes its first admin.

Check the connection

Check the connection (the last step of Create a workspace, and Settings → Workspace) looks at each part of the set-up in turn and says what to do about anything that isn't right, with Open Firebase console on the right page:

• Firebase project: the config is a project's, and it answers.
• Email and password sign-in: it's turned on in Authentication.
• Database: the database named in the workspace is there.
• Security rules: the Breakpatch rules for this version are published to it.

It only reads: a sign-in with a made-up address (which never signs anyone in) and one read of the database. The sign-in counts towards Firebase's limit on failed sign-ins from your address, so after many checks in a row it may say it couldn't tell for a few minutes.

Edit connection

If the details on this Mac are wrong (the config, the database ID, Who can sign in or the name), choose Edit connection on the sign-in screen or in Settings → Workspace. It changes only this Mac's saved details, never the workspace, then checks the connection with them. Changing the Firebase project or the database makes it another workspace to this Mac: its licence seat under the old details is given back, and it takes one again when you sign in.

Run history is kept 90 days

In a team workspace, runs and suite runs are kept for 90 days, then deleted: by Firestore's TTL policy when it's set up (step 4 of Create a workspace, recommended), and otherwise by an admin's Breakpatch, once a day. Breakpatch from before this can't add runs to the workspace: update it on every Mac, the local runner and your CI machines. Tests, shared steps and their versions are only deleted when you delete them, 30 days after (see Get a deleted app or test back).

Lists that only grow show the newest first: the Runs tab and Run history show the latest 20 runs, with Show older runs for 20 more, and Version history the latest 20 versions, with Show older versions.

Breakpatch keeps a copy of the workspace on each Mac, so opening it again only fetches what changed since, which keeps your Firebase bill small. After an update that changes the security rules, publish them again (Settings → Workspace → Copy security rules): until then Breakpatch reads everything each time, as before.

Invite your team

Settings → Workspace → Invite teammates → Copy link. Send it any way you like. It works for everyone and doesn't expire.

Teammates don't need the licence key. Once your admin has entered it in Settings → Licence, everyone who opens the link and signs in gets a seat by themselves. The link holds only the workspace details, never the licence.

https://breakpatch.dev/connect#c=eyJuYW1lIjoiU3ltVGVycmEi…

Clicking it opens a page with Open in Breakpatch and Install Breakpatch. The link also sets up each person's licence, so they don't enter a key: they take a seat when they sign in. The workspace details are in the part after #, which browsers never send to a server.

Save as file gives the same thing as a .bpworkspace file, handy for a shared drive, CI or the local runner. Double-click it to connect.

Teammates can also paste the link on the connect screen: Host it yourself → Got a link or a .bpworkspace file? → Or paste the link here.

In an encrypted workspace, Copy link with the key gives the same link with the key that opens the tests in its # part, so a teammate's Mac reads them at once; it's the one to send, and Copy link without the key gives the plain one. Send it only to teammates. The teammate's Mac takes the key only once they choose to join the workspace. Without the key, an admin lets each new Mac in: see Encryption and the recovery code.

Members and roles Team

Everyone from your email domain joins as a member the first time they sign in (in a hosted workspace, everyone you invite). Admins change roles in Settings → Members.

Only admins can change roles, and nobody can change their own. Remove from workspace stops someone opening it; their tests and runs stay. In an encrypted workspace, removing an admin or making them a member also withdraws their Mac's signature (see Encryption and the recovery code). If they sign in again with an account from your domain, they join as a member again, so to keep someone out, disable their account in Firebase Authentication.

Runner and CI accounts sign in once and join as a member. Then an admin changes their role. In a hosted workspace, a runner Mac and CI use tokens instead, and people are members or admins.

Version history Team

Every save is a new version, with an optional note: What changed? → Save as version 4. Versions are never deleted on their own: only with their test or shared steps, 30 days after those are deleted.

On a test or shared steps, open Version history to see each version, who saved it, what changed since the one before and the runs on that version. Restore version 2 saves it again as the newest version. The current one stays in the history.

Every run shows the version it tested. Shared steps show which tests use the latest version and which are kept on a fixed one.

Mark as released (on a test's version) picks the version your pipelines run with breakpatch-ci --version released, so edits still in progress don't break the build. The version list shows it as Released. Clear release mark takes it off.

Fixed automatically Team

When a button has moved, the AI assistant looks for it using its What to look for description, checks it's the same thing, and the run carries on. The step is marked Fixed automatically, and the suite result is Passed with fixes.

Nothing changes until someone decides. In the report:

• Accept new position: the next run looks in the new spot.
• Dismiss: the next run looks in the old spot again.

Settings → Automatic fixing:

• Fix moved buttons automatically.
• Fail the test if anything needed fixing. The run still finds the button, but the test counts as failed. Useful before a release.

These apply to runs on the Mac where you set them.

The first few runs of a new test also learn which parts of the page change by themselves, like clocks and carousels, so they don't fail the test.

Why did this fail? Team

"Couldn't find the Save button" says what failed. Why did this fail? says what changed, so you can tell a real bug from a change someone made on purpose.

Open the run report, pick the failed step and press Why did this fail? under the reason. After a few seconds the AI assistant answers, marked From the AI assistant:

• What changed, in a sentence or two. For example: "The Save button now reads “Save changes”", "The Save button isn't where it was. It's now at the top right of the screen", "The Save button is hidden behind the “Cookies” dialog" or "The page shows an error: “Page not found”".
• Likely cause: it moved, its text changed, the page changed, the page was slow to load, the page shows an error, or it looks like a real bug.
• Suggested: re-record this step, accept the change by re-recording it, give the page longer with a wait before the step, or report it as a bug.

It looks at the screenshot of the failure and what the page said was on screen then, and compares them with how the step was recorded. It only says what it found: if the button is still where it was, it never says it moved. When it can't tell, it says so; compare the two screens instead.

• It's worked out when you ask, never during the run, so it never slows a run down. Once it has answered, the answer is kept with the run, on every Mac in the workspace, and Copy includes it.
• It needs the AI assistant on this Mac, a licence that includes it, and the failure's screenshot, which stays on the Mac that ran the test. For a run on another Mac (the local runner, or a teammate's), ask on that Mac.
• It explains steps that couldn't find what they act on, didn't change the page, or left the screen looking different. A missing saved secret, a failed set-up call or a run you stopped already say it all.
• Everything stays on the Mac: the screenshot and what the page said are kept next to each other in the screenshots folder, like any failure screenshot.

Schedules Team

Open a suite and set its Schedule: the days and a 24-hour time, like Mon–Fri, 06:00. The time is the local runner's clock. With no schedule, the suite is By request only: the Run button, CI or another tool.

The suite shows its next run. If the runner was off or asleep when a run was due, the missed run happens once when it's back, not once for every time it missed, and only if it was due in the last 24 hours: an older one is skipped, and the suite runs at its next time as usual. A run that starts 5 minutes late or more says so in its result message and HTML report, for example "This 8:00 run was missed, so it ran when Breakpatch opened at 9:14."

With Solo on a tests folder there's no runner: the time is this Mac's clock, and the suite runs while Breakpatch is open. A time missed while Breakpatch was closed, or the Mac was asleep, runs once when it opens the folder again (or the Mac wakes), with the same 24-hour limit, and each test's run history and report show the same note.

The local runner Team

One Mac per workspace that stays on and runs scheduled suites and run requests for everyone.

1. Install Breakpatch on that Mac and connect it to the workspace.
2. Sign in with a separate account (for example runner@yourcompany.com) so runs don't show under a person's name. Give it the Local runner role.
3. Settings → Local runner → Use this Mac as the local runner.

It then stays awake while it's plugged in, opens at login and starts in runner mode after a restart or update. The checklist in Settings → Local runner says Stays awake; if macOS won't let it, it says Can't stay awake and why: then turn off sleep for that Mac in System Settings. Runner mode shows what it's doing: waiting, running (with a live view), the queue and the last runs. Pause after this test stops it cleanly. Press Resume to carry on.

It uses the Standard AI assistant, like every Mac. With 32 GB of memory or more you can download the Larger one in the same place, but it's rarely needed.

Record and run on the same kind of machine. Tests pass most reliably on the runner when they're recorded on a Mac like it, with the same version of Breakpatch. A test recorded with another version of the test browser runs with Allow for small differences between systems (Settings → Screen checks, on unless you turn it off) on the runner Mac, and its report says so when a check fails. See Recorded on another system.

HTML reports. Settings → Local runner → Save an HTML report of every suite run saves each suite run as the same web page as Export, in a folder you choose on the runner Mac (named after the suite, the day and the time). Turn off Include screenshots there if others can open the folder who shouldn't see them. If the folder is also on a web server, or a shared drive with web links, enter its web address (https:// is added if you leave it out; it's saved when you leave the field): the Slack and Teams result messages then have an Open the HTML report button, and the JSON has htmlReportUrl. Without one, the report is only saved: a file on the runner Mac isn't something others can open from a message. If a report can't be saved, the runner's status says so.

Anyone can press Run on runner on a suite. If the runner is offline, the request waits until it's back. The runner tells the workspace it's there every 90 seconds, and whenever what it's doing changes; after 5 minutes without that, the other Macs show it as offline.

Queue rules

• First in, first out.
• The newest request for a suite wins: a waiting request for the same suite is removed, and a running one stops after its current step. Both are recorded as Replaced by a newer request.
• If the runner is offline, requests wait. Missed schedules run once when it's back, if they were due in the last 24 hours.

Run requests Team

The runner starts a suite whenever a document is added here:

projects/<your-project>/databases/breakpatch/documents/runRequests

Settings → Local runner → How run requests work shows the exact path for your workspace. If the workspace uses the project's (default) database, the path has (default) where this one has breakpatch (and so do the examples below). Each document is one request. Let Firestore make its ID.

The runner picks it up within a few seconds, queues it and deletes the document. The Run button and schedules add these for you.

How other tools add requests is up to you. A few examples:

From CI with a CI account

Create a user such as ci@yourcompany.com and give it the ci role (see Security rules). Store its email and password, your Firebase web API key and your project ID as CI secrets.

TOKEN=$(curl -s "https://identitytoolkit.googleapis.com/v1/accounts:signInWithPassword?key=$FIREBASE_API_KEY" \
  -H 'Content-Type: application/json' \
  -d "{\"email\":\"$BP_CI_EMAIL\",\"password\":\"$BP_CI_PASSWORD\",\"returnSecureToken\":true}" | jq -r .idToken)

DB="projects/$FIREBASE_PROJECT/databases/breakpatch/documents"

curl -s -X POST "https://firestore.googleapis.com/v1/$DB:commit" \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d "{\"writes\":[{
    \"update\":{\"name\":\"$DB/runRequests/$(openssl rand -hex 10)\",\"fields\":{
      \"suiteId\":{\"stringValue\":\"smoke-7f3a\"},
      \"requestedBy\":{\"stringValue\":\"CI · build $BUILD_NUMBER\"}}},
    \"updateTransforms\":[{\"fieldPath\":\"createdAt\",\"setToServerValue\":\"REQUEST_TIME\"}],
    \"currentDocument\":{\"exists\":false}}]}"

Works the same in Codemagic, GitHub Actions, Bitrise or Jenkins.

From a script (Node)

import { initializeApp } from 'firebase/app';
import { getAuth, signInWithEmailAndPassword } from 'firebase/auth';
import { getFirestore, collection, addDoc, serverTimestamp } from 'firebase/firestore';

const app = initializeApp(firebaseConfig);
await signInWithEmailAndPassword(getAuth(app), process.env.BP_CI_EMAIL, process.env.BP_CI_PASSWORD);
await addDoc(collection(getFirestore(app, 'breakpatch'), 'runRequests'), {
  suiteId: 'smoke-7f3a',
  requestedBy: 'Release script',
  createdAt: serverTimestamp(),
});

From an automation tool

n8n, Zapier or Make can add a Firestore document on any event: a deploy, a merged pull request, a button in a chat. Use the same path and fields, with createdAt set to the server's time.

From CI with breakpatch-ci Team

breakpatch-ci runs tests on the CI machine itself and tells your pipeline whether they passed, so a failing test stops the build. It's the same engine as the app, with no window. It reads the tests straight from your workspace, a whole suite or one test, and each run shows up in the app's run history like any other, marked CI. So the workspace stays the one place where tests are edited. It can also run a test file from a tests folder in your repo.

To run a suite on the local runner Mac instead, add a run request.

What you need

• A Breakpatch Team licence with a free machine licence for the CI machine. Machine licences count separately from people's seats.
• A Mac with Apple Silicon and macOS 14 or later, for example GitHub's macos-15 runners or a Codemagic Mac. Linux (x86_64 and arm64, a Raspberry Pi too) and Windows x64 work as a preview: they run tests, but record them on a Mac. See breakpatch-ci on Linux, breakpatch-ci on Windows and Raspberry Pi runner.
• Python 3.11 on that machine. When there's none, the install command downloads it.
• One run at a time per machine licence. To run tests in parallel, give each parallel job its own machine licence and its own BREAKPATCH_MACHINE_ID.
• The workspace's .bpworkspace file (Settings → Workspace → Invite teammates → Save as file), committed to your repo. It holds only the workspace's public details, never a password or key.
• A CI account: a user such as ci@yourcompany.com with the CI role (see Members and roles). Have it sign in to Breakpatch once, then change its role in Settings → Members. Its email and password go in your CI's secrets.
• Hosted by Breakpatch needs neither: a CI token and BREAKPATCH_WORKSPACE=hosted:<workspace> instead, with the machine key, since hosted workspaces are encrypted.
• The workspace's latest security rules: they let the CI account read tests and add its runs. After updating Breakpatch, an admin publishes them again (Settings → Workspace → Copy security rules).

Install

curl -fsSL https://breakpatch.dev/install-ci | sh

It installs breakpatch-ci and the test browser in ~/.breakpatch-ci and links the command into ~/.local/bin. It checks every download against the release's checksums, and needs no administrator password. In GitHub Actions the next steps can run breakpatch-ci straight away; elsewhere, add ~/.local/bin to PATH or use the full path. Running it again updates to the latest version, BREAKPATCH_VERSION=1.2.3 installs a given one, and sh -s -- --uninstall removes it. You can read the script first.

Run a suite from the workspace

breakpatch-ci run --workspace team.bpworkspace --suite smoke-7f3a

--suite takes the suite's ID (on the suite, under Start it from anywhere) or its name. The tests run one after another, as in the app: shared steps filled in, repeats, set-up and clean-up calls. Each test's run goes into its run history, and the suite's result shows as its last run in Suites. The log says which test is running and, when one fails, the step and why.

Run one test from the workspace

breakpatch-ci run --workspace team.bpworkspace --test web-app/Qx81cT2mNz

--test takes <app ID>/<test ID>, or the test ID alone: open the test's Version history and copy it under Run it from CI. A value ending in .json is read as a test file instead.

Which version runs

• --version latest (the default): the latest saved version, as the Run button runs it.
• --version released: the version marked in the test's Version history with Mark as released, so edits still in progress don't break the build. In a suite, tests with no released version are left out and listed in the result; if none has one, nothing runs and it ends with exit code 2.
• --version 3 (with --test only): that saved version.

Shared steps run the version each test's step pins, or their latest, as in the app.

Signing in. Set these from your CI's secrets. They're read once and never passed on to the browser, typed into a page or written to the log:

• BREAKPATCH_CI_EMAIL: the CI account's email address.
• BREAKPATCH_CI_PASSWORD: its password.
• BREAKPATCH_MACHINE_KEY: in an encrypted workspace, the machine key (bpmk1_…, Settings → Workspace → Encryption → Machine key). It opens the tests, and the runs breakpatch-ci writes back are encrypted with the workspace's key too.

Only an account with the CI role is accepted. To cut a pipeline off, disable the account in Firebase Authentication, or change its password.

--label TEXT says who asked, for example --label "GitHub · build 412". It's shown with the suite run in the app (the default is CI).

Run a test file

breakpatch-ci run --test breakpatch-tests/apps/web-app/tests/log-in.json

Shared steps are read from apps/<app>/shared/ next to tests/. The run isn't saved anywhere.

The result. It prints the result as JSON: for one test, each step's result; for a suite, the counts and each test's result, with the failed step and why. It ends with one of four exit codes:

If a run can't be saved in the workspace (for example the security rules are out of date), the log says so and the JSON has "saved": false. The exit code is still the test's result.

Reports for your CI. Add --junit results.xml to write the result as JUnit XML, which GitHub, GitLab and Jenkins show as test results, and --html report.html for the same report as the app's Export, screenshots included, to keep as a build artifact. --html-no-screenshots leaves the screenshots out (they can show personal data). Both work for a suite, a workspace test and a test file. The folder must exist; a file that can't be written is said in the log and doesn't change the exit code.

breakpatch-ci run --workspace team.bpworkspace --suite smoke-7f3a --junit results.xml --html report.html

The machine licence. Set these in the CI job's environment:

• BREAKPATCH_LICENCE_KEY: your licence key, stored as a CI secret. It's used to take a machine licence and is never saved or typed into a page. A CI token, the machine key or the recovery code put there by mistake is refused and never sent: breakpatch-ci says which it is and where it goes.
• BREAKPATCH_WORKSPACE: your workspace's Firebase project ID, for example acme-breakpatch. Or pass --workspace team.bpworkspace (then you don't need it). For a workspace hosted by Breakpatch, hosted:<workspace>, as Settings → Local runner → Add a CI token shows it.
• BREAKPATCH_MACHINE_ID: a fixed name for this pipeline, for example github-acme-web. CI machines are often new for every job; with a fixed name every job reuses the same machine licence instead of taking a new one.
• BREAKPATCH_LICENCE_FILE: where the licence is kept between runs. Keep this file between jobs, with your CI's cache: then most runs don't need to check online, and the usage counts it collects get sent with the next check (see Licences and seats).

breakpatch-ci licence status shows the licence (and takes a machine licence if needed). breakpatch-ci licence release gives the machine licence back, for example before you stop using a pipeline. An admin can also free it in the back office.

Saved secrets. A test that writes a saved secret, for example STAGING_PASSWORD, takes it from the environment variable BP_SECRET_STAGING_PASSWORD (a - or . in the name is written _), and from no other variable. Store the value as a CI secret. --secret STAGING_PASSWORD (you can give it more than once) limits which secrets a run may use. They're never the licence key or the CI account's password.

Where secrets may be used. The pipeline decides, not the test:

• Tests from the workspace. Anyone in your workspace can change a test's start page or an app's address. So a pipeline's secrets must not go wherever a test points. Name each secret's sites with --secret NAME=https://site, separating several sites with commas, for example --secret STAGING_PASSWORD=https://staging.acme.com. The secret is then typed only on those sites, and sent only to set-up and clean-up calls on them. Before anything runs, breakpatch-ci checks every test in the run. If a test uses a secret that isn't listed, or whose sites don't include the test's start page, its app's address or a call that sends it, nothing runs. It ends with exit code 2 and says which test and site.
• A test file is in your repo, so changes to it go through your usual review. --secret NAME without sites types it only on the test's start site, as before. You can also give its sites.

Keep the sites to your own test environments. A workspace member who can edit tests still can't make a pipeline type its secrets anywhere else.

Options

• --screenshots DIR keeps a screenshot of the step that failed, to upload as a build artifact.
• --auto-fix lets the AI assistant find a button that moved, as Fixed automatically does in the app. It only works on a self-hosted Mac where Breakpatch is installed and its AI assistant is downloaded (Settings → AI assistant), with a licence that includes Fixed automatically. Anywhere else the run carries on without fixing and says so. A simple runner never fixes.
• --fail-on-fix fails the run when a step needed fixing.
• --strict-systems turns off Allow for small differences between systems for this run (see below).
• --notify-url URL sends the result to Slack, Microsoft Teams or any web address when the run ends, as the local runner does (see Result messages). Anyone with the address can post to your channel, so keep it in a CI secret and set it as BREAKPATCH_NOTIFY_URL instead: then it never shows in the log. --notify-kind slack|teams|webhook says which format to send (Breakpatch picks it from the address), and --notify-when failures sends only failed runs (the default is every). The JSON gets "notified", with what Slack or Teams said if it didn't work; the exit code is still the test's result. When it starts failing, and when it passes again isn't available in CI, since the CI account can't read the suite's last result.

Record and run on the same kind of machine. Screen checks compare the page with how it looked when the step was recorded, and another system can draw text a little differently. Tests recorded on a Mac pass most reliably on a Mac with the same Breakpatch version. When a test was recorded on another system, breakpatch-ci says so in one line when the run starts, allows for small differences as the app's Allow for small differences between systems does (--strict-systems turns that off), and adds systemMismatch to the JSON, with the explanation when a check fails. See Recorded on another system.

GitHub Actions

jobs:
  ui-tests:
    runs-on: macos-15
    concurrency: breakpatch-ci          # one run at a time per machine licence
    env:
      BREAKPATCH_LICENCE_KEY: ${{ secrets.BREAKPATCH_LICENCE_KEY }}
      BREAKPATCH_MACHINE_ID: github-acme-web
      BREAKPATCH_LICENCE_FILE: ${{ github.workspace }}/.breakpatch/licence.json
      BREAKPATCH_CI_EMAIL: ${{ secrets.BREAKPATCH_CI_EMAIL }}
      BREAKPATCH_CI_PASSWORD: ${{ secrets.BREAKPATCH_CI_PASSWORD }}
      BP_SECRET_STAGING_PASSWORD: ${{ secrets.STAGING_PASSWORD }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - uses: actions/cache@v4
        with:
          path: .breakpatch
          key: breakpatch-licence-${{ github.run_id }}
          restore-keys: breakpatch-licence-
      - run: curl -fsSL https://breakpatch.dev/install-ci | sh
      - run: >-
          breakpatch-ci run --workspace team.bpworkspace --suite smoke-7f3a --version released
          --label "GitHub · build ${{ github.run_number }}" --secret STAGING_PASSWORD=https://staging.acme.com --screenshots shots
      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: breakpatch-screenshots
          path: shots

Codemagic (codemagic.yaml)

workflows:
  ui-tests:
    name: UI tests
    instance_type: mac_mini_m2
    environment:
      groups:
        - breakpatch                    # BREAKPATCH_LICENCE_KEY, BREAKPATCH_CI_EMAIL, BREAKPATCH_CI_PASSWORD, BP_SECRET_STAGING_PASSWORD
      vars:
        BREAKPATCH_MACHINE_ID: codemagic-acme-web
        BREAKPATCH_LICENCE_FILE: $HOME/.breakpatch/licence.json
    cache:
      cache_paths:
        - $HOME/.breakpatch
    scripts:
      - name: Install breakpatch-ci
        script: |
          command -v python3.11 || brew install python@3.11
          curl -fsSL https://breakpatch.dev/install-ci | sh
      - name: Run the UI tests
        script: |
          $HOME/.local/bin/breakpatch-ci run --workspace team.bpworkspace --suite smoke-7f3a --version released \
            --label "Codemagic · build $BUILD_NUMBER" --secret STAGING_PASSWORD=https://staging.acme.com --screenshots shots
    artifacts:
      - shots/**

To run a test file from the repo instead, use --test breakpatch-tests/apps/web-app/tests/log-in.json, leave out the sign-in variables, and --secret STAGING_PASSWORD may leave out its sites.

Troubleshooting

• "breakpatch-ci needs Python 3.11": install it (on a Mac, brew install python@3.11; in GitHub Actions, actions/setup-python), or point BREAKPATCH_PYTHON at it, and install again.
• "command not found: breakpatch-ci": add ~/.local/bin to PATH, or run ~/.local/bin/breakpatch-ci.
• Exit code 3 with "no machine licences left": each pipeline that runs at the same time needs its own machine licence. Set a fixed BREAKPATCH_MACHINE_ID so jobs reuse one, free old ones in the back office, or ask your admin to add one.
• Exit code 3 after the clock changed or a long time offline: the machine checks its licence online at least once a week. Make sure it can reach https://account.breakpatch.dev.
• "The saved secret STAGING_PASSWORD isn't on this Mac": set BP_SECRET_STAGING_PASSWORD in the job, and add the name to --secret if you use it.
• "The browser isn't installed yet": run the install command again. If BP_BROWSERS_PATH or PLAYWRIGHT_BROWSERS_PATH is set in the job, breakpatch-ci looks there instead: unset it.
• Linux: the browser doesn't start: install the libraries it needs with sudo ~/.breakpatch-ci/current/bin/python -m playwright install-deps chromium.
• Exit code 2 with "wrong email or password": check BREAKPATCH_CI_EMAIL and BREAKPATCH_CI_PASSWORD in the job's secrets.
• Exit code 2 with "not CI" or "isn't a member of this workspace": the account needs the CI role. Sign in to Breakpatch with it once, then an admin changes its role in Settings → Members.
• Exit code 2 with "The workspace refused the CI account": publish the latest security rules (Settings → Workspace → Copy security rules).
• Exit code 2 with "This workspace's tests are encrypted" or "isn't this workspace's machine key": set BREAKPATCH_MACHINE_KEY to the workspace's current machine key. After an admin makes a new one, update the secret.
• Exit code 2 with "would use the secret … which --secret … doesn't list": a test's start page, app address or set-up call is on a site you didn't give that secret. If the site is right, add it: --secret NAME=https://site1,https://site2. If not, someone changed the test: check its Version history.
• "Left out: no version is marked as released": open the test's Version history and Mark as released the version CI should run, or use --version latest.
• Checks fail in CI but pass in the app: look for the line "this test was recorded on…" at the start of the log. Re-record the test on a machine like the CI machine, or run it on a Mac.

breakpatch-ci on Linux

breakpatch-ci runs on 64-bit Linux, on x86_64 and on arm64 (a Raspberry Pi 4 or 5, or an arm server), as a preview: it runs tests there, and you record them on a Mac. It needs a system from the last few years (glibc 2.28 or later): Ubuntu 22.04 or later, Debian 12 or later, Raspberry Pi OS Bookworm (64-bit) or later, Fedora. A 32-bit system isn't supported, even on a 64-bit Raspberry Pi.

The install command is the same. When the machine has no Python 3.11 (Ubuntu 24.04 and Raspberry Pi OS come with a newer one), it downloads Python 3.11 into ~/.breakpatch-ci and checks it against the checksum written in the install command. To use your own, set BREAKPATCH_PYTHON.

Chromium needs some system libraries. Install them once per machine, as an administrator:

sudo ~/.breakpatch-ci/current/bin/python -m playwright install-deps chromium

breakpatch-ci setup installs the test browser again if it's missing (it needs no licence) and prints that line too.

On Linux, Chromium's own sandbox is off by default, because containers and most CI machines can't start it. On a machine that can (your own Linux server, a Raspberry Pi), turn it on with BP_SANDBOX=1.

GitHub Actions on Linux. The same job as above, with runs-on: ubuntu-24.04 (or ubuntu-24.04-arm for arm64). You don't need actions/setup-python. Add the libraries before the run:

      - run: curl -fsSL https://breakpatch.dev/install-ci | sh
      - run: sudo ~/.breakpatch-ci/current/bin/python -m playwright install-deps chromium

GitLab CI

ui-tests:
  image: ubuntu:24.04
  resource_group: breakpatch-ci        # one run at a time per machine licence
  variables:
    BREAKPATCH_MACHINE_ID: gitlab-acme-web
    BREAKPATCH_LICENCE_FILE: $CI_PROJECT_DIR/.breakpatch/licence.json
  cache:
    key: breakpatch-licence
    paths: [.breakpatch]
  script:
    - apt-get update && apt-get install -y --no-install-recommends curl ca-certificates
    - curl -fsSL https://breakpatch.dev/install-ci | sh
    - ~/.breakpatch-ci/current/bin/python -m playwright install-deps chromium
    - >-
      ~/.local/bin/breakpatch-ci run --workspace team.bpworkspace --suite smoke-7f3a --version released
      --label "GitLab · pipeline $CI_PIPELINE_ID" --secret STAGING_PASSWORD=https://staging.acme.com --screenshots shots
  artifacts:
    when: on_failure
    paths: [shots]

Set BREAKPATCH_LICENCE_KEY, BREAKPATCH_CI_EMAIL, BREAKPATCH_CI_PASSWORD and BP_SECRET_STAGING_PASSWORD as masked CI/CD variables. The job runs as root in the container, which the install command allows.

breakpatch-ci on Windows

breakpatch-ci runs on 64-bit Windows 10 and 11 on x64, as a preview: it runs tests there, and you record them on a Mac. Windows on arm isn't supported yet. Install it from PowerShell (no administrator rights needed):

irm https://breakpatch.dev/install-ci.ps1 | iex

It works like the install command for Mac and Linux. It installs breakpatch-ci and the test browser into %LOCALAPPDATA%\breakpatch-ci, uses Python 3.11 if it's installed (or downloads it, checked, into that folder), and adds %LOCALAPPDATA%\breakpatch-ci\bin to your PATH. Open a new terminal afterwards. For a given version, set $env:BREAKPATCH_VERSION = "1.2.3" first. To remove it, set $env:BREAKPATCH_UNINSTALL = "1" and run the same command.

GitHub Actions on Windows

jobs:
  ui-tests:
    runs-on: windows-latest
    concurrency: breakpatch-ci
    env:
      BREAKPATCH_LICENCE_KEY: ${{ secrets.BREAKPATCH_LICENCE_KEY }}
      BREAKPATCH_MACHINE_ID: github-acme-web-windows
      BREAKPATCH_LICENCE_FILE: ${{ github.workspace }}\.breakpatch\licence.json
      BREAKPATCH_CI_EMAIL: ${{ secrets.BREAKPATCH_CI_EMAIL }}
      BREAKPATCH_CI_PASSWORD: ${{ secrets.BREAKPATCH_CI_PASSWORD }}
      BP_SECRET_STAGING_PASSWORD: ${{ secrets.STAGING_PASSWORD }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/cache@v4
        with:
          path: .breakpatch
          key: breakpatch-licence-${{ github.run_id }}
          restore-keys: breakpatch-licence-
      - run: irm https://breakpatch.dev/install-ci.ps1 | iex
        shell: pwsh
      - run: >-
          breakpatch-ci run --workspace team.bpworkspace --suite smoke-7f3a --version released
          --label "GitHub · build ${{ github.run_number }}" --secret STAGING_PASSWORD=https://staging.acme.com --screenshots shots

The install command adds its folder to GITHUB_PATH, so the next steps can run breakpatch-ci.

A Windows PC as a runner. To run a suite on a schedule without the app, use Task Scheduler. breakpatch-ci service writes the task for you; run schtasks from a terminal opened as administrator:

breakpatch-ci service --windows --suite smoke-7f3a --workspace C:\breakpatch\team.bpworkspace --schedule "Mon..Fri 06:00" --version released --xml smoke.xml
schtasks /Create /TN "Breakpatch\smoke-7f3a" /XML smoke.xml /RU breakpatch /RP *

The task runs breakpatch-ci run --workspace … --suite … at those times, as the account you give (breakpatch here, a local account made for it), whether or not it's logged on. schtasks asks for that account's password. --schedule takes days and a time, such as Mon..Fri 06:00 or Sat,Sun 08:15, or hourly. Give the account the same variables as a CI job: sign in as it, then System → Advanced system settings → Environment Variables. The task itself holds no secrets. Chromium runs without a desktop. To remove the task: schtasks /Delete /TN "Breakpatch\smoke-7f3a" /F.

Raspberry Pi runner

A Raspberry Pi makes a cheap, always-on machine that replays your workspace's suites on a schedule and reports into the workspace, like a CI job that never stops. It's breakpatch-ci on Linux, so it's a preview too, and it isn't a runner of its own yet: a systemd timer starts each run (below). It replays tests. It doesn't record them (record on a Mac).

Which Pi. These are our recommendations; we haven't measured every model yet.

• Raspberry Pi 4 with 4 GB: a simple runner. It replays tests, checks the screen and runs suites on a schedule. It doesn't fix moved buttons.
• Raspberry Pi 4 with 8 GB: should be fine for nightly and hourly replays of normal web pages. It's a full runner, with longer waits because its processor is slow. It doesn't fix moved buttons either: the AI assistant only runs on a Mac today.
• Raspberry Pi 5 with 8 GB: faster, and the better buy if you're buying one now.
• A small x64 mini PC (for example one with an Intel N100): faster again, and the best choice for Flutter apps (below). It uses the same Linux install as any CI machine.
• Use an SSD over USB 3 (or NVMe on a Pi 5) rather than an SD card, which wears out with screenshots and logs. Give it cooling (a Pi 4 slows down at 80 °C) and the official power supply (27 W on a Pi 5).
• Install a 64-bit system: Raspberry Pi OS Bookworm (64-bit) or later, or Ubuntu 24.04 for arm64.
• On a Pi 5, the standard kernel uses 16 KB memory pages. If Chromium misbehaves, add kernel=kernel8.img to /boot/firmware/config.txt and restart: that kernel uses 4 KB pages.

Runner tiers

Each time breakpatch-ci starts a run, it checks how much memory the machine has and how fast its processor is. That takes less than a second. The memory picks the tier: under 6 GB, it's a simple runner. The processor's speed only sets how long it waits. The first lines of the log say which tier and why, for example:

breakpatch-ci: simple runner (4 GB of memory, under the 6 GB the full tier needs). Replay, screen checks and schedules only: no AI assistant and no fixing. Waits are 2× as long (BP_TIMINGS_SCALE changes that).

A Raspberry Pi 4 with 4 GB is a simple runner. A Pi 4 with 8 GB is a full runner with longer waits: the processor's speed only sets the waits.

The tier is also in the JSON result, as runner (tier, reason, memoryGb, cpuSpeed and timingsScale), and in the --html report, on the Runner line under Machine.

To try the other tier, set BREAKPATCH_TIER=simple or BREAKPATCH_TIER=full, or add --tier simple to breakpatch-ci run. That's for testing: choosing full doesn't give a machine more memory.

Flutter web apps. Replay works, because it compares the screen and needs no AI assistant, but Flutter is the hardest case for a Pi:

• Flutter draws everything with WebGL, and a Pi's headless Chromium draws it in software. The first load and each step's settling are slow: use the slow-machine setting (BP_TIMINGS_SCALE, below).
• Software drawing on an arm Linux machine looks slightly different from a Mac. Re-record Flutter tests on the same kind of machine as the runner (or allow for small differences between systems, which breakpatch-ci does by default), and check the first runs.
• There's no automatic fixing for Flutter buttons that moved: nothing can read inside a canvas, and the AI assistant is too slow on a Pi.

A Pi 4 with 8 GB suits nightly Flutter runs, a Pi 5 is better, and an x64 mini PC is best.

Slow machines. On a slow processor breakpatch-ci waits longer for a page to settle, for a step's first check to match, for pages to load and for the start page: 2× on a Pi 4, on either tier. It doesn't stretch the watch for moving parts before each step, so a page that's ready on time costs nothing extra. To choose yourself, set BP_TIMINGS_SCALE to a number from 1 to 10: it wins over the tier's. Use 2 for a Pi 5 and 3 for a Pi 4 with Flutter. If steps fail with "didn't settle" or "not there yet", raise it. For a runner set up as below, put the line BP_TIMINGS_SCALE=3 in /etc/breakpatch/breakpatch.env.

Set it up

1. Make a user for the runner, and install breakpatch-ci as that user:
   Copy

   sudo useradd --create-home --shell /usr/sbin/nologin breakpatch
   sudo -H -u breakpatch sh -c 'curl -fsSL https://breakpatch.dev/install-ci | sh'
   sudo /home/breakpatch/.breakpatch-ci/current/bin/python -m playwright install-deps chromium

2. Have ready a machine licence, the workspace file and a CI account, as for any CI machine (see From CI with breakpatch-ci). Then let breakpatch-ci service write the service and its timer:
   Copy

   sudo /home/breakpatch/.local/bin/breakpatch-ci service --suite smoke-7f3a --workspace team.bpworkspace \
     --email ci@yourcompany.com --machine-id pi-runner-1 --schedule "Mon..Fri 06:00" \
     --version released --secret STAGING_PASSWORD=https://staging.acme.com

   It asks for the licence key, the CI account's password, the machine key (only for an encrypted workspace) and each saved secret you name with --secret. What you type isn't shown, and it never goes on the command line. It keeps each one in a file of its own in /etc/breakpatch that only root can read. Then it starts the timer.--suite takes the suite's ID (on the suite, under Start it from anywhere). --schedule takes systemd's OnCalendar= form: Mon..Fri 06:00 (the default), *-*-* 02:30 for every night, or hourly. For more suites, run it again with another --suite. The workspace, the licence key and the password are kept from the first time.

3. Start a first run now, and read its log:
   Copy

   sudo systemctl start breakpatch-suite@smoke-7f3a        # waits until the suite has run
   journalctl -u breakpatch-suite@smoke-7f3a               # its log and result
   systemctl list-timers 'breakpatch-*'                    # the next runs

The service passes the licence key, the CI account's password and saved secrets as systemd credentials: only the run can read them, and they never appear in systemctl show or the log. It turns on Chromium's sandbox (BP_SANDBOX=1). Failure screenshots go to /home/breakpatch/breakpatch-shots. The files it writes are the ones in the Breakpatch repository's docs/systemd/, if you'd like to read them first. --root DIR writes them under a folder of your choice instead and starts nothing.

Without root. breakpatch-ci service --user … writes the service in your own account instead (~/.config/systemd/user), with the secrets in ~/.config/breakpatch/secrets.env, which only you can read. Use systemctl --user and journalctl --user in place of sudo systemctl and journalctl. So that it runs while you're logged out, run sudo loginctl enable-linger $USER once.

Remove it. sudo /home/breakpatch/.local/bin/breakpatch-ci service --suite smoke-7f3a --remove stops the suite's timer and removes its files. After the last suite, it also removes the service, the settings and the secrets.

What's different from the app's local runner

• The schedule is the timer's. The days and times set on a suite in the app aren't used.
• Runs show in the run history marked CI, not as the local runner's.
• There's no Run now from the app, and the app shows the runner as offline.

Copying the SD card or SSD. Each Pi needs its own machine ID. If you copy a system that has already started once, run sudo rm /etc/machine-id && sudo systemd-machine-id-setup on the copy, and give it its own BREAKPATCH_MACHINE_ID (--machine-id). Otherwise both Pis use one machine licence and push each other out.

Result messages Team

After each run, the local runner can send the suite's result where your team talks: Slack, Microsoft Teams, or any web address that takes a message. Set it in the suite (After each run, send the result to). Only admins can set or see it.

When. After every run, Only when it fails, or When it starts failing, and when it passes again: the first failed run after a pass, then the first pass after that. A run that was replaced by a newer request is only sent with After every run.

Slack. Make an incoming webhook for the channel (a Slack app with Incoming Webhooks turned on) and paste its address, https://hooks.slack.com/services/…. The message has the result and the counts with an Open the report button beside them, then one line per failed test with the step and why (the AI assistant's sentence, when there is one, is marked as the AI assistant's), and the screenshot, if you include it, right after the failure it shows. The notification says how many tests failed and which.

Microsoft Teams. In the channel, add the Workflows template Send webhook alerts to a channel (or a workflow that starts with When a Teams webhook request is received), and paste its address. It ends in environment.api.powerplatform.com. The message is an Adaptive Card with the same things, and the same line as its notification preview. Microsoft turned off the old Office 365 connector addresses (….webhook.office.com) in Teams, so Breakpatch doesn't take them. If Teams says it wants a sign-in, set Who can trigger the flow to Anyone in the workflow's first step. Older Workflows addresses on logic.azure.com are being replaced by Microsoft: if the test message fails, copy the workflow's address again.

Include the failed step's screenshot (Slack and Teams, off by default). Slack and Teams can only show a picture from a web address, so the runner puts the screenshot of the step that failed in your workspace's own Firebase Storage, at a random address. Anyone who has the message can open the picture. It needs Storage turned on in your Firebase project (new projects need the pay-as-you-go plan for it) and its rules published: Settings → Workspace → Copy storage rules, then Firebase console → Storage → Rules → Publish. The first time, the console asks whether Storage may read Firestore: allow it, because the rules check that whoever adds a picture is one of the workspace's people, its runner or its CI account. Update the runner Mac before you publish these rules: older apps put pictures where the new rules don't allow. Without the rules, the message goes without the picture. Pictures aren't deleted for you: to keep Storage small, add a lifecycle rule in the Google Cloud console (Cloud Storage → your bucket → Lifecycle → delete objects older than 90 days).

The address is a secret. Anyone who has it can post to your channel. So it isn't stored with the suite: only admins and the runner can read it, not members or the CI account. Once saved it shows masked, like hooks.slack.com/services/•••••; press Change to paste a new one. It never goes in a message, a warning or the log.

Send a test message posts a sample result to the address before you rely on it, and shows what Slack or Teams answered, for example "Slack doesn't know this webhook any more".

If the address doesn't answer, the runner tries 3 times over 5 minutes. If Slack or Teams refuses the message (a removed webhook, say), it stops at once. Either way the runner's status says so.

The report link. Open the report goes to https://breakpatch.dev/report#r=…, which opens the run's report in Breakpatch on a Mac, and on a phone says to open it on a Mac. The part after # never reaches a server.

A web address gets the result as JSON, in a POST:

{
  "suite": "Smoke",
  "suiteId": "smoke-7f3a",
  "result": "failed",
  "counts": { "total": 6, "passed": 4, "fixed": 0, "failed": 1, "notRun": 1 },
  "failures": [
    { "test": "Create a project", "step": "Step 5: Click Done", "reason": "couldn't find the Done button" }
  ],
  "requestedBy": "CI · build 412",
  "note": "",
  "startedAt": "2026-09-24T14:52:03.000Z",
  "durationSeconds": 292,
  "reportLink": "breakpatch://report/web-app/run_8f21c",
  "reportUrl": "https://breakpatch.dev/report#r=web-app/run_8f21c",
  "text": "Smoke failed: 1 of 6 tests failed, 1 not run · Create a project · 4 min 52 s · started by CI · build 412"
}

result is one of passed, passed_with_fixes, failed or replaced. reportLink opens the report in Breakpatch on a Mac, and reportUrl does the same from anywhere (both are empty when there's no run to open). A failure can also have explanation, the AI assistant's sentence on why, and the message imageUrl, the screenshot, when the suite includes one, and htmlReportUrl, the runner's saved HTML report, when its folder has a web address. Use text as it is, or build your own message. Any relay works: n8n, Zapier, an email service. Discord and Google Chat take it through such a relay.

breakpatch-ci sends the same messages with --notify-url.

Updating. Result addresses saved by an earlier Breakpatch were kept with the suite, where the CI account could read them. When an admin opens the updated app, it moves them to where only admins and the runner can read them. Update the runner Mac first, then publish the new security rules. Older apps can't save in a workspace once the updated app has saved there (they say to update).

Create an issue Team

On a failed step in a report, Create issue writes it up in GitHub, Linear or Jira Cloud in one click: the title ("Create a project: step 5 Click Done failed: couldn't find the Done button"), the steps up to the failure, what was expected and what was seen, the reason, the app, the start address, the test version, who or what ran it (a person, the local runner or CI), the Mac, the time and a link to the report. If the AI assistant explained the failure, its explanation goes in too. Afterwards the report shows Open issue instead, for everyone in the team.

The screenshot. Jira gets the failed step's screenshot as an attachment, and Linear as a picture in the issue. GitHub has no way to attach one, so it's put in your workspace's Firebase Storage and linked, as for result messages (it needs Storage and its rules); without Storage the issue goes without it. The screenshot is on the Mac that ran the test, so an issue made on another Mac, or from a runner or CI run, has no picture.

Set it up in Settings → Issue trackers:

1. Where issues go (admins, for the whole team): the GitHub repository (acme/web) and labels, the Linear team's key (ENG), or the Jira site (acme.atlassian.net), project key (WEB), issue type (Bug unless you say otherwise) and labels.
2. Your token, on this Mac (each person): issues are made in your name, with your own token.
   • GitHub: a fine-grained personal access token with access to the repository and Issues: Read and write.
   • Linear: a personal API key (Linear → Settings → Security & access).
   • Jira: an API token with the email you sign in to Jira with.
   Check and save asks the tracker whether the token works, then keeps it in this Mac's Keychain. It's only used to make issues, and never goes to the workspace, your teammates or Breakpatch. Remove takes it off this Mac.

If a tracker refuses, Breakpatch says why in plain words, for example "GitHub said the token can't create issues in acme/web".

Create issue comes with a Breakpatch Team licence that includes it; licences get it at their next check. In Community, Copy → Markdown, for an issue gives the same text to paste yourself.

Security rules Team

Copy rules in workspace setup gives rules with your email domain filled in. They say:

• Only signed-in accounts from your email domain, with a confirmed email address, can read or write. Runner and CI accounts are let in by their role.
• There are four roles: member, admin, runner and ci, as in Members and roles. Whoever connects the workspace is the first admin.
• Versions can only be added. History can't be edited.
• Deleting an app, a test, shared steps or a suite marks it as deleted (Recently deleted), and only whoever may delete it can mark it or take the mark off. Only then can it, its versions and, for an app, everything in it be deleted for good. Breakpatch from before this can't delete.
• runRequests accepts a small document of a fixed shape from members and from accounts with the ci role. Only the runner can read or delete them.
• Accounts with the ci role read apps, tests, shared steps, their versions and suites, and add runs marked ci as themselves. They can't change anything, or read members, runs or the licence.
• A test's released version must be one that's saved.
• Every change to apps, tests, shared steps or suites also updates workspace/changes, a small document that tells the other Macs what changed, so they don't read everything again. Apps from before this can't save until they're updated.
• Runs and suite runs must carry expiresAt, 90 days after they're saved, which the TTL policy deletes them by. It can't be changed. Runs from Breakpatch or breakpatch-ci from before this are refused, so update them first.
• Only admins set where a suite's result goes. Its address is kept apart from the suite, and only admins and the runner can read it.
• Anyone in the team can note the issue made from a run, and change nothing else about it. Only admins set where issues go. Tokens for GitHub, Linear and Jira are never in the workspace.
• In an encrypted workspace, names, steps, addresses and results are stored encrypted, and the rules check only their size. Only admins can encrypt what's there again (turning encryption on, a new key), and nothing else about it. The copies of the key for the recovery code and the machine key are for admins (the machine key's for the runner and CI too), and each Mac's request to be let in is its owner's.

After an update. The rules carry a date (search what you copy for These rules are dated), and each Breakpatch version needs its own date's rules or newer: this version's are dated 2026-10-06. In a workspace in your own Firebase, an admin copies them again after updating (Settings → Workspace → Copy security rules) and publishes them in the Firebase console → Firestore → your database → Rules. Until then, Check the connection says the rules for this version aren't published, and each Mac reads the whole workspace every time it opens it. The 2026-10-06 rules also make sure a removed admin can't change which Mac's signature was withdrawn (see Encryption and the recovery code). Hosted by Breakpatch, the rules are always up to date.

To give an account the ci role, have it sign in to Breakpatch once, then change its role in Settings → Members. Or, in the Firebase console, set role to "ci" in its document members/<user id> in the workspace's database. Disable the user in Firebase Authentication to cut it off.

A service account using the Firebase Admin SDK skips the rules altogether. Use a normal user with the ci role instead: breakpatch-ci only signs in with one.

Licences and seats Team

A Team licence has seats for people and machine licences for the local runner and CI machines. The licence belongs to the workspace.

• Hosted by Breakpatch needs no key in the app: Breakpatch gives each person their seat when they sign in. The rest of this section is the same.
• The admin sets up the licence once (in a workspace of your own): Settings → Licence → Enter licence key (it looks like BP-XXXX-XXXX-XXXX-XXXX). The key is on your licence page after you buy: see The back office.
• Members don't enter a key. Everyone else gets a seat by themselves when they join through the invite link or sign in. Only admins can see the key. Until the admin has entered it, members see "Your admin hasn't set up the licence for this workspace yet."
• Out of seats: the member sees "Your team is out of seats. Your admin can see you're waiting and can add a seat or free one." Admins see who is waiting in Settings → Licence.
• A new key: if the admin replaces the key, members' seats follow once an admin opens Breakpatch.
• The local runner and CI machines use a machine licence. Machines count separately from people. CI machines take theirs with the key: see From CI with breakpatch-ci.
• Each person takes a seat when they sign in. A seat someone already holds keeps refreshing. Signing out gives the seat back; so does Release this Mac (admins).
• A seat is tied to the Macs it's used on: one person can use it on a few of their own Macs, and a copy of the Keychain on another Mac doesn't work. On too many Macs you see "Your seat is already used on too many Macs. Ask your admin to free one, then sign in again."
• A seat nobody has used for 30 days is freed automatically. An admin can also free one in the back office.
• Breakpatch checks the licence when it opens and every day, and keeps working for up to 30 days without a connection. If this Mac's clock is set back by more than a day, the licence stops working until Breakpatch can check it online again: "This Mac's clock is behind. Set the right date and time, then reconnect to check your licence."
• Saved secrets in CI. breakpatch-ci takes a saved secret's value only from an environment variable named BP_SECRET_<NAME>, for example BP_SECRET_STAGING_PASSWORD for STAGING_PASSWORD (a - or . in the name is written _), never from other variables. --secret NAME (you can give it more than once) limits which secrets a run may use. It types them only on the test's start site, or, for tests from the workspace, only on the sites the pipeline gives each one (--secret NAME=https://site, see From CI with breakpatch-ci).
• Usage counts go with the licence check. Each check sends how many tests were created and how many runs there were (by hand, from schedules, on the local runner and in CI), passed and failed, and on how many days Breakpatch was used, since the last check. Only numbers: never test names, addresses, steps or screenshots. The licence check already knows the licence and seat, so your admin sees the numbers per person and machine in the back office. They're part of how the licence works, so there's no switch for them in Team. breakpatch-ci counts its runs the same way in its licence file and sends them with its next check; on CI machines that don't keep that file between jobs they aren't sent.

Without a licence, Breakpatch keeps working as Community (tests, recording and running on this Mac) and Team features are off. A quiet banner under the title bar says why: "Breakpatch Team needs a licence. Your admin enters the key once in Settings → Licence, then everyone gets a seat when they sign in.", "Your team is out of seats. Ask your admin to add one.", "Reconnect to check your licence." (after 30 days offline) or "Your licence has expired." A runner with no machine licence left gets "Your team has no machine licences left. Ask your admin to add one."

With Solo, the messages are for one person: "This Solo licence is in use on another Mac. Free it at account.breakpatch.dev, then try again.", "Your Solo licence has expired. Renew it at account.breakpatch.dev." or "This Solo licence is already used by someone else. Solo is for one person: for more people, choose Team." The banner then has a button for what to do there, such as Free the other Mac or Renew, and Home doesn't offer Upgrade to Team while it shows.

Other messages you may see:

• "This licence key isn't recognised. Check it and try again."
• "This licence has expired. Ask your admin to renew it."
• "This licence is no longer active. Ask your admin for help."
• "This seat was freed. Sign in again to take a seat."
• "Your team is using more seats than it pays for, so this Mac's seat was freed. Ask your admin to add seats, then sign in again."
• "This licence was activated on another Mac. Sign in again on this one."

The back office Team

Your licence page is at account.breakpatch.dev. Enter your email and open the sign-in link we send you. There's no password.

If you're one of your licence's admins, you see:

• Your plan, when it renews and how many seats are used, for example 8 of 10 seats used.
• Who holds each seat and when it was last seen, and the machines.
• Free a seat, for someone who has left or a Mac you no longer use. Freeing a seat lets someone else take it.
• Usage: tests created and runs each week for the last 12 weeks (by hand, schedules, local runner and CI), the pass rate, and the same per person and machine.

Once Team is on sale: if you bought it on the pricing page, sign in with the email you bought with: the first time, you see Your new licence key (shown once). Press Show my licence key and keep it somewhere safe, because it isn't shown again. The same page has Manage billing (invoices, payment method, cancelling, through Paddle, our reseller) and Change seats (seats and extra machine licences, charged or credited straight away, pro rata for the rest of the billing period; credits go against your next payments; Solo has no seats to change). A cancelled licence keeps working until the end of the period you paid for.

For a licence you didn't buy on the site, contact Breakpatch to add seats or renew. If you see "No licences for this email", ask whoever bought Breakpatch Team to add you as an admin.

Encryption and the recovery code Team

Breakpatch can encrypt a workspace's tests on your Macs, so the workspace's database keeps only unreadable text. Workspaces hosted by Breakpatch are always encrypted. For a workspace in your own Firebase, an admin turns it on in Settings → Workspace → Encryption (publish this version's security rules first: Copy security rules). It can't be turned off again.

What's encrypted. Test, app, suite and shared-step names and descriptions; steps, with what they click and type, their addresses and labels; start addresses; run results, their steps and "Why did this fail?" explanations; where results and issues go. What isn't: ids, times, version numbers, pass or fail and counts, when a run expires, and who is in the team with which role. The workspace needs those to check who may do what and to delete old runs, and they say nothing about what your tests do.

The key. Each workspace has one key, made on the first admin's Mac and kept in its Keychain. It never leaves your Macs unencrypted, and Breakpatch never has it. A teammate's Mac gets it one of two ways:

• An invite link with the key: Settings → Workspace → Copy link with the key. The key is in the part after #, which browsers never send to a server. Send it only to teammates.
• An admin lets the Mac in: a Mac without the key shows This Mac can't read the workspace's tests yet with four words, for example otter maple tulip quartz. The admin sees it under Macs waiting to be let in, checks the words with its owner, and presses Let in. If this Mac doesn't know that admin yet, it then shows An admin let this Mac in. Check it was them. with the admin's Mac's words: compare them with the ones in their Settings → Workspace → Encryption and confirm. A Mac takes a key only from an admin it trusts (from the invite link, or one you confirmed, or one a trusted admin vouched for); anything else is refused and said, for example The words didn't match, so this Mac didn't take the key.

Until then, that Mac shows names as Locked and can't save. After a new key, a Mac that hasn't got it yet says This Mac doesn't have the workspace's newest key yet: it can read, but not save, until an admin's Mac passes the key on the next time it opens the workspace. Once encryption is on, content that isn't encrypted (saved by an older Breakpatch, for example) shows as Locked too and isn't saved that way again; the workspace's rules and breakpatch-ci refuse it as well.

The recovery code. When encryption is turned on, the admin sees a recovery code once, like BPR1-50M6-HA79-55MT-KTHA-DANE-PAVB-NFH. Save it with Save recovery kit (PDF) or Copy code, keep it in your password manager or with your company's papers, then type back the 4 characters Breakpatch asks for. Until you do, admins see Save the workspace's recovery code under the title bar. The window doesn't close with Esc or a click outside it; Skip, I'll make a new one later asks first, because the code isn't shown again. Breakpatch never sees the code and can't make it again. A recovery code (BPR1-…) pasted where a licence key goes is refused, and never sent.

• I lost access to the key: on a new Mac, sign in as an admin and open Settings → Workspace → Encryption → I lost access to the key, then type the recovery code. Any Mac that still has the key can also let the new one in.
• Make a new recovery code stops the old one working. Save the new one straight away.
• If every Mac with the key is lost and nobody has the recovery code, the tests saved so far can't be read again, by you or by Breakpatch. An admin can Start again with a new key (under the help text in I lost access to the key, and it asks first): new tests and runs are encrypted with it, and older ones stay locked.

The local runner and CI use the machine key (bpmk1_…): Make one under Machine key, then enter it on the runner Mac (Settings → Workspace → Encryption) and save it as BREAKPATCH_MACHINE_KEY in CI (From CI with breakpatch-ci). A new machine key stops the old one working.

A new key. After someone leaves the team, Breakpatch offers Make a new key (it's also in Settings → Workspace → Encryption). Every Mac still in the team gets it, the recovery code and machine key keep working, and what's in the workspace is encrypted again with it in the background. Once that's done, each Mac deletes the older keys from its Keychain, so a copy of an old key opens nothing new. Anyone who had access may have kept copies of what they could read.

When an admin leaves. When an admin is removed, or made a member, another admin's Breakpatch withdraws their Mac's signature: from then on, the team's Macs don't take a key from it or trust a Mac it let in later, even if that Mac is kept or stolen. Keys taken from them before stay, and Breakpatch signs again what only they had signed. Admins then see Ana left the team with the workspace's key, with Make a new key. In your own Firebase, do it, since they could keep a copy of the key; hosted by Breakpatch, their access has already ended, so it's extra care. Not now hides the note on that Mac until someone else leaves; Settings → Workspace → Encryption still says a new key is due. In a workspace in your own Firebase, Make a new key also asks you to check they can't still open your Firebase project (Google Cloud console → IAM) and to remove them there if they can: Breakpatch can't see who has access to your project. If Breakpatch couldn't withdraw the signature, admins see The team's Macs still trust an admin who left. Hosted by Breakpatch, it tries again when the workspace changes; in your own Firebase, press Copy security rules on the note, publish them in the Firebase console and open the workspace again. A recovery code or machine key they made stops working, because they saw it. Admins see Make a new recovery code or Make a new machine key; a new key made after they left comes with a new recovery code to save. After a new machine key, enter it on the runner Mac and update BREAKPATCH_MACHINE_KEY in CI. If an admin's Mac was the only one a teammate's Mac trusted, that Mac asks its person to check another admin's words the next time it's let in. Someone made admin again gets new keys on their Mac, and another admin's Mac vouches for them.

Turning it on, or a new key, encrypts what's already there in the background. The admin's Mac doing it shows how far it got in a thin line under the title bar, for example Encrypting your tests… 34 of 120, or Encrypting your tests with the new key… (tests count with their versions; runs, shared steps and suites count too). Keep working, and Your tests are encrypted. says when it's done. If you close the app, it carries on the next time an admin opens the workspace, from where it was. If it stops, a note says why, for example Encrypting your tests stopped at 40 of 120. Breakpatch couldn't reach the workspace., and Try again carries on; what's encrypted so far stays encrypted. Breakpatch versions from before encryption can't save to an encrypted workspace, so update everyone first.