Why real SMS breaks CI

Pointing your test suite at a live messaging provider fails in four predictable ways:

  • Cost. Every verification sends a billable message. A suite that runs 200 times a day across PRs turns your test budget into an invoice line item.
  • Latency. Real delivery takes seconds to minutes. A 30-second OTP wait inside a 90-second Playwright timeout is a coin flip.
  • Carrier filtering. Carriers aggressively filter A2P traffic to short-code ranges and test-looking numbers. Messages silently vanish; your test can’t distinguish “code not sent” from “code lost.”
  • Nondeterminism. Codes arrive out of order, retries collide, shared test numbers accumulate state from other runs. The result is exactly the kind of intermittent red described in our guide to flaky tests.

Some teams buy dedicated test phone numbers or negotiate sandbox access. Both still route through carrier networks, so latency and loss remain — you’ve paid for a faster coin flip, not determinism. Number pools shared across CI jobs introduce a subtler failure: two pipelines verifying against the same number concurrently, each consuming the other’s codes. The fix is removing the network from the loop entirely, which is what service clones are for.

01 · ISSUE

SMS code created

Record the message and its issued-at time.

captured
02 · VALID CASE

Code accepted

Independent attempt: submit the fresh code before its validity window closes.

account verified
03 · EXPIRED CASE

Old code rejected

Independent attempt: submit the original code after its validity window.

expiry asserted
04 · RESEND CASE

Fresh code only

Independent attempt: request again and keep the original code invalid.

retry asserted
An OTP is a lifecycle, not one inputAfter one issuance, exercise valid, expired, and resend cases independently from a clean start.

Inspect the messages captured by your test service

A Twilio clone implements the provider’s API contract — sending messages, looking them up and verification codes — but keeps every message in a local, inspectable store. Your application doesn’t change: it still POSTs to a base URL, still reads back status callbacks. Only the base URL points at the clone.

Example to adapt: helper functions, accounts, and endpoints depend on your test project. Adapt this example to your project.

Code example
# App code is unchanged except TWILIO_BASE_URL
curl -X POST "$CLONE/2010-04-01/Accounts/ACxxx/Messages.json" \
  -d To=+15550100042 -d From=+15550001111 \
  -d Body="Your code is 491722"

# The clone stores it deterministically — fetch it like the real API
curl "$CLONE/2010-04-01/Accounts/ACxxx/Messages.json?To=%2B15550100042"
# → {"messages":[{"body":"Your code is 491722","status":"delivered", ...}]}

Determinism is the point: for a given seed, the clone issues the same codes, in the same order, with the same timestamps. Your assertions stop guessing and start knowing. And because nothing leaves the machine, there’s no cost per run and no PII touching a third party — the same safety argument we make for third-party APIs in general.

Test what happens when a code expires

OTP expiry is where hand-rolled tests give up. Nobody wants to wait five minutes for a code to lapse, so expiry logic ships untested until it breaks. A clone with a virtual clock solves this cleanly: advance simulated time, then act.

Code example
// Advance the clone's clock past expiry, then try the stale code
await clock.advance({ seconds: 301 });
await page.fill('#otp-input', codeFromClone);   // issued at t=0
await page.click('text=Verify');
await expect(page.getByText('Code expired')).toBeVisible();

// Request again → new code, old code rejected even if typed correctly
const fresh = await clone.getLatestCode('+15550100042');
expect(fresh).not.toBe(codeFromClone);

This covers the whole family of time bugs: resend throttles, retry lockouts after N attempts, “code valid for X minutes” copy drifting from server truth, and replay attempts. These are precisely the behaviors attackers probe, which makes them worth automating rather than spot-checking by hand.

Check delivery callbacks and signatures

Most integrations don’t poll — they consume webhooks: delivery receipts, inbound replies (STOP/HELP), verification status events. If your clone only fakes the REST API, you’ve tested half the surface. Look for three things in any substitute:

  • Lifecycle parity. queued → sent → delivered transitions fire in order, with realistic timing you control.
  • Signature correctness. Callbacks carry valid request signatures so your verification middleware is exercised, not bypassed.
  • Inbound simulation. You can inject an inbound “STOP” reply and assert your suppression logic fires.
Code example
// Simulate an inbound STOP and verify opt-out handling
await clone.injectInboundSms({ from: '+15550100042', body: 'STOP' });
await expect.poll(() => clone.suppressionList())
  .toContain('+15550100042');

Webhook parity also changes what you can test about ordering. Real providers retry undelivered callbacks and can deliver events out of order under load; a clone lets you replay those conditions on demand instead of waiting for a bad day in production. Assert that a duplicate verification.completed event is absorbed idempotently, that a status callback for a message your app never sent is rejected, and that signature failures return 403 rather than 500 — small checks that each map to a real outage class.

Connect the clone to your test runner

In practice the pattern is small: point your app’s provider credentials at the clone before boot, run your flows, pull codes out of the clone when the UI asks for them.

Code example
// Playwright fixture
export async function otpFor(phone) {
  const msg = await clone.latestMessageTo(phone);
  return msg.body.match(/\b(\d{6})\b/)[1];
}
test('signup verifies by phone', async ({ page }) => {
  await signup(page, '+15550100042');
  await page.fill('#otp', await otpFor('+15550100042'));
  await expect(page.getByRole('heading', { name: 'Welcome' })).toBeVisible();
});

The Molar Twilio clone reference lists supported SMS paths and limits. Configure test numbers and endpoints, inspect the captured message, and verify the account state after the code is submitted. For adjacent steps, see email testing and payment testing.

Keep one smoke check against the live provider behind a manual trigger. Clones verify your logic; a weekly manual pass confirms the vendor hasn’t changed their contract underneath you.