Method choice is a budget

A green CI badge does not mean the build is safe to ship. I have watched fuzz passes and hundred-case UI suites miss production failures — vendor-specific rotation bugs, logic errors behind a rare branch, boundary values at Integer.MAX_VALUE — because the team treated “we tested” as one checkbox. Each technique buys a different kind of confidence at a different cost. The engineering question is not whether you tested; it is which failure class earns the next hour of automation.

Below I pick four failure classes I actually fear on mobile and UI work, and one primary method that earns its keep for each. Everything else folds into a short toolbox note — not a glossary tour.

Crash and corruption — fuzz first

When the failure you fear is a native crash, an unhandled exception, or memory corruption at a boundary (JNI, intent extras, protobuf decode), fuzz testing is the budget winner. Feed invalid, unexpected, or random inputs at volume. You do not need a formal model — point a fuzzer at a parser or IPC handler and let it hammer.

What it proves: the process stays up under hostile input. What it does not: a layout that renders wrong but never throws looks like success. Highly conditional UI bugs (“locale is ar and night mode toggles mid-animation”) may never appear in random bytes.

On Android, fuzzing intent extras, malformed content URIs, or sync payloads is high leverage. It is a poor sole strategy for Compose visual regressions. Spend fuzz hours where a crash is the oracle you trust.

UI is wrong but does not crash — need an oracle

When the failure is “the screen looks or behaves wrong,” crash oracles are useless. You need a method whose pass/fail comes from intended behavior, not from “no exception.”

Two budget shapes work here, and they are not interchangeable:

• Model-based testing when the flow has an explicit state machine you can maintain — navigation graph, permission × deep-link matrix, Compose screen contracts. Illegal transitions and unreachable states are the oracle. Cost sits in model upkeep; payoff is combinatorial coverage hand-written cases never finish.
• Stochastic / random event sequences when you mainly need stress on ordering — tap, rotate, back, grant permission, repeat. Without a behavioral model it is fuzz for UI state machines. It surfaces ordering assumptions (recorder infrastructure is a classic victim) but still needs a separate oracle for “UI is wrong.”

If your fear is visual or navigation correctness, buy model or replay-backed oracles before buying more random taps. Random alone is cheap chaos without a verdict.

Rare branch or infeasible path — steer into the code

When the failure lives behind a deep predicate — attachment metadata parsers, migration utilities, pure logic modules — random and UI suites waste budget. Concolic (concrete + symbolic) testing runs on real inputs, then solves path constraints to mutate the next branch flip. Full symbolic execution can prove path feasibility or contradiction; concolic usually costs less because it follows one concrete trace at a time.

x = read();
if (x > 3) {
  y = 1;
  if (x < 0) y = 2;  // contradictory under x > 3
} else y = 3;

Symbolic substitution shows y = 2 is unreachable. That is the kind of claim fuzz cannot make and UI tests rarely reach.

Budget traps: path explosion, opaque syscalls, and library boundaries stall analysis. Use coverage-guided or concolic hybrids on the module you fear; do not pretend the whole Activity graph is a symbolic target. Pair with equivalence partitioning and boundary cases when writing the unit specs those tools feed — empty / one / full page, min / max / just-inside edges — so the cheap layer still hits fence posts humans forget.

Product metric, not pass/fail — A/B

When the failure you fear is “users bounce” or “the rewrite is slower in the wild,” lab oracles lie. A/B (split) testing exposes cohorts to two implementations and compares metrics — tap-through, retention, crash rate — with statistical gates.

Figure 1. Split traffic between variants; significance gates promotion.

Staged UI rollouts fit here: a rewritten list vs baseline, placement of a promotional slot. The oracle is a product metric. Budget includes traffic, ethical bucketing, and guardrails so a losing variant does not quietly harm core flows. Do not spend A/B budget proving what a unit test already owns.

Also in the toolbox

Once the four fear classes above have a primary method, leftover techniques are supporting tools — not parallel “must cover everything” chapters:

Production Android work stacks them: boundary/ECP under unit tests, fuzz on native attack surface, model or replay for navigation, A/B for user-visible experiments. The anti-pattern is one cheap method everywhere, then surprise when the failure class you feared was never in that method’s oracle.

Spend the next automation hour on the method whose oracle matches the failure you cannot ship with — not on padding the suite until CI stays green.

References

• An Introduction to Fuzzing — OWASP
• Model-based testing fundamentals (Android Developers)
• Cadar, Dunbar, Engler, KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Systems Programs — symbolic execution foundations
• Kohavi, Longbotham, Sommerfield, Henne, Controlled experiments on the web — A/B methodology