policy/v2: add policytester compat test runner

Pin headscale's accept/reject decision and error body against
Tailscale SaaS by replaying captures recorded from a real tailnet.
Mirrors the tailscale_grants_compat_test.go pattern: glob over
testdata/policytest_results/, one t.Run per file, parse-or-SetPolicy
error must contain the captured api_response_body.message.

errPolicyTestsFailed is "test(s) failed" — Tailscale's literal body —
so substring match works against captured response bodies. Per-test
detail (src, dst, expected vs got) is preserved below the prefix for
the CLI / config-reload paths that don't have an audit endpoint.

knownPolicyTesterDivergences gates the 12 mismatches the captures
will surface so the suite stays green; engine fixes in follow-up
commits drop the entries as each is resolved.

Updates #1803
This commit is contained in:
Kristoffer Dalby 2026-04-29 14:28:50 +00:00
parent b29ae25356
commit 7bc701179b
2 changed files with 179 additions and 6 deletions

View file

@ -26,10 +26,11 @@ import (
// errPolicyTestsFailed wraps the rendered failure body so callers can
// type-assert when they need to react differently to test failures vs. parse
// errors. The Error() output is the user-facing message and is intended to
// match Tailscale SaaS verbatim once the corpus is captured via tscap.
// errors. The Error() prefix is "test(s) failed", the same string Tailscale
// SaaS returns in the api_response_body.message — see
// hscontrol/policy/v2/testdata/policytest_results/.
var (
errPolicyTestsFailed = errors.New("policy tests failed")
errPolicyTestsFailed = errors.New("test(s) failed")
errTestDestinationNoIP = errors.New("destination resolved to no IP addresses")
)
@ -76,9 +77,11 @@ type PolicyTestResults struct {
Results []PolicyTestResult `json:"results"`
}
// Errors renders the failure body. Format is intended to byte-exact match
// Tailscale SaaS once captured via tscap; until the corpus lands, the
// strings below are best-effort and will be updated to match.
// Errors renders the per-test failure breakdown joined by newlines.
// Tailscale SaaS itself only returns the literal "test(s) failed" — we
// keep the per-test detail because it is significantly more useful in
// CLI / config-reload paths where the user does not have a separate
// audit endpoint to consult.
func (r PolicyTestResults) Errors() string {
if r.AllPassed {
return ""