Run The Integration Test Suite

Use this guide when you want to run Preflight’s live integration tests against a real Windows VM. The suite exercises the full end-to-end execution path — bootstrap, auth, transport, guard, oracle, cleanup, and idempotency — through a single module (registry) as the proving slice, repeated over every configured transport (WinRM and/or SSH-to-Windows).

Prerequisites

• A Windows VM on the same network as your dev machine (or otherwise reachable)
• The preflight repository checked out on your dev machine
• Go 1.21+ on your dev machine
• Network connectivity from your dev machine to the VM on port 5985 (WinRM) and/or port 22 (SSH)

1. Get A Windows VM

VMware Fusion (macOS)

The fastest path is a free Windows evaluation VM from Microsoft:

# Download a Windows 11 Dev Environment VM
brew install wget
wget -O ~/Downloads/win11-dev.vmwarevm.zip \
  'https://aka.ms/windev_VM_vmware'

# Extract and open in VMware Fusion
cd ~/Downloads
unzip win11-dev.vmwarevm.zip
open win11-dev.vmwarevm

The Windows 11 Dev Environment VMs come with Visual Studio and developer tools pre-installed. They expire after 90 days, which makes them ideal disposable test targets.

After the VM boots:

1. Complete the OOBE (accept defaults, set a local user password)
2. Note the IP address shown on the login screen, or find it later with ipconfig inside the VM
3. Ensure both machines are on the same network (NAT or bridged)

VirtualBox (cross-platform)

Microsoft also publishes Hyper-V and VirtualBox images from the same download page. The bootstrap scripts work identically regardless of hypervisor.

2. Run The Bootstrap Scripts

Setup is split into three scripts so identity and each transport are independent: provision the account once, then enable whichever transports you want. Inside the Windows VM, open PowerShell as Administrator.

Provision the test account (always required)

# Set the password for the pf-test user (use a strong, unique password)
$env:PREFLIGHT_TEST_WINRM_PASS = 'YourStrongPassword123!'

# Run directly from the repo (or copy scripts/dev/bootstrap-user-vm.ps1 to the VM first)
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass -Force
iex ((New-Object System.Net.WebClient).DownloadString(
  'https://raw.githubusercontent.com/bluecadet/preflight/main/scripts/dev/bootstrap-user-vm.ps1'
))

This creates the pf-test local admin user and writes the sacrificial sentinel, then prints the connection vars (with the password) for your .env.test. If the VM has no internet access, copy the script over and run .\bootstrap-user-vm.ps1 — you will be prompted for the password.

Two setup steps are not covered by the scripts; run them manually in the same elevated PowerShell session:

# A second throwaway admin account for the become integration tests, which
# exercise credential delegation to a non-connecting user.
$pass2 = ConvertTo-SecureString "password" -AsPlainText -Force
New-LocalUser "pf-become" -Password $pass2 -PasswordNeverExpires
Add-LocalGroupMember -Group "Administrators" -Member "pf-become"

# Allow remote token elevation (required for WinRM admin sessions).
New-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System" `
  -Name LocalAccountTokenFilterPolicy -Value 1 -PropertyType DWord -Force

Enable WinRM

iex ((New-Object System.Net.WebClient).DownloadString(
  'https://raw.githubusercontent.com/bluecadet/preflight/main/scripts/dev/bootstrap-winrm-vm.ps1'
))

Enables WinRM over HTTP with Basic auth on port 5985, adds pf-test to Remote Management Users, and opens the firewall. Secret-free — it reuses the account from the provision step.

Enable SSH-to-Windows

iex ((New-Object System.Net.WebClient).DownloadString(
  'https://raw.githubusercontent.com/bluecadet/preflight/main/scripts/dev/bootstrap-ssh-vm.ps1'
))

Installs OpenSSH Server, starts sshd, and opens the firewall for port 22. Preflight authenticates over SSH with password auth by default (the same pf-test password), so no key generation is required — key auth is optional via PREFLIGHT_TEST_SSH_KEY.

3. Set The Environment Variables

The test harness reads individual KEY=VALUE pairs from the environment or from a .env.test file. Create .env.test at the repo root (gitignored; never commit it):

PREFLIGHT_TEST_WINRM_HOST=192.168.x.x
PREFLIGHT_TEST_WINRM_PORT=5985
PREFLIGHT_TEST_WINRM_USER=pf-test
PREFLIGHT_TEST_WINRM_PASS=YourStrongPassword123!

# Optional: SSH-to-Windows for the same VM (requires OpenSSH Server)
PREFLIGHT_TEST_SSH_HOST=192.168.x.x
PREFLIGHT_TEST_SSH_PORT=22
PREFLIGHT_TEST_SSH_USER=pf-test
PREFLIGHT_TEST_SSH_PASS=YourStrongPassword123!
# PREFLIGHT_TEST_SSH_KEY=/path/to/id_rsa  # optional, password auth is default

PREFLIGHT_TEST_WINRM_PORT and PREFLIGHT_TEST_SSH_PORT default to 5985 and 22 respectively when omitted. Each transport is independently optional — set only the vars for the transports you want to test.

Security note: The password appears in the file in plain text because the WinRM transport sends it as Basic auth. Only use this against disposable VMs. Never point these vars at a production machine.

4. Run The Test

The test runner loads .env.test automatically — no source or direnv needed. Variables already exported in your shell take precedence over the file.

The suite is behind the integration build tag, so it is excluded from a plain go test ./... entirely. Run the whole suite through the Makefile target, which adds the tag for you:

make test-integration

Or run a single test directly — the -tags integration flag is required, or go test reports no tests at all:

go test -tags integration -v -run TestIntegration_Registry ./internal/target/

Expected output when both WinRM and SSH are configured:

=== RUN   TestIntegration_Registry
=== RUN   TestIntegration_Registry/winrm
--- PASS: TestIntegration_Registry/winrm (XX.XXs)
=== RUN   TestIntegration_Registry/ssh
--- PASS: TestIntegration_Registry/ssh (XX.XXs)
--- PASS: TestIntegration_Registry (XX.XXs)

To run all integration tests (both the multi-transport registry test and the WinRM-only tests for other modules):

go test -tags integration -v -run 'TestIntegration|TestWinRMIntegration' ./internal/target/

Tests named TestIntegration_* run every configured transport as a subtest, so you can filter by transport name:

# Registry test via SSH only / WinRM only
go test -tags integration ./internal/target/ -run TestIntegration_Registry/ssh -v
go test -tags integration ./internal/target/ -run TestIntegration_Registry/winrm -v

Tests prefixed TestWinRMIntegration_* connect via WinRM directly and run only when PREFLIGHT_TEST_WINRM_HOST / _USER / _PASS are set.

go test -tags integration ./internal/target/ -run TestWinRMIntegration_WindowsFeature -v -timeout 5m

Skipping behaviour

Each transport is independently opt-in. When the env vars for a transport are unset, its subtest skips cleanly:

=== RUN   TestIntegration_Registry
=== RUN   TestIntegration_Registry/winrm
    integration_registry_test.go:XX: PREFLIGHT_TEST_WINRM_HOST / _USER / _PASS not set
--- SKIP: TestIntegration_Registry/winrm (0.00s)
=== RUN   TestIntegration_Registry/ssh
    integration_registry_test.go:XX: PREFLIGHT_TEST_SSH_HOST / _USER / _PASS not set
--- SKIP: TestIntegration_Registry/ssh (0.00s)
--- SKIP: TestIntegration_Registry (0.00s)

When no transports are configured, the parent test also skips. CI jobs stay green without any configuration changes.

Sentinel guard

If a transport points at a machine that is missing the sacrificial sentinel, the test hard-skips with a loud message instead of mutating the target:

=== RUN   TestIntegration_Registry
=== RUN   TestIntegration_Registry/winrm
    winrm_integration_harness_test.go:XX: sacrificial sentinel not found on target ...
--- SKIP: TestIntegration_Registry/winrm (0.00s)

Test Anatomy

TestIntegration_Registry is wrapped in forEachTransport, which runs the same body function once per configured transport:

1. Gate per transport: Skips the transport subtest when its env vars are unset (HOST/USER/PASS independently per transport)
2. Sentinel guard: Asserts HKLM\SOFTWARE\PreflightTest\IsSacrificial=1 via the PowerShellRunner interface
3. Cleanup: t.Cleanup removes the per-run registry key regardless of how far the test gets
4. Present: Creates a DWORD value, verifies via independent oracle
5. Idempotent: Re-check and re-apply both return StatusOK
6. Dry-run: Check-only with a different value predicts StatusChanged; oracle confirms the actual value was not mutated
7. Drift: Mutates the value behind the module’s back via PowerShell, asserts Check detects it and Apply converges back
8. Absent: Removes the value, then removes the entire key; oracle confirms both, then asserts idempotent re-check returns StatusOK

The independent oracle is load-bearing: asserting only through the module’s own Check() would pass a module whose Check and Apply share a bug.

Adding A New Integration Test

To add a new module to the integration suite:

1. Add a TestIntegration_<Module> function that calls forEachTransport
2. Register cleanup via t.Cleanup (use the PowerShellRunner interface)
3. Write an independent oracle that reads state without using the module’s Check method
4. Use mustExecute for every Execute step (collapses err+status assertion)
5. Assert both correctness (oracle matches expectation) and idempotency (rerun Check/Apply produce StatusOK)
6. For coverage completeness, include dry-run and drift branches (see the registry test for the pattern)

When a module cannot operate over a given transport (e.g. WinRM symlink limitation for windows_feature), gate the operation with a capability check and t.Skip with a clear reason rather than t.Fatal.

Troubleshooting

If every test fails at the sacrificial-sentinel check — the first PowerShell call — with errors like Starting the CLR failed with HRESULT 80004005, STATUS_DLL_INIT_FAILED (0xC0000142), or STATUS_COMMITMENT_LIMIT (0xC000012D), the VM itself can no longer launch powershell.exe. This is endpoint resource exhaustion, not a code failure: the VM is out of committed memory or its non-interactive desktop heap is exhausted, often after many runs have accumulated WinRM shells.

1. Reboot the VM and re-run. This clears the exhaustion and is the usual fix.
2. If it recurs across runs, raise the VM’s WinRM quotas — MaxShellsPerUser and MaxMemoryPerShellMB under WSMan:\localhost\Shell — and, if process launches keep failing, the non-interactive desktop-heap SharedSection value under HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\SubSystems\Windows.

The tests register a Close() cleanup that releases each target’s persistent shell, so a single suite run should not leak shells; the reboot guidance applies mainly when a VM has been driven into a bad state by older runs or other workloads.

Re-run scripts/dev/bootstrap-winrm-vm.ps1 on the VM if you suspect the WinRM configuration has drifted. For a completely fresh start, revert the VM to a snapshot or redeploy the evaluation image.

Related Docs

• Troubleshoot remote connections
• Run a playbook against remote hosts
• Targets, transports, and plugins