# Integration Tests v2 (CDP Harness)

CDP-based UI integration test harness for Kiro. Interact with the product like a user would — no extra extensions required.

## How it works

Kiro is an Electron app. This harness launches the Kiro installation from `.vscode-test/` (downloaded by the existing test infrastructure) with `--remote-debugging-port=6789` and connects via Chrome DevTools Protocol over WebSocket. It injects JavaScript into the renderer to find elements, click buttons, type text, and read what's on screen.

## Prerequisites

Make sure Kiro is downloaded:

```bash
# From the kiro-extension root
npm run vscode-test:install
```

## Quick start

```bash
cd integration-tests-v2
npm install
npm run build

# Run unit tests (mock server, event stream encoder, etc.)
npm test

# Run e2e tests (launches Kiro, connects via CDP)
npm run test:e2e
```

## Directory layout

```
integration-tests-v2/
  __workspace__/     # Isolated workspace opened by Kiro during tests
  __userdir__/       # Isolated user-data dir (gitignored, recreated each run)
  __extensions__/    # Isolated extensions dir (gitignored, recreated each run)
  src/
    setup/           # Global setup/teardown and per-suite test env
    rts-mock/        # Mock server for CodeWhisperer runtime APIs
    tests/           # E2E test suites
    cdp.ts           # Low-level CDP connection
    harness.ts       # High-level test harness API
    selectors.ts     # CSS selectors for Kiro's webview
    types.ts         # Public types
    index.ts         # Barrel exports
```

## Writing a test suite

Every suite uses the `useTestEnv()` fixture. It owns the whole lifecycle:
suite setup/teardown, a clean-state reset before every test, and
unmatched-request verification after every test.

```typescript
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { useTestEnv } from '../../setup/test-env.js';
import { WorkspaceFiles, TIMEOUTS } from '../../test-utils.js';
import { str } from '../../ui-strings.js';
import { ToolName } from '../../rts-mock/tool-names.js';

const PREFIX = `my-feature-${Date.now()}`;

describe('my feature', () => {
  const env = useTestEnv();
  let files: WorkspaceFiles;

  beforeAll(() => {
    // Anything that reads env state belongs here, not in the describe body —
    // the environment exists from beforeAll onward.
    files = new WorkspaceFiles(env.workspacePath);
  });

  afterAll(() => {
    files.cleanup();
  });

  it(
    'does the thing',
    async () => {
      // 1. Script the model response (streaming expectations are per-test;
      //    they were reset automatically before this test ran)
      env.mock.streaming.mockToolCall('tool-1', ToolName.FsWrite, { path: `${PREFIX}.txt`, text: 'hi' }, 'Done.');

      // 2. Drive the UI like a user
      const result = await env.kiro.chat('Create the file', 'Done.');
      expect(result.success).toBe(true);

      // 3. Assert on every observable outcome: file system, tool result,
      //    outcome card UI (via str() for customer-facing copy), telemetry
    },
    TIMEOUTS.CHAT_TEST,
  );
});
```

What the fixture guarantees:

- **Before each test**: streaming expectations, the streaming request log,
  and telemetry counters are reset. Tests never see a previous test's
  scripted responses or metrics.
- **After each test**: if the product made a streaming request that no
  expectation matched (the mock answered 400), the test fails. A suite whose
  tests legitimately produce unscripted requests can opt out with
  `useTestEnv({ verifyUnmatched: false })` — always leave a comment saying
  why.
- **Per suite**: fresh workspace, window reload (restarts the extension
  host), fresh session. REST expectations (models, usage limits) are armed
  once per suite via `onCommonStartup` and survive across tests.

Suites whose tests interfere through workspace files or chat sessions can
broaden the per-test reset: `useTestEnv({ resetScopes: ['mocks', 'workspace', 'session'] })`.

Conventions that keep suites reliable:

- Unique file names per suite via a `PREFIX` that embeds `Date.now()`.
- Assert customer-facing copy through `str(category, key)` from
  `ui-strings.ts` (typo'd keys fail to compile; copy changes are a one-file
  fix).
- Hooks that must run before the window reload (e.g. seeding auth tokens)
  are registered _before_ the `useTestEnv()` call — hooks run in
  registration order. See the governance suites for the pattern.
- Don't re-do the fixture's work in a suite `beforeEach` (quiesce +
  streaming/telemetry reset). Suites that need a fresh session per test say
  so declaratively — `useTestEnv({ resetScopes: ['mocks', 'session'] })` —
  and the fixture applies scopes churn-first (session reset, settle, THEN
  mock reset), so session-setup requests never land on freshly armed
  expectations. A manual post-churn reset is only sanctioned for churn the
  scopes don't model (`reloadWindow()`, per-section session resets as in
  permissions.test.ts) — there, too, reset the streaming mock _last_.

## Harness usage outside vitest

```typescript
import { KiroTestHarness } from './harness.js';

const kiro = new KiroTestHarness({ cdpPort: 6789 });
await kiro.connect(0);

await kiro.chat('Build a todo app', 'expected response text');
await kiro.takeSnapshot('agent');

kiro.dispose();
```

The harness surface is defined by two interfaces — `ChatUI`
(`src/chat-ui.ts`) for the chat webview and `IdeUI` (`src/ide-ui.ts`) for
the main window. `KiroTestHarness` exposes every method of both directly
(runtime delegation), so `env.kiro.<anything>` works. To add a harness
capability: add the signature to the interface, implement it in
`ChatHarness` / `IdeHarness`. Nothing else — the facade picks it up
automatically.

## Test timeout budgets

When adding a test that chains several chat turns or session-tab switches,
budget it with `TIMEOUTS.MULTI_TURN_TEST` rather than `TIMEOUTS.CHAT_TEST` —
see the constant's doc comment in `src/test-utils.ts` for why.
