test: harden AI analysis path
This commit is contained in:
@@ -0,0 +1,43 @@
|
||||
# Harden AI Analysis Path Implementation Plan
|
||||
|
||||
## Steps
|
||||
|
||||
1. Add pure validator tests for `modules/ai/types.ts`.
|
||||
2. Add/adjust analysis citation helper behavior in `modules/ai/server/analysis.ts`.
|
||||
3. Add analysis service tests for missing config, invalid output, provider errors, retrieval degradation, and redaction.
|
||||
4. Add knowledge service tests for scope, disabled entries, organization isolation, and embedding failure.
|
||||
5. Add AI route tests for permission failures, invalid input, service failure propagation, success responses, and audit calls.
|
||||
6. Improve knowledge UI and analysis dialog failure copy.
|
||||
7. Clarify `ai:manage` permission description.
|
||||
8. Run targeted AI tests, then project verification.
|
||||
|
||||
## Validation Commands
|
||||
|
||||
- `pnpm test -- modules/ai/types.test.ts`
|
||||
- `pnpm test -- modules/ai/server/analysis.test.ts modules/ai/server/knowledge.test.ts app/api/ai/ai-routes.test.ts`
|
||||
- `pnpm lint`
|
||||
- `pnpm type-check`
|
||||
- `pnpm test`
|
||||
- `pnpm build`
|
||||
|
||||
## Risky Files
|
||||
|
||||
- `modules/ai/server/analysis.ts`
|
||||
- `modules/ai/server/knowledge.ts`
|
||||
- `modules/ai/types.ts`
|
||||
- `app/api/ai/*/route.ts`
|
||||
- `modules/core/server/permissions.ts`
|
||||
|
||||
## Review Gates
|
||||
|
||||
- Citation helper must reject unknown citation IDs and avoid appending unused citations.
|
||||
- Tests must not call real AI providers.
|
||||
- Tests must not require a live PostgreSQL instance unless explicitly scoped to migration smoke testing.
|
||||
- No new `any`, non-null assertions, `@ts-ignore`, or `@ts-expect-error`.
|
||||
- Failed history records must remain sanitized.
|
||||
|
||||
## Rollback Points
|
||||
|
||||
- After citation strictness: revert helper and related tests if provider output compatibility proves too brittle.
|
||||
- After route tests: routes should remain behavior-compatible except status/copy assertions.
|
||||
- After UI copy: copy changes can be reverted without affecting backend contracts.
|
||||
Reference in New Issue
Block a user