feat(khira): complete remediation, performance optimization, and Cloudflare Turnstile integration
- Backend: - Fix ListTickets parameter order mismatch for assignee filtering - Replicate all 13 PostgreSQL tables and columns during disaster recovery sync and rollback - Implement server-side session store with HttpOnly cookies and Bearer tokens - Add profile update verification requiring current password - Whitelist allowed file uploads and apply strict security headers (nosniff, CSP) - Enforce role check on error deletion (admin/lead/tester only) and authenticate SSE stream - Align foreign key user_id for saved filters and api tokens - Add ReopenTicket endpoint and workflow - Integrate Cloudflare Turnstile verification with testing/production key support - Frontend: - Remove liquid glass displacement filter and lens capsule for high-performance UI - Eliminate recursive backdrop-filters and heavy inset shadows - Add debounced search and optimize SSE re-rendering - Wire Saved Views filter bar with creation, active indicators, and deletion - Add Linked & Related Tickets management UI to sheet - Seed 6 demo personas in PostgreSQL with instant sign-in - Add Cloudflare Turnstile widgets to authentication modals - Add keyboard arrow navigation to Command Palette - Image Manipulator, Docs & Tooling: - Sync image manipulator updates, docs, and test coordinates
This commit is contained in:
@@ -0,0 +1,323 @@
|
||||
# Khira Comprehensive Defect Remediation & Cloudflare Turnstile Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Eliminate all performance bottlenecks (nested backdrop-filters, un-debounced search, heavy SVG filters), remediate all security flaws (unauthenticated header spoofing, arbitrary file uploads, weak password hashing), resolve all backend logic defects (argument mismatch in ListTickets, disaster recovery data loss, inability to move to Open), fix all frontend anomalies (demo logins, upload key mismatches, missing links/attachments in sheets, dead features), and integrate Cloudflare Turnstile bot protection into Khira.
|
||||
|
||||
**Architecture:**
|
||||
1. **Frontend Optimization**: Replace recursive multi-stage `--glass-blur` on dynamic elements with composited obsidian surfaces, throttle search input with a 200ms debounce, optimize the SVG liquid glass lens, and prevent duplicate canvas rendering.
|
||||
2. **Security & Turnstile**: Implement secure session cookie/token verification (stopping `X-Khira-User` header spoofing), upgrade password hashing, enforce file extension whitelists with safe headers, require current password on profile updates, and add Turnstile CAPTCHA verification on auth endpoints with fallback dev keys.
|
||||
3. **Backend & DB Integrity**: Fix the parameter order mismatch in `ListTickets`, update `SyncDatabases` and `RollbackDatabases` to replicate all 13 tables and missing columns, add a `reopen` endpoint for Open status transitions, and seed all demo users.
|
||||
4. **UI & Feature Completeness**: Fix demo accounts, dual-key upload responses (`attachment.url` and `url`), re-fetch ticket details on sheet open to show attachments/links, implement Saved Filters and Ticket Relations in UI, fix light theme CSS, and add keyboard navigation to the Command Palette.
|
||||
|
||||
**Tech Stack:** Go 1.22+, Chi Router, PostgreSQL 16 (`khira_primary` & `khira_backup`), Vanilla JS (ES6+), CSS Custom Properties, Cloudflare Turnstile API.
|
||||
|
||||
---
|
||||
|
||||
## Global Constraints
|
||||
- Do not break existing API contracts or response structures expected by other dashboard tools.
|
||||
- PostgreSQL database credentials must default to `KHIRA_PRIMARY_DB_URL` and `KHIRA_BACKUP_DB_URL` with local defaults.
|
||||
- Always use the local python venv when running scripts: `source /home/creserver/extras/local_venv/bin/activate.fish`.
|
||||
- Cloudflare Turnstile must support standard Cloudflare dummy testing keys (`1x00000000000000000000AA` / `1x0000000000000000000000000000000AA`) as well as environment variables `KHIRA_TURNSTILE_SITE_KEY` and `KHIRA_TURNSTILE_SECRET_KEY`.
|
||||
|
||||
---
|
||||
|
||||
## Review Focus
|
||||
1. `ListTickets` query parameter alignment: `status`, `priority`, `app`, `assignee`, `reporter`, `query`, `ticket_type`.
|
||||
2. Disaster recovery database sync/rollback across all 13 tables and newly added columns (`due_date`, `sla_hours`, `sla_breached`, `active`, `last_active_at`, `deactivated_at`).
|
||||
3. Secure session cookie vs spoofed `X-Khira-User` headers.
|
||||
4. File upload extension whitelisting and direct SVG/HTML execution mitigation.
|
||||
5. Turnstile validation failure handling without blocking headless testing or demo logins.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Fix Critical Backend Bug: `ListTickets` Parameter Order Mismatch
|
||||
|
||||
**Files:**
|
||||
- Modify: `internal/khira/handlers.go:394-412`
|
||||
- Modify: `internal/khira/store.go:731-795`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: HTTP Query params (`status`, `priority`, `app`, `assignee`, `creator`/`reporter`, `role`, `type`/`ticket_type`, `q`)
|
||||
- Produces: Correctly filtered `[]*Ticket` matching all specified criteria
|
||||
|
||||
- [ ] **Step 1: Check existing broken behavior with curl**
|
||||
Run: `curl -s -H "X-Khira-User: admin" "http://127.0.0.1:10000/khira/api/tickets?assignee=admin"`
|
||||
Expected: Currently returns `{"count":0,"tickets":null}` due to argument mismatch.
|
||||
|
||||
- [ ] **Step 2: Fix argument order in `handleListTickets` in `internal/khira/handlers.go`**
|
||||
Extract `app := r.URL.Query().Get("app")`, `assignee`, `creator := r.URL.Query().Get("creator")` (fallback to `reporter`), `q`, `ticketType`.
|
||||
Call `s.Store.ListTickets(status, priority, app, assignee, creator, q, ticketType)` in the exact order matching `store.go`.
|
||||
|
||||
- [ ] **Step 3: Update `ListTickets` in `internal/khira/store.go` to support role-based filtering**
|
||||
If `role` is provided, add `LOWER(created_by_role) = LOWER(...) OR LOWER(assigned_to_role) = LOWER(...)`. Ensure SQL query correctly parameterizes all conditions.
|
||||
|
||||
- [ ] **Step 4: Recompile and verify ticket filtering**
|
||||
Run: `go build -o bin/nexus-gateway cmd/nexus-gateway/main.go` and restart/test gateway.
|
||||
Run: `curl -s -H "X-Khira-User: admin" "http://127.0.0.1:10000/khira/api/tickets?assignee=admin"`
|
||||
Expected: Returns KHR-101 (`count: 1`).
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Fix Disaster Recovery: Complete All Tables & Columns in DB Sync/Rollback
|
||||
|
||||
**Files:**
|
||||
- Modify: `internal/khira/db.go:290-670`
|
||||
- Modify: `internal/khira/handlers.go:1399-1460`
|
||||
- Modify: `web/khira/app.js:1950-1965`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `primaryDB`, `backupDB` connections
|
||||
- Produces: Fully consistent mirrored database state with zero dropped tables or columns
|
||||
|
||||
- [ ] **Step 1: Update `SyncDatabases` in `internal/khira/db.go`**
|
||||
Include all 13 tables:
|
||||
- `users`: add columns `active`, `last_active_at`, `deactivated_at`.
|
||||
- `tickets`: add columns `due_date`, `sla_hours`, `sla_breached`.
|
||||
- Add replication for: `ticket_tags`, `error_tags`, `ticket_relations`, `saved_filters`, `api_tokens`.
|
||||
Truncate cascade query must cleanly handle all tables.
|
||||
|
||||
- [ ] **Step 2: Update `RollbackDatabases` in `internal/khira/db.go`**
|
||||
Mirror the exact same 13 tables and complete column set when restoring from `backupDB` to `primaryDB`.
|
||||
|
||||
- [ ] **Step 3: Fix `adminSyncDB()` in `web/khira/app.js` and `handleDBSync` in `handlers.go`**
|
||||
Make `handleDBSync` accept authenticated admin session without requiring re-prompt, or prompt cleanly in `app.js` like `adminRollbackDB()`.
|
||||
Ensure `adminSyncDB` sends the admin password if requested.
|
||||
|
||||
- [ ] **Step 4: Verify DB Sync and Rollback**
|
||||
Test `/khira/api/admin/db/sync` and `/khira/api/admin/db/status` via curl and UI. Confirm all tables (including tags, relations, tokens) retain counts.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Backend Security Hardening: Authentication, Session Tokens & Safe File Uploads
|
||||
|
||||
**Files:**
|
||||
- Modify: `internal/khira/handlers.go:31-55, 325-366, 934-1004`
|
||||
- Modify: `internal/khira/handlers_enterprise.go:300-370`
|
||||
- Modify: `internal/khira/store.go:116-240, 1458-1468`
|
||||
- Modify: `internal/khira/events.go:74-119`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Login credentials, session cookies, auth tokens, upload multipart form
|
||||
- Produces: Cryptographically verified user actor, sanitized file uploads, protected admin routes
|
||||
|
||||
- [ ] **Step 1: Implement Session Tokens in Auth Store**
|
||||
In `internal/khira/store.go`:
|
||||
Generate a secure session token (32 random hex bytes) on successful `AuthenticateUser`. Store in an active sessions map or table with user ID and expiry (7 days).
|
||||
Set HTTP-only session cookie `khira_session` on `/auth/login` and `/auth/register`. Clear on `/auth/logout`.
|
||||
|
||||
- [ ] **Step 2: Secure `getActor`**
|
||||
In `internal/khira/handlers.go`:
|
||||
Validate the session token from `khira_session` cookie or `Authorization: Bearer <token>` against the session store or `api_tokens`.
|
||||
Deprecate unauthenticated raw `X-Khira-User` header unless accompanied by a valid session cookie or valid API token, preventing arbitrary identity spoofing.
|
||||
|
||||
- [ ] **Step 3: Require Current Password for Profile Updates**
|
||||
In `handleUpdateProfile`:
|
||||
If `req.NewPassword` or `req.Password` is provided, require `req.CurrentPassword`. Verify with `AuthenticateUser(actor.Username, req.CurrentPassword)`. Reject with 401 Unauthorized if invalid.
|
||||
|
||||
- [ ] **Step 4: Protect File Uploads against Stored XSS**
|
||||
In `handleUpload`:
|
||||
Enforce extension whitelist: `.png`, `.jpg`, `.jpeg`, `.webp`, `.gif`, `.txt`, `.log`, `.pdf`, `.json`.
|
||||
Reject `.html`, `.htm`, `.svg`, `.js`, `.sh`, `.py`, `.exe` with `400 Invalid file type`.
|
||||
Add response headers on `/khira/uploads/*`:
|
||||
`X-Content-Type-Options: nosniff`, `Content-Security-Policy: default-src 'none'; img-src 'self' data:; style-src 'unsafe-inline'`, and `Content-Disposition: inline` (for safe images) or `attachment` (for other files).
|
||||
Return both `"attachment": att` and `"url": att.URL` in the JSON response.
|
||||
|
||||
- [ ] **Step 5: Restrict Error Telemetry Deletion and Authenticate SSE**
|
||||
In `DeleteError`: Require `s.isAdmin(actor)` or `actor.Role == RoleTester || actor.Role == RoleLead`.
|
||||
In `handleEventsSSE`: Require `s.getActor(r) != nil` before subscribing.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Frontend Performance Optimization (Eliminate UI Lag & High Resource Usage)
|
||||
|
||||
**Files:**
|
||||
- Modify: `web/khira/styles.css:50-85, 720-805, 1620-1660`
|
||||
- Modify: `web/khira/app.js:225-285, 355-440, 2215-2225, 2310-2335`
|
||||
- Modify: `web/khira/index.html:18-36`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: UI rendering events, scroll, search input, drag and drop
|
||||
- Produces: 60+ FPS responsive UI with lightweight compositing and zero typing lag
|
||||
|
||||
- [ ] **Step 1: Eliminate recursive nested `backdrop-filter` in `styles.css`**
|
||||
Remove `backdrop-filter: var(--glass-blur)` from `.col`, `.card`, `.stat`, `.fld`, etc.
|
||||
Replace with clean, hardware-composited semi-transparent dark surfaces:
|
||||
```css
|
||||
background: rgba(14, 18, 28, 0.88);
|
||||
border: 1px solid rgba(255, 255, 255, 0.08);
|
||||
```
|
||||
Retain `backdrop-filter: blur(12px)` ONLY on non-nested overlays (`.app-header`, `.modal-overlay`, `.sheet`).
|
||||
Remove multi-layer inset box-shadows that force heavy paint cycles.
|
||||
|
||||
- [ ] **Step 2: Optimize Kube.io Liquid Glass Precision Lens**
|
||||
When lens is closed, ensure SVG filter and container are completely unrendered (`display: none`).
|
||||
Optimize the lens capsule to use lightweight CSS magnification/border refraction rather than a 7-stage SVG displacement map that forces CPU rasterization on mouse move.
|
||||
|
||||
- [ ] **Step 3: Debounce Global Search (`#q`)**
|
||||
In `web/khira/app.js`:
|
||||
Add a 200ms debounce timer to `#q` input listener:
|
||||
```javascript
|
||||
let searchTimer = null;
|
||||
document.addEventListener('input', e => {
|
||||
if (e.target.id === 'q') {
|
||||
clearTimeout(searchTimer);
|
||||
searchTimer = setTimeout(() => {
|
||||
S.q = e.target.value.trim().toLowerCase();
|
||||
render(false);
|
||||
}, 200);
|
||||
}
|
||||
});
|
||||
```
|
||||
Do not rewrite `#fa` options on search keystrokes unless users changed.
|
||||
|
||||
- [ ] **Step 4: Fix Duplicate Canvas Redraw in SSE**
|
||||
In `handleRealtimeEvent()`:
|
||||
Remove the duplicate `renderA(false)` call on line 2326, since `refresh(false)` already re-renders the active section when data changes.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Seed Demo Accounts & Fix Landing Page Authentication
|
||||
|
||||
**Files:**
|
||||
- Modify: `internal/khira/db.go:245-285`
|
||||
- Modify: `web/khira/app.js:2250-2255`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Demo buttons click events (`charlie`, `alice`, `bob`, `diana`, `eva`, `admin`)
|
||||
- Produces: Successful one-click sign in for all demo personas
|
||||
|
||||
- [ ] **Step 1: Seed all 6 demo accounts in `seedDefaultUsers` (`db.go`)**
|
||||
Seed:
|
||||
- `admin` (System Administrator, `team_lead`, pass: `admin` and `admin123`)
|
||||
- `charlie` (Charlie Patel, `tester`, pass: `charlie123`)
|
||||
- `alice` (Alice Sharma, `ios_dev`, pass: `alice123`)
|
||||
- `bob` (Bob Anderson, `backend_dev`, pass: `bob123`)
|
||||
- `diana` (Diana Prince, `android_dev`, pass: `diana123`)
|
||||
- `eva` (Eva Chen, `designer`, pass: `eva123`)
|
||||
|
||||
- [ ] **Step 2: Update demo button handlers in `app.js`**
|
||||
Ensure `admin` demo button uses `admin123` or `admin`, matching seeded passwords.
|
||||
|
||||
- [ ] **Step 3: Verify all 6 demo logins**
|
||||
Test login via curl or browser for all 6 personas.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Cloudflare Turnstile Integration
|
||||
|
||||
**Files:**
|
||||
- Create: `internal/khira/turnstile.go`
|
||||
- Modify: `internal/khira/handlers.go:103-112, 205-255`
|
||||
- Modify: `web/khira/index.html:10-15, 45-62`
|
||||
- Modify: `web/khira/app.js:2230-2265`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `cf-turnstile-response` token from client
|
||||
- Produces: Verified server-side challenge via Cloudflare Turnstile API (`https://challenges.cloudflare.com/turnstile/v0/siteverify`)
|
||||
|
||||
- [ ] **Step 1: Implement Cloudflare Turnstile verification in `internal/khira/turnstile.go`**
|
||||
Read `KHIRA_TURNSTILE_SECRET_KEY` (fallback to official Cloudflare dummy key `1x0000000000000000000000000000000AA` which always passes).
|
||||
Implement:
|
||||
```go
|
||||
func VerifyTurnstile(token, remoteIP string) (bool, error)
|
||||
```
|
||||
POST to `https://challenges.cloudflare.com/turnstile/v0/siteverify` with `secret`, `response`, and `remoteip`.
|
||||
|
||||
- [ ] **Step 2: Add Turnstile endpoint & verification in `handlers.go`**
|
||||
Expose `GET /khira/api/auth/turnstile-config` returning `{"site_key": "..."}`.
|
||||
Verify token in `handleLogin`, `handleRegister`, and `handleForgotPassword`. Allow bypass for demo accounts if triggered via demo button.
|
||||
|
||||
- [ ] **Step 3: Add Turnstile Widget to Frontend**
|
||||
In `web/khira/index.html`:
|
||||
Add `<script src="https://challenges.cloudflare.com/turnstile/v0/api.js?render=explicit" async defer></script>`.
|
||||
Add container `<div id="turnstileWidget" class="turnstile-container"></div>` to `#fIn` and `#fUp`.
|
||||
|
||||
- [ ] **Step 4: Wire Turnstile in `app.js`**
|
||||
Fetch site key from `/khira/api/auth/turnstile-config`.
|
||||
Render widget explicitly via `turnstile.render('#turnstileWidget', { sitekey: ... })`.
|
||||
Include token in submit payloads.
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Fix Ticket Lifecycle (Moving to Open, Sheet Attachments/Links & Upload Keys)
|
||||
|
||||
**Files:**
|
||||
- Modify: `internal/khira/handlers.go:120-135, 934-1004`
|
||||
- Modify: `internal/khira/store.go:935-965`
|
||||
- Modify: `web/khira/app.js:485-498, 995-1030, 1315-1335, 2070-2090`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Card drag/drop, ticket detail sheet open, image uploads
|
||||
- Produces: Reliable ticket lifecycle transitions, persisted attachments/links in detail drawer, reliable image uploads
|
||||
|
||||
- [ ] **Step 1: Add Reopen / Move to Open in Store & Handler**
|
||||
In `internal/khira/store.go`:
|
||||
Implement `ReopenTicket(ticketID string, actor *User, remarks string) (*Ticket, error)` which updates `status = 'open'`, resets `hold_reason` and `completion_remarks`.
|
||||
Register `POST /khira/api/tickets/{id}/reopen` in `handlers.go`.
|
||||
Support `"open"` in `BulkAction` status updates.
|
||||
|
||||
- [ ] **Step 2: Update `initBoardDragDrop` in `app.js`**
|
||||
When dropped in `open`: Call `/khira/api/tickets/${id}/reopen`.
|
||||
|
||||
- [ ] **Step 3: Fix Upload JSON Response & Client Handlers**
|
||||
In `handlers.go` `handleUpload`: return both `{"status": "ok", "url": att.URL, "attachment": att}`.
|
||||
In `app.js`:
|
||||
In `newTicket()`: `uploadedImageUrl = upJson.url || upJson.attachment?.url;`
|
||||
In `profile()`: `curImgUrl = upJson.url || upJson.attachment?.url;`
|
||||
|
||||
- [ ] **Step 4: Fetch Fresh Ticket Details in `sheet(id)`**
|
||||
In `sheet(id)`: Call `const fullT = await api('/khira/api/tickets/' + encodeURIComponent(id));` so full attachments, links, and tags are always displayed.
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Implement Missing Features & Polish Frontend Defects
|
||||
|
||||
**Files:**
|
||||
- Modify: `web/khira/index.html:110-115, 205-225`
|
||||
- Modify: `web/khira/app.js:9-45, 1120-1165, 1830-1925, 2195-2205`
|
||||
- Modify: `web/khira/styles.css:75-90`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: UI events for filters, palette, themes, notifications
|
||||
- Produces: Fully functional Saved Filters, General Category dropdown, arrow-key navigable Command Palette, clean Light Theme, and missing icons
|
||||
|
||||
- [ ] **Step 1: Add General Category & Department in `newTicket()`**
|
||||
Add Category dropdown (`CATS`) and Scope/Department text field/chips in `#modalCreateTicket` / `newTicket()`. Pass to payload.
|
||||
|
||||
- [ ] **Step 2: Implement Saved Filters UI**
|
||||
In `app.js`: Add handlers to fetch `/khira/api/filters`, render chips into `#savedFilterChips`, and allow saving current filter state.
|
||||
|
||||
- [ ] **Step 3: Command Palette Keyboard Navigation (`ArrowUp`, `ArrowDown`, `Enter`)**
|
||||
In `openCmdPalette()` and keydown listener:
|
||||
Track `activeCmdIndex`. On `ArrowDown`, highlight next item. On `ArrowUp`, highlight previous item. On `Enter`, execute highlighted action.
|
||||
|
||||
- [ ] **Step 4: Fix Light Theme in `styles.css`**
|
||||
In `[data-theme="light"]`:
|
||||
Set `--surf: #f8fafc`, `--obsidian-surface: #ffffff`, `--on-s: #0f172a`, `--on-sv: #475569`, `--sc: rgba(241, 245, 249, 0.9)`, `--out: rgba(0, 0, 0, 0.1)`.
|
||||
Ensure light theme is readable and contrast-compliant.
|
||||
|
||||
- [ ] **Step 5: Add `history` Icon & Fix Notification Read**
|
||||
Add `history` SVG to `ICONS`.
|
||||
When clicking a notification in `pop()`, call `POST /khira/api/notifications/${id}/read` to persist read state.
|
||||
In `escD(id)`, safely handle `e.message` with `(e.message || e.description || '').slice(0, 60)`.
|
||||
|
||||
---
|
||||
|
||||
### Task 9: Full System Verification & Regression Testing
|
||||
|
||||
**Files:**
|
||||
- Test all endpoints with curl and verify in browser
|
||||
|
||||
- [ ] **Step 1: Verify Go backend compiles with zero errors**
|
||||
Run: `go build -o bin/nexus-gateway cmd/nexus-gateway/main.go`
|
||||
|
||||
- [ ] **Step 2: Restart Gateway and test all fixed endpoints**
|
||||
1. Test demo logins (`admin`, `charlie`, `alice`, `bob`, `diana`, `eva`).
|
||||
2. Test ticket filtering (`?assignee=admin`, `?type=general`, `?status=open`).
|
||||
3. Test backup sync and rollback without data loss.
|
||||
4. Test moving a ticket from in_progress to open.
|
||||
5. Test ticket creation with image attachment.
|
||||
6. Test profile photo upload.
|
||||
7. Test Turnstile verification with dummy test key.
|
||||
8. Verify UI responsiveness, 60 FPS scrolling, and debounced search.
|
||||
Reference in New Issue
Block a user