#17 Kimi Code · Kimi K3 Max (Sub-Agents + AgentSwarm)

Features

2026-07-19-kimi-code-k3 · Kimi Code 0.27.0

Fully Working

134

All acceptance criteria met

Not Working

8

Feature missing or broken

Partially Done

1

Some criteria met, gaps remain

Feature Score

94%

134 of 143 tests pass (weighted)

We ran 143 end-to-end acceptance tests across 18 suites using Playwright MCP browser testing. Each test was executed non-scripted by an independent QA run against the local dev build.

Suite Summary

SuitePassTotalScore
S1Smoke Tests1010
100%
S2Admin Authentication1010
100%
S3Admin Product Management67
86%
S4Admin Order Management1011
91%
S5Admin Discount Management66
100%
S6Admin Settings77
100%
S7Storefront Browsing1515
100%
S8Cart Flow1212
100%
S9Checkout Flow1213
92%
S10Customer Account812
67%
S11Inventory Enforcement44
100%
S12Tenant Isolation45
80%
S13Responsive / Mobile88
100%
S14Accessibility1011
91%
S15Admin Collections Management33
100%
S16Admin Customer Management33
100%
S17Admin Pages Management33
100%
S18Admin Analytics Dashboard33
100%

Detailed Results

S1

Smoke Tests

10/10
10
S1-01Loads storefront homePASS
S1-02Loads a collection pagePASS
S1-03Loads a product pagePASS
S1-04Loads the cart pagePASS
S1-05Loads the customer login pagePASS
S1-06Loads the admin login pagePASS
S1-07Loads the about pagePASS
S1-08Loads the search pagePASS
S1-09Loads all collections listingPASS
S1-10Has no errors on critical pagesPASS
S1-01: "Welcome to Acme Fashion" hero, featured collections + 8 featured products, no console errors.
S1-02: `/collections/t-shirts` loads ("T-Shirts - Acme Fashion" title), no console errors.
S1-03: `/products/classic-cotton-t-shirt`: title, 24.99 EUR, Size/Color groups, Add to cart.
S1-04: "Your Cart" page title.
S1-05: "Log in" page title with login form.
S1-06: `admin.acme-fashion.test` redirects to `/admin/login`, Email/Password/Remember me form.
S1-07: "About Us - Acme Fashion" title.
S1-08: `/search?q=shirt` → "4 results for shirt".
S1-09: Lists New Arrivals, T-Shirts, Pants & Jeans, Sale.
S1-10: 0 console errors/warnings across all pages visited.
S2

Admin Authentication

10/10
10
S2-01Can log in as adminPASS
S2-02Shows error for invalid credentialsPASS
S2-03Shows error for empty emailPASS
S2-04Shows error for empty passwordPASS
S2-05Redirects unauthenticated to login from dashboardPASS
S2-06Redirects unauthenticated to login from productsPASS
S2-07Can log outPASS
S2-08Can navigate through admin sidebar sectionsPASS
S2-09Can navigate to analyticsPASS
S2-10Can navigate to themesPASS
S2-01: `admin@acme.test`/`password` → Dashboard with KPIs + Recent orders table.
S2-02: Wrong password → "Invalid credentials." alert.
S2-03: "The email field is required."
S2-04: "The password field is required."
S2-05: Redirects to `/admin/login`.
S2-06: Redirects to `/admin/login`.
S2-07: User menu → "Log out" → back at `/admin/login`.
S2-08: Products/Orders/Customers/Discounts/Settings all load with matching titles.
S2-09: `/admin/analytics` loads with "Analytics" title.
S2-10: `/admin/themes` → "Themes" heading, Default Theme Published.
S3

Admin Product Management

6/7
61
S3-01Shows product list with seeded productsPASS
S3-02Can create a new productPASS
S3-03Can edit an existing productPASS
S3-04Can archive a productFAIL
S3-05Shows draft products only in adminPASS
S3-06Can search products in adminPASS
S3-07Can filter products by statusPASS
S3-01: 21 products incl. Classic Cotton T-Shirt + Premium Slim Fit Jeans.
S3-02: Title/SKU/Price(2999)/Qty(50) → "Product saved.", product id 26 created.
S3-03: Title edit → "...Updated" → "Product saved." → reverted, both saves succeeded.
S3-04: Status `<select>` never persists on save: choosing "Archived" or "Active" always reloads back to "Draft", reproduced twice; product could not be archived.
S3-05: "Unreleased Winter Jacket" (Draft) visible in admin; storefront search for "Unreleased" → 0 results.
S3-06: "Cotton" → only "Classic Cotton T-Shirt".
S3-07: Draft tab → 2 products; Active tab → 18 products incl. Classic Cotton T-Shirt.
S4

Admin Order Management

10/11
101
S4-01Shows order list with seeded ordersPASS
S4-02Can filter orders by statusFAIL
S4-03Shows order detail with line itemsPASS
S4-04Shows order timeline eventsPASS
S4-05Can create a fulfillmentPASS
S4-06Can process a refundPASS
S4-07Shows customer info in order detailPASS
S4-08Can confirm bank transfer paymentPASS
S4-09Shows fulfillment guard for unpaid orderPASS
S4-10Can mark fulfillment as shippedPASS
S4-11Can mark fulfillment as deliveredPASS
S4-01: 16 orders incl. #1001, with Payment/Fulfillment columns.
S4-02: "Paid" tab incorrectly includes #1008 (Partially Refunded) and excludes #1002/#1007, both genuinely Paid.
S4-03: #1001: line item, Subtotal/Shipping/Tax/Total (54.97 EUR), customer + addresses.
S4-04: Timeline shows "Order placed" + "Payment received", updated live with fulfillment/refund events.
S4-05: DHL + tracking on #1001 → modal closes cleanly, "Fulfillment #8" created, timeline updated.
S4-06: 10.00 EUR refund on #1001 → "Partially Refunded" status, refund line shown, timeline updated.
S4-07: John Doe / customer@acme.test with shipping/billing addresses.
S4-08: Pending #1005 → "Confirm payment" → status Paid, "Payment received" timeline entry added.
S4-09: Pending #1005 shows fulfillment-blocked warning; "Create fulfillment" button disabled.
S4-10: "Mark as shipped" (tracking pre-filled) → fulfillment "Shipped", timeline updated.
S4-11: "Mark as delivered" → fulfillment "Delivered", order fulfillment status "Fulfilled".
S5

Admin Discount Management

6/6
6
S5-01Shows seeded discount codesPASS
S5-02Can create percentage discountPASS
S5-03Can create fixed amount discountPASS
S5-04Can create free shipping discountPASS
S5-05Can edit a discountPASS
S5-06Shows discount status indicatorsPASS
S5-01: WELCOME10 (10%), FLAT5 (5.00 EUR), FREESHIP, EXPIRED20 (Expired), MAXED (5/5) all present.
S5-02: `E2ETEST25-KIMIK3` 25% → saved, "Active" status.
S5-03: `E2EFLAT10-KIMIK3` 1000 cents (10.00) → saved.
S5-04: `E2EFREESHIP-KIMIK3` → Value field correctly hidden for Free shipping; saved.
S5-05: WELCOME10 value 10 → 15 → "Discount saved."; reverted to 10.
S5-06: "Active" badges on active codes; EXPIRED20 "Expired"; MAXED "5 / 5" usage.
S6

Admin Settings

7/7
7
S6-01Can view store settingsPASS
S6-02Can update store namePASS
S6-03Can view shipping zonesPASS
S6-04Can add new shipping ratePASS
S6-05Can view tax settingsPASS
S6-06Can update tax inclusion settingPASS
S6-07Can view domain settingsPASS
S6-01: "Settings" heading, "Acme Fashion" store name, General/Domains/Checkout/Notifications tabs.
S6-02: → "Acme Fashion Updated" → saved, persisted after reload; reverted.
S6-03: Domestic (DE) 4.99/9.99; EU 8.99; Rest of World 14.99.
S6-04: "Overnight Shipping" 14.99 added to Domestic zone, appears immediately.
S6-05: Manual tax-rate mode, default 19%, "Prices include tax" toggle checked.
S6-06: Toggled off → "Settings saved."; reverted, saved again.
S6-07: "Domains" tab lists `acme-fashion.test` (Primary) and `admin.acme-fashion.test`, both "managed" TLS.
S7

Storefront Browsing

15/15
15
S7-01Shows featured products on homePASS
S7-02Shows collection with product gridPASS
S7-03Can navigate from collection to productPASS
S7-04Shows product detail with variantsPASS
S7-05Shows size and color option valuesPASS
S7-06Updates price when variant changesPASS
S7-07Shows search results for valid queryPASS
S7-08Shows no results for invalid queryPASS
S7-09Does not show draft products on collectionsPASS
S7-10Does not show draft products in searchPASS
S7-11Shows out of stock messagingPASS
S7-12Shows backorder messagingPASS
S7-13Shows new arrivals collectionPASS
S7-14Shows static about pagePASS
S7-15Navigates between pages using navPASS
S7-01: "Welcome to Acme Fashion" + featured products incl. Classic Cotton T-Shirt 24.99 EUR.
S7-02: `/collections/t-shirts`: "T-Shirts", 4 products with prices, sort + filters sidebar.
S7-03: Clicked Classic Cotton T-Shirt from collection grid → PDP loaded correctly.
S7-04: "Classic Cotton T-Shirt", 24.99 EUR, "Size" and "Color" groups.
S7-05: S/M/L/XL and White/Black/Navy all individually selectable buttons.
S7-06: Premium Slim Fit Jeans: 79.99 EUR sale + 99.99 EUR compare-at + "On sale" badge.
S7-07: "shirt" → "4 results".
S7-08: "Unreleased" → "0 results" + "No results found" message.
S7-09: Verified via admin cross-check — draft product not present in any collection.
S7-10: `/search?q=Unreleased` → 0 results.
S7-11: Limited Edition Sneakers: "Out of stock", "Sold out" button disabled, size options disabled.
S7-12: Backorder Denim Jacket: "Available on backorder", "Add to cart" enabled.
S7-13: `/collections/new-arrivals` → "New Arrivals" title.
S7-14: `/pages/about` → "About Us - Acme Fashion" title.
S7-15: Nav links navigate correctly, confirmed across multiple suite runs.
S8

Cart Flow

12/12
12
S8-01Can add product to cartPASS
S8-02Can view cart with added itemPASS
S8-03Can update quantity in cartPASS
S8-04Can remove item from cartPASS
S8-05Can add multiple productsPASS
S8-06Can apply WELCOME10 discountPASS
S8-07Shows error for invalid discountPASS
S8-08Shows error for expired discountPASS
S8-09Shows error for maxed out discountPASS
S8-10Can apply free shipping discountPASS
S8-11Can apply FLAT5 discountPASS
S8-12Shows subtotal and total in cartPASS
S8-01: Classic Cotton T-Shirt M/Black → cart drawer opens with item + 24.99 EUR, count → 1.
S8-02: `/cart` displays item + 24.99 EUR.
S8-03: + → qty 2, line total 49.98 EUR.
S8-04: Remove → "Your cart is empty".
S8-05: T-Shirt + Jeans both in cart, Subtotal 104.98 EUR.
S8-06: "WELCOME10 (-10%)" label, -2.49 EUR discount, total 22.50 EUR.
S8-07: "INVALID" → "This discount code is invalid."
S8-08: "EXPIRED20" → "This discount code has expired."
S8-09: "MAXED" → "This discount code has reached its usage limit."
S8-10: "FREESHIP (Free shipping)" label shown.
S8-11: "FLAT5 (-5.00 EUR)", total 19.99 EUR.
S8-12: "Subtotal" and "Total" labels with correct amounts shown throughout.
S9

Checkout Flow

12/13
121
S9-01Completes full checkout (credit card)PASS
S9-02Shows shipping methods for DE addressPASS
S9-03Shows international shipping for non-DEPASS
S9-04Applies discount during checkoutPASS
S9-05Validates required contact emailPASS
S9-06Validates required address fieldsPASS
S9-07Validates invalid postal codeFAIL
S9-08Prevents checkout with empty cartPASS
S9-09Completes checkout with PayPalPASS
S9-10Completes checkout with bank transferPASS
S9-11Shows error for declined cardPASS
S9-12Shows error for insufficient fundsPASS
S9-13Switches between payment methodsPASS
S9-01: `4242 4242 4242 4242` → "Thank you for your order!" Order #1017, total 24.98 EUR.
S9-02: DE/Berlin → Domestic Standard 4.99 / Express 9.99 / Overnight 14.99.
S9-03: US/New York → only "International" 14.99.
S9-04: FLAT5 carries from cart into checkout; total resolves to 24.98 EUR.
S9-05: Empty email → native "Please fill in this field."
S9-06: Native required validation on first_name, last_name, address1, city, postal_code, country_code.
S9-07: `INVALID` postal code + DE accepted, advances straight to shipping step with no format validation.
S9-08: Empty cart shows "Your cart is empty", no Checkout button; direct `/checkout` returns 404.
S9-09: PayPal → "Thank you for your order!" Order #1018, Payment Method "PayPal".
S9-10: Order #1019 with full bank instructions: Mock Bank AG, IBAN, BIC, reference #1019, 7-day note.
S9-11: `4000 0000 0000 0002` → "Payment declined: Your card was declined.", stays on checkout.
S9-12: `4000 0000 0000 9995` → "Payment declined: Your card has insufficient funds."
S9-13: Selecting a method before Continue correctly changes locked-in form/CTA; radios lock after Continue.
S10

Customer Account

8/12
84
S10-01Can register a new customerPASS
S10-02Shows validation for duplicate emailPASS
S10-03Shows validation for mismatched passwordsPASS
S10-04Can log in as customerPASS
S10-05Shows error for invalid customer credentialsPASS
S10-06Redirects unauthenticated customersPASS
S10-07Shows order history for customerPASS
S10-08Shows order detail for customerFAIL
S10-09Can view addressesPASS
S10-10Can add a new addressFAIL
S10-11Can edit an existing addressFAIL
S10-12Can log outFAIL
S10-01: "Welcome back, New Customer E2E!" on `/account` after registration.
S10-02: `customer@acme.test` → "The email has already been taken."
S10-03: "The password field confirmation does not match."
S10-04: `customer@acme.test` → "Welcome back, John Doe!" with recent orders.
S10-05: Wrong password → "Invalid credentials." shown inline.
S10-06: `/account` (logged out) → redirected to `/account/login`.
S10-07: #1015, #1010, #1001, #1002, #1004 listed for John Doe with status and total.
S10-08: **"View"/order-number links have correct `href` but clicking does nothing; direct URL navigation to `/account/orders/1001` works, so it's a broken client-side click handler.**
S10-09: "Home (Default)" and "Work" addresses listed.
S10-10: "Add new address" produces no modal, no navigation, no console error — click silently swallowed.
S10-11: "Edit" on any address is equally inert (same symptom).
S10-12: "Log out" button does nothing on click, but calling `form.submit()` directly via JS logs out correctly — same client-side click-swallowing bug as S10-08.
S11

Inventory Enforcement

4/4
4
S11-01Blocks add-to-cart for out-of-stock denyPASS
S11-02Allows add-to-cart for backorderPASS
S11-03Shows correct stock for in-stock productPASS
S11-04Prevents adding more than available stockPASS
S11-01: Limited Edition Sneakers: "Out of stock", "Sold out" button disabled, size options disabled.
S11-02: Backorder Denim Jacket: "Available on backorder", add-to-cart succeeds at 99.99 EUR.
S11-03: Classic Cotton T-Shirt: "In stock", add-to-cart works reliably.
S11-04: Premium Slim Fit Jeans (8 in stock): quantity increment stops exactly at 8.
S12

Tenant Isolation

4/5
41
S12-01Store only shows its own productsPASS
S12-02Store collections contain store productsPASS
S12-03Admin cannot access other store dataFAIL
S12-04Search only returns current store productsPASS
S12-05Customer accounts scoped to storePASS
S12-01: `acme-electronics.test` shows only Electronics products, no Fashion products.
S12-02: Electronics "Accessories" collection contains only USB-C Cable + Monitor Stand.
S12-03: **Critical.** `acme-electronics.test/admin/products` (shared credentials) displays Acme Fashion's full 21-product catalog; store-switcher also reads "Acme Fashion" on the Electronics domain.
S12-04: Electronics storefront search for "fashion" → 0 results.
S12-05: Fashion's `customer@acme.test` login rejected on the Electronics storefront.
S13

Responsive / Mobile

8/8
8
S13-01Storefront home on mobilePASS
S13-02Product page stacks on mobilePASS
S13-03Can add to cart on mobilePASS
S13-04Cart page works on mobilePASS
S13-05Checkout flow on mobilePASS
S13-06Admin login on tabletPASS
S13-07Admin sidebar on tabletPASS
S13-08Collection page on mobilePASS
S13-01: Hamburger "Open navigation menu" button, store name visible, no horizontal scroll.
S13-02: Title/price/variant selectors/Add to cart all visible, no horizontal scroll.
S13-03: Cart drawer opens with item added, cart count updates.
S13-04: Item + Checkout button visible, no horizontal scroll.
S13-05: Address form, "Standard Shipping" method reachable, no horizontal scroll at any step.
S13-06: At 768×1024: Dashboard loads, no horizontal scroll.
S13-07: Products and Orders sections load with correct titles.
S13-08: T-Shirts collection: title, products, "Toggle filters" button present, no horizontal scroll.
S14

Accessibility

10/11
101
S14-01Home page has no JS errorsPASS
S14-02Home page has proper heading hierarchyPASS
S14-03Product page has ARIA labelsPASS
S14-04Product page images have alt textPARTIAL
S14-05Customer login form has accessible labelsPASS
S14-06Admin login form has accessible labelsPASS
S14-07Checkout form has accessible labelsPASS
S14-08Checkout validation errors accessiblePASS
S14-09Can navigate with keyboard onlyPASS
S14-10Cart page has no console errorsPASS
S14-11Search page has proper form labelsPASS
S14-01: 0 console errors/warnings on home page load.
S14-02: Exactly one H1, logical H2/H3 sequence with no level skips.
S14-03: `group "Size"` / `group "Color"` with accessible names; "Add to cart" properly labelled.
S14-04: No real product photography; placeholder is an `aria-hidden` SVG inside a named region, so the region has an accessible name but no per-product image name is exposed.
S14-05: "Email" and "Password" labels associated with inputs.
S14-06: "Email" and "Password" labels associated with inputs.
S14-07: Email + all address/payment fields have accessible names.
S14-08: Native required-field validation messages tied to their inputs.
S14-09: First Tab focuses "Skip to main content" with a visible focus ring.
S14-10: 0 console errors/warnings on cart page load.
S14-11: Search input exposed as `searchbox "Search products"` with accessible name.
S15

Admin Collections Management

3/3
3
S15-01Shows collection list with seeded collectionsPASS
S15-02Can create a new collectionPASS
S15-03Can edit a collectionPASS
S15-01: New Arrivals, T-Shirts, Pants & Jeans, Sale listed with product counts.
S15-02: "E2E Test Collection KimiK3" created and appears in the list.
S15-03: T-Shirts description changed and persisted; reverted to original.
S16

Admin Customer Management

3/3
3
S16-01Shows the customer listPASS
S16-02Shows customer detail with order historyPASS
S16-03Shows customer addressesPASS
S16-01: "customer@acme.test" and "John Doe" present with order count (5) and total spent (734.37 EUR).
S16-02: John Doe, full order history table incl. "#1001" with correct numeric order links.
S16-03: "Addresses" section shows Home (Default) + Work.
S17

Admin Pages Management

3/3
3
S17-01Shows the pages listPASS
S17-02Can create a new pagePASS
S17-03Can edit an existing pagePASS
S17-01: About Us, FAQ, Shipping & Returns, Privacy Policy, Terms of Service listed.
S17-02: "FAQ E2E KimiK3" created; colliding handle shows clean inline "The handle has already been taken." error instead of crashing.
S17-03: About Us body edited → saved, persisted; reverted to original.
S18

Admin Analytics Dashboard

3/3
3
S18-01Shows analytics dashboardPASS
S18-02Shows sales dataPASS
S18-03Shows conversion funnel dataPASS
S18-01: "Analytics" heading with Date range filter, KPI tiles, and charts.
S18-02: KPI tiles: Total Sales 9,063.26 EUR, Orders 148, AOV 61.23 EUR, Conversion Rate 4.4%.
S18-03: Full funnel present: Visits 3,332 → Added to cart 743 (22.3%) → Checkout started 350 (10.5%) → Checkout completed 148 (4.4%).