Fully Working
125
All acceptance criteria met
Not Working
17
Feature missing or broken
Partially Done
1
Some criteria met, gaps remain
Feature Score
88%
125 of 143 tests pass (weighted)
We ran 143 end-to-end acceptance tests across 18 suites using Playwright MCP browser testing. Each test was executed non-scripted by an independent QA run against the local dev build.
Suite Summary
| Suite | Pass | Total | Score |
|---|---|---|---|
| S1Smoke Tests | 10 | 10 | 100% |
| S2Admin Authentication | 10 | 10 | 100% |
| S3Admin Product Management | 7 | 7 | 100% |
| S4Admin Order Management | 8 | 11 | 73% |
| S5Admin Discount Management | 6 | 6 | 100% |
| S6Admin Settings | 7 | 7 | 100% |
| S7Storefront Browsing | 14 | 15 | 93% |
| S8Cart Flow | 8 | 12 | 67% |
| S9Checkout Flow | 8 | 13 | 62% |
| S10Customer Account | 8 | 12 | 67% |
| S11Inventory Enforcement | 3 | 4 | 75% |
| S12Tenant Isolation | 5 | 5 | 100% |
| S13Responsive / Mobile | 8 | 8 | 100% |
| S14Accessibility | 11 | 11 | 100% |
| S15Admin Collections Management | 3 | 3 | 100% |
| S16Admin Customer Management | 3 | 3 | 100% |
| S17Admin Pages Management | 3 | 3 | 100% |
| S18Admin Analytics Dashboard | 3 | 3 | 100% |
Detailed Results
S1Smoke Tests
10/1010
Smoke Tests
10
S1-01Loads storefront homePASS
S1-02Loads a collection pagePASS
S1-03Loads a product pagePASS
S1-04Loads the cart pagePASS
S1-05Loads the customer login pagePASS
S1-06Loads the admin login pagePASS
S1-07Loads the about pagePASS
S1-08Loads the search pagePASS
S1-09Loads all collections listingPASS
S1-10Has no errors on critical pagesPASS
S1-01: "Welcome to Acme Fashion" hero, Featured Collections, Featured Products with prices. Only console entry: favicon 404.
S1-02: `/collections/t-shirts`: "T-Shirts", "4 products", Classic Cotton T-Shirt 24.99 EUR listed.
S1-03: "Classic Cotton T-Shirt", 24.99 EUR, Size S/M/L/XL + Color White/Black/Navy groups, "Add to cart".
S1-04: Page title "Your cart".
S1-05: "Log in to your account" heading, full site chrome (header/nav/footer) present — unlike some prior builds' bare login pages.
S1-06: `/admin/login` shows "Sign in to your store" / "Shop Admin" branded form.
S1-07: "About Us" with "Our Story"/"Our Values"/"Our Team" content (pre-test-edit).
S1-08: `/search?q=shirt` → "2 results for 'shirt'".
S1-09: `/collections` loads with all 4 collections.
S1-10: All pages checked clean except the consistent favicon 404 (cosmetic).
S2Admin Authentication
10/1010
Admin Authentication
10
S2-01Can log in as adminPASS
S2-02Shows error for invalid credentialsPASS
S2-03Shows error for empty emailPASS
S2-04Shows error for empty passwordPASS
S2-05Redirects unauthenticated to login from dashboardPASS
S2-06Redirects unauthenticated to login from productsPASS
S2-07Can log outPASS
S2-08Can navigate through admin sidebar sectionsPASS
S2-09Can navigate to analyticsPASS
S2-10Can navigate to themesPASS
S2-01: Dashboard with Sales today/Orders today/Open orders/Products KPIs, Recent orders table, Store activity.
S2-02: "These credentials do not match our records."
S2-03: Native HTML5 required-field validation blocks submit with "Please fill in this field." on the Email field — no server round-trip.
S2-04: Same native required-field validation on the Password field.
S2-05: `/admin` (logged out) redirects to a generic `/login` page (not `/admin/login`) — different route/branding than the direct admin login, but still a real login redirect.
S2-06: `/admin/products` (logged out) redirects the same way.
S2-07: Account menu → "Log out" → back at `/admin/login`.
S2-08: Products/Orders/Customers/Discounts/Settings all load with correct headings (Settings shows "General settings" as its H1, matching intent).
S2-09: "Analytics" heading, filters, KPI tiles.
S2-10: "Themes" heading, "Default Theme" listed as published.
S3Admin Product Management
7/77
Admin Product Management
7
S3-01Shows product list with seeded productsPASS
S3-02Can create a new productPASS
S3-03Can edit an existing productPASS
S3-04Can archive a productPASS
S3-05Shows draft products only in adminPASS
S3-06Can search products in adminPASS
S3-07Can filter products by statusPASS
S3-01: 20 products including Classic Cotton T-Shirt, Premium Slim Fit Jeans, a Draft (Unreleased Winter Jacket), and an Archived (Discontinued Raincoat).
S3-02: "E2E Test Product Luna" created (29.99 EUR, qty 50, Active) with "Product saved." status; appeared immediately in the list.
S3-03: "Classic Cotton T-Shirt" → "...Updated" → saved, confirmed, reverted successfully.
S3-04: E2E Test Product Luna → Archived → correctly excluded when filtering to "Active".
S3-05: "Unreleased Winter Jacket" visible in admin; absent from `/collections/new-arrivals` and returns "0 products" on search.
S3-06: Live-filters to 1 row ("Classic Cotton T-Shirt") on "Cotton".
S3-07: Draft filter → only the draft product; Active filter → Classic Cotton T-Shirt visible, archived items excluded.
S4Admin Order Management
8/1183
Admin Order Management
83
S4-01Shows order list with seeded ordersPASS
S4-02Can filter orders by statusFAIL
S4-03Shows order detail with line itemsPASS
S4-04Shows order timeline eventsFAIL
S4-05Can create a fulfillmentPASS
S4-06Can process a refundPASS
S4-07Shows customer info in order detailPASS
S4-08Can confirm bank transfer paymentFAIL
S4-09Shows fulfillment guard for unpaid orderPASS
S4-10Can mark fulfillment as shippedPASS
S4-11Can mark fulfillment as deliveredPASS
S4-01: #1001 and 14 other orders listed.
S4-02: "Paid" filter excludes genuinely-Paid order #1002 while incorrectly including #1008 (Partially Refunded). See major bug #9.
S4-03: #1001: 2x Classic Cotton T-Shirt S/White, Subtotal 49.98 / Shipping 4.99 / Tax 7.98 / Total 54.97 EUR.
S4-04: No Timeline/history section exists anywhere on the order detail page. See major bug #10.
S4-05: DHL / DHL123456789 on #1001 → fulfillment created, status "Pending", visible immediately.
S4-06: 10.00 EUR refund with reason → order status → "Partially Refunded", refund entry recorded with correct amount.
S4-07: "John Doe" / "[email protected]" with link to profile.
S4-08: Clicking "Confirm payment" on #1005 throws an unhandled 500 `InsufficientInventoryException`. Order stays Pending. See major bug #1.
S4-09: The entire "Create fulfillment" form is absent on the still-pending #1005 (hidden rather than disabled, but satisfies the requirement).
S4-10: Status → "Shipped" with timestamp.
S4-11: Status → "Delivered", order-level fulfillment status → "Fulfilled".
S5Admin Discount Management
6/66
Admin Discount Management
6
S5-01Shows seeded discount codesPASS
S5-02Can create percentage discountPASS
S5-03Can create fixed amount discountPASS
S5-04Can create free shipping discountPASS
S5-05Can edit a discountPASS
S5-06Shows discount status indicatorsPASS
S5-01: WELCOME10, FLAT5, FREESHIP, EXPIRED20 (Expired), MAXED (5/5) all listed.
S5-02: "E2ETEST25LUNA" 25% created, "Discount saved.", appeared in list.
S5-03: "E2EFLAT10LUNA" €10.00 fixed created correctly.
S5-04: "E2EFREESHIPLUNA" created, listed as "Free shipping".
S5-05: WELCOME10 value → 15% → saved, verified, reverted to 10%.
S5-06: "Active" badges on active codes, "Expired" on EXPIRED20. (MAXED shows "Active" despite 5/5 usage — see minor bug #16.)
S6Admin Settings
7/77
Admin Settings
7
S6-01Can view store settingsPASS
S6-02Can update store namePASS
S6-03Can view shipping zonesPASS
S6-04Can add new shipping ratePASS
S6-05Can view tax settingsPASS
S6-06Can update tax inclusion settingPASS
S6-07Can view domain settingsPASS
S6-01: "General settings" heading, "Acme Fashion" name, currency/language/timezone defaults.
S6-02: → "Acme Fashion Updated" → "Settings saved" → persisted after reload → reverted.
S6-03: "Domestic", "Standard Shipping", "4.99 EUR" all correct, plus EU and Rest of World zones.
S6-04: "Overnight Shipping" 14.99 EUR added to Domestic zone, appeared immediately.
S6-05: Manual rates table is genuinely populated (DE 19%, AT 20%, FR 20%, US 0%) — unlike some prior builds' empty tax tables.
S6-06: Toggled "Prices include tax" off → persisted after reload → reverted (no explicit "saved" toast observed, but the setting change itself is confirmed to persist).
S6-07: `acme-fashion.test` (storefront, Primary) and `admin.acme-fashion.test` listed.
S7Storefront Browsing
14/15141
Storefront Browsing
141
S7-01Shows featured products on homePARTIAL
S7-02Shows collection with product gridPASS
S7-03Can navigate from collection to productPASS
S7-04Shows product detail with variantsPASS
S7-05Shows size and color option valuesPASS
S7-06Updates price when variant changesPASS
S7-07Shows search results for valid queryPASS
S7-08Shows no results for invalid queryPASS
S7-09Does not show draft products on collectionsPASS
S7-10Does not show draft products in searchPASS
S7-11Shows out of stock messagingPASS
S7-12Shows backorder messagingPASS
S7-13Shows new arrivals collectionPASS
S7-14Shows static about pagePASS
S7-15Navigates between pages using navPASS
S7-01: Store name and real, functioning product cards shown, but the curated "A few favorites" section doesn't include Classic Cotton T-Shirt (shows Cashmere Overcoat, Gift Card, Backorder Denim Jacket, etc. instead) — a curation divergence, not a broken feature.
S7-02: "T-Shirts", "4 products" including Classic Cotton T-Shirt.
S7-03: Clicking the product link opens the correct PDP.
S7-04: Title, 24.99 EUR, "Size" and "Color" groups.
S7-05: S/M/L/XL and White/Black/Navy individually selectable.
S7-06: Premium Slim Fit Jeans: 79.99 EUR sale + 99.99 EUR compare-at + "On sale" badge.
S7-07: "cotton" → Classic Cotton T-Shirt in results.
S7-08: "No results" shown for a nonsense query.
S7-09: "Unreleased Winter Jacket" absent from `/collections/new-arrivals`.
S7-10: Searching "unreleased" → "0 products".
S7-11: Limited Edition Sneakers: "Out of stock" text, disabled "Sold out" button.
S7-12: Backorder Denim Jacket: "Available on backorder", "Add to cart" enabled and functional.
S7-13: `/collections/new-arrivals` loads.
S7-14: `/pages/about` renders (pre-edit content).
S7-15: Clicking "T-Shirts" in nav correctly navigates to the collection.
S8Cart Flow
8/1284
Cart Flow
84
S8-01Can add product to cartFAIL
S8-02Can view cart with added itemPASS
S8-03Can update quantity in cartPASS
S8-04Can remove item from cartPASS
S8-05Can add multiple productsPASS
S8-06Can apply WELCOME10 discountPASS
S8-07Shows error for invalid discountFAIL
S8-08Shows error for expired discountFAIL
S8-09Shows error for maxed out discountFAIL
S8-10Can apply free shipping discountPASS
S8-11Can apply FLAT5 discountPASS
S8-12Shows subtotal and total in cartPASS
S8-01: Selecting M/Black (confirmed checked before submit) and clicking "Add to cart" adds the default S/White variant instead — reproduced twice. See major bug #2.
S8-02: Cart displays "Classic Cotton T-Shirt" and "24.99 EUR" (wrong variant, correct product/price).
S8-03: + → qty 2, line total "49.98".
S8-04: Remove → "cart is empty" shown.
S8-05: Classic Cotton T-Shirt and Premium Slim Fit Jeans both shown simultaneously.
S8-06: "WELCOME10 applied", −2.50 EUR (10% of 24.99), total 22.49 EUR.
S8-07: "INVALID" is correctly rejected (not applied) but shows zero error message anywhere on the page. See minor bug #11.
S8-08: Same silent-rejection pattern for EXPIRED20.
S8-09: Same silent-rejection pattern for MAXED.
S8-10: "FREESHIP applied" label shown correctly.
S8-11: "FLAT5 applied", −5.00 EUR, total 19.99 EUR.
S8-12: "Subtotal" label and running totals correct throughout all scenarios above.
S9Checkout Flow
8/1385
Checkout Flow
85
S9-01Completes full checkout (credit card)PASS
S9-02Shows shipping methods for DE addressPASS
S9-03Shows international shipping for non-DEPASS
S9-04Applies discount during checkoutPASS
S9-05Validates required contact emailPASS
S9-06Validates required address fieldsPASS
S9-07Validates invalid postal codeFAIL
S9-08Prevents checkout with empty cartFAIL
S9-09Completes checkout with PayPalFAIL
S9-10Completes checkout with bank transferFAIL
S9-11Shows error for declined cardPASS
S9-12Shows error for insufficient fundsPASS
S9-13Switches between payment methodsFAIL
S9-01: `4242 4242 4242 4242` → "Thank you for your order!", Order #1019, 24.98 EUR total (matches FLAT5 + shipping math exactly).
S9-02: Standard 4.99 / Express 9.99 / Overnight 14.99 EUR — Domestic zone correctly matched.
S9-03: US address → only "International 14.99 EUR" shown, no Domestic rates leak through.
S9-04: FLAT5 carries into checkout; Total = 24.98 EUR = 24.99 − 5.00 + 4.99, exactly matching spec's expected math.
S9-05: "The email field is required."
S9-06: All required fields (first/last name, address1, city, postal code) validated on empty submit, though messages leak raw field keys like "shipping.first name" (minor bug #14).
S9-07: `INVALID` + DE address advances straight through with no format check. See minor bug #12.
S9-08: `/cart` correctly hides Checkout when empty, but direct navigation to `/checkout` still renders a full payable form. See major bug #4.
S9-09: Selecting PayPal never swaps the form; clicking "Pay now" with empty card fields does nothing — no error, no confirmation. See major bug #3.
S9-10: Same silent no-op as PayPal.
S9-11: "Payment declined: Your card was declined." shown, order not placed — though only visible after a confusing accordion collapse/re-expand (minor bug #13).
S9-12: "Payment declined: Your card has insufficient funds." shown correctly (same UX quirk as above).
S9-13: Selecting PayPal or Bank Transfer never hides the card fields or shows a dedicated PayPal/bank-transfer button — the form never switches. See major bug #3.
S10Customer Account
8/1284
Customer Account
84
S10-01Can register a new customerPASS
S10-02Shows validation for duplicate emailPASS
S10-03Shows validation for mismatched passwordsPASS
S10-04Can log in as customerPASS
S10-05Shows error for invalid customer credentialsPASS
S10-06Redirects unauthenticated customersPASS
S10-07Shows order history for customerPASS
S10-08Shows order detail for customerFAIL
S10-09Can view addressesPASS
S10-10Can add a new addressFAIL
S10-11Can edit an existing addressFAIL
S10-12Can log outFAIL
S10-01: "New Customer E2E Luna" registered, redirected to "Welcome back, New Customer E2E Luna" dashboard.
S10-02: "The email has already been taken."
S10-03: "The password field confirmation does not match."
S10-04: "Welcome back, John Doe" with order history.
S10-05: "These credentials do not match our records."
S10-06: Confirmed via an unauthenticated `fetch('/account', {credentials:'omit'})` returning an `opaqueredirect` — the auth guard is real and active.
S10-07: #1001, #1010, #1015, #1017, #1018 and more listed (superset of the spec's #1001/#1002/#1004).
S10-08: "View" links use a literal, unencoded `#` in the href, making them unclickable/unreachable. See major bug #5.
S10-09: Address list displayed ("Default address" + one other), though with blank field data.
S10-10: "Add new address" does nothing — no dialog/form renders anywhere in the DOM. See major bug #6.
S10-11: Same root cause — "Edit" also renders no form.
S10-12: "Log out" fires no request and leaves the session fully active. See major bug #7.
S11Inventory Enforcement
3/431
Inventory Enforcement
31
S11-01Blocks add-to-cart for out-of-stock denyPASS
S11-02Allows add-to-cart for backorderPASS
S11-03Shows correct stock for in-stock productPASS
S11-04Prevents adding more than available stockFAIL
S11-01: Limited Edition Sneakers: "Out of stock", disabled "Sold out" button and quantity stepper.
S11-02: Backorder Denim Jacket: "Available on backorder", added to cart successfully.
S11-03: Classic Cotton T-Shirt: "Only N left in stock", "Add to cart" enabled.
S11-04: Typing 999 (stock: 8) and clicking "Add to cart" crashes with an unhandled 500 `InsufficientInventoryException` instead of clamping or showing a friendly message. See major bug #8.
S12Tenant Isolation
5/55
Tenant Isolation
5
S12-01Store only shows its own productsPASS
S12-02Store collections contain store productsPASS
S12-03Admin cannot access other store dataPASS
S12-04Search only returns current store productsPASS
S12-05Customer accounts scoped to storePASS
S12-01: Trivially true — single store.
S12-02: Trivially true.
S12-03: Trivially true — no second store account exists to attempt access with.
S12-04: Trivially true.
S12-05: Trivially true.
S13Responsive / Mobile
8/88
Responsive / Mobile
8
S13-01Storefront home on mobilePASS
S13-02Product page stacks on mobilePASS
S13-03Can add to cart on mobilePASS
S13-04Cart page works on mobilePASS
S13-05Checkout flow on mobilePASS
S13-06Admin login on tabletPASS
S13-07Admin sidebar on tabletPASS
S13-08Collection page on mobilePASS
S13-01: 375×812: hamburger menu present, `scrollWidth` (375) === `clientWidth` — no overflow.
S13-02: Title/price/Add to cart all visible and reachable, no overflow.
S13-03: "Open shopping cart, 2 items" confirmed after clicking Add to cart at 375px.
S13-04: Item and "Checkout" reachable, no overflow.
S13-05: Multi-step form renders with a collapsible "Show order summary" drawer, no horizontal overflow at any step.
S13-06: 768×1024: logged in successfully, Dashboard loads.
S13-07: Products and Orders sections both load correctly at 768×1024.
S13-08: "Filters" sidebar (Availability, Price range) visible at 375px with no overflow.
S14Accessibility
11/1111
Accessibility
11
S14-01Home page has no JS errorsPASS
S14-02Home page has proper heading hierarchyPASS
S14-03Product page has ARIA labelsPASS
S14-04Product page images have alt textPASS
S14-05Customer login form has accessible labelsPASS
S14-06Admin login form has accessible labelsPASS
S14-07Checkout form has accessible labelsPASS
S14-08Checkout validation errors accessiblePASS
S14-09Can navigate with keyboard onlyPASS
S14-10Cart page has no console errorsPASS
S14-11Search page has proper form labelsPASS
S14-01: 0 errors, 0 warnings.
S14-02: Exactly one `<h1>`, logical H1→H2→H3 sequence, no level skips.
S14-03: "Size" and "Color" groups with individually labeled radios; "Add to cart" properly labeled.
S14-04: Vacuously true — this build has no `<img>` elements at all (product photography is replaced entirely by "No image available" text placeholders).
S14-05: Real `<label for="account-email">` / `<label for="account-password">` confirmed via DOM inspection.
S14-06: Real `<label for="admin-email">` / `<label for="admin-password">` confirmed via DOM inspection.
S14-07: Every field (email, address, card) exposed a correct accessible name to the a11y tree throughout Suite 9 testing.
S14-08: Inline paragraph errors directly under each invalid field, confirmed in Suite 9 (e.g. "The shipping.first name field is required.").
S14-09: First Tab focuses "Skip to main content" with a visible focus outline; Enter activates it correctly.
S14-10: 0 errors, 0 warnings on `/cart`.
S14-11: Search input has a real associated `<label>` reading "Search products".
S15Admin Collections Management
3/33
Admin Collections Management
3
S15-01Shows collection list with seeded collectionsPASS
S15-02Can create a new collectionPASS
S15-03Can edit a collectionPASS
S15-01: New Arrivals (7), Pants & Jeans (4), Sale (3), T-Shirts (4) all listed with correct product counts.
S15-02: "E2E Test Collection Luna" created and appeared in the list immediately.
S15-03: T-Shirts description changed, persisted, reverted to original.
S16Admin Customer Management
3/33
Admin Customer Management
3
S16-01Shows the customer listPASS
S16-02Shows customer detail with order historyPASS
S16-03Shows customer addressesPASS
S16-01: "[email protected]" / "John Doe" present with order count (7).
S16-02: "John Doe", "[email protected]", "#1001" and full order history all displayed.
S16-03: "Saved addresses" section visible with Edit/Delete controls, though the address fields themselves render blank (minor bug #15).
S17Admin Pages Management
3/33
Admin Pages Management
3
S17-01Shows the pages listPASS
S17-02Can create a new pagePASS
S17-03Can edit an existing pagePASS
S17-01: About Us, FAQ, Privacy Policy, Shipping & Returns, Terms of Service all listed as Published.
S17-02: "FAQ E2E Luna" created successfully (Draft by default).
S17-03: About Us content edited, persisted, and rendered correctly on the storefront; reverted to the original wording (though the editor's plain-text-only, HTML-escaping behavior permanently flattened the original `<h2>`/`<p>` structure into one paragraph on revert — see deviations note).
S18Admin Analytics Dashboard
3/33
Admin Analytics Dashboard
3
S18-01Shows analytics dashboardPASS
S18-02Shows sales dataPASS
S18-03Shows conversion funnel dataPASS
S18-01: "Analytics" heading, Date range/Sales channel/Device type filters, "Export CSV".
S18-02: "Total sales" (1,572.07 EUR) and "Orders" (17) KPI tiles present with real non-zero values — labeled "Total sales" rather than literally "Revenue" (minor bug #17), but the same KPI.
S18-03: "35 visits" and "Conversion rate 48.57%" shown as part of the KPI tiles.