Fully Working
135
All acceptance criteria met
Not Working
4
Feature missing or broken
Partially Done
4
Some criteria met, gaps remain
Feature Score
96%
135 of 143 tests pass (weighted)
We ran 143 end-to-end acceptance tests across 18 suites using Playwright MCP browser testing. Each test was executed non-scripted by an independent QA run against the local dev build.
Suite Summary
| Suite | Pass | Total | Score |
|---|---|---|---|
| S1Smoke Tests | 10 | 10 | 100% |
| S2Admin Authentication | 10 | 10 | 100% |
| S3Admin Product Management | 6 | 7 | 86% |
| S4Admin Order Management | 11 | 11 | 100% |
| S5Admin Discount Management | 6 | 6 | 100% |
| S6Admin Settings | 6 | 7 | 86% |
| S7Storefront Browsing | 15 | 15 | 100% |
| S8Cart Flow | 12 | 12 | 100% |
| S9Checkout Flow | 12 | 13 | 92% |
| S10Customer Account | 10 | 12 | 83% |
| S11Inventory Enforcement | 3 | 4 | 75% |
| S12Tenant Isolation | 4 | 5 | 80% |
| S13Responsive / Mobile | 8 | 8 | 100% |
| S14Accessibility | 10 | 11 | 91% |
| S15Admin Collections Management | 3 | 3 | 100% |
| S16Admin Customer Management | 3 | 3 | 100% |
| S17Admin Pages Management | 3 | 3 | 100% |
| S18Admin Analytics Dashboard | 3 | 3 | 100% |
Detailed Results
S1Smoke Tests
10/1010
Smoke Tests
10
S1-01Loads storefront homePASS
S1-02Loads a collection pagePASS
S1-03Loads a product pagePASS
S1-04Loads the cart pagePASS
S1-05Loads the customer login pagePASS
S1-06Loads the admin login pagePASS
S1-07Loads the about pagePASS
S1-08Loads the search pagePASS
S1-09Loads all collections listingPASS
S1-10Has no errors on critical pagesPASS
S1-01: "Welcome to Acme Fashion" hero, 4 collection cards, 8 featured products (lazy-loaded via Livewire, brief "Loading products" state then renders), no console errors.
S1-02: `/collections/t-shirts`: "T-Shirts", 4 products, sort + filters sidebar.
S1-03: `/products/classic-cotton-t-shirt`: title, 24.99 EUR, labelled Size/Color radiogroups, Add to cart.
S1-04: "Your Cart" heading, empty-state copy when no items.
S1-05: "Log in to your account" with Email/Password + Create-account link.
S1-06: "Sign in to your store" admin form.
S1-07: `/pages/about` → "About Us" with Our Story / Our Values / Our Team sections.
S1-08: `/search?q=shirt` → "4 results for shirt", no facet duplication.
S1-09: Lists New Arrivals, T-Shirts, Pants & Jeans, Sale.
S1-10: No JS exceptions or 500s across all pages checked across the entire session.
S2Admin Authentication
10/1010
Admin Authentication
10
S2-01Can log in as adminPASS
S2-02Shows error for invalid credentialsPASS
S2-03Shows error for empty emailPASS
S2-04Shows error for empty passwordPASS
S2-05Redirects unauthenticated to login from dashboardPASS
S2-06Redirects unauthenticated to login from productsPASS
S2-07Can log outPASS
S2-08Can navigate through admin sidebar sectionsPASS
S2-09Can navigate to analyticsPASS
S2-10Can navigate to themesPASS
S2-01: Dashboard with KPI tiles, orders-over-time chart, top products, conversion funnel.
S2-02: "Invalid credentials" alert tied to the email field.
S2-03: Native HTML5 "Please fill in this field." on the required email input.
S2-04: Native HTML5 required-field validation on password.
S2-05: Redirects to `/admin/login`.
S2-06: Redirects to `/admin/login`.
S2-07: Account menu → "Log out" → `/admin/login`.
S2-08: Products / Orders / Customers / Discounts / Settings all load with correct headings.
S2-09: `/admin/analytics` → "Analytics" with KPIs, revenue chart, top products table.
S2-10: `/admin/themes` loads.
S3Admin Product Management
6/761
Admin Product Management
61
S3-01Shows product list with seeded productsPASS
S3-02Can create a new productPARTIAL
S3-03Can edit an existing productPASS
S3-04Can archive a productPASS
S3-05Shows draft products only in adminPASS
S3-06Can search products in adminPASS
S3-07Can filter products by statusPASS
S3-01: 21 products incl. Classic Cotton T-Shirt + Premium Slim Fit Jeans; status/type filters, pagination.
S3-02: Entering price as `29.99` fails silent server-side validation (`variants.0.price must be an integer`) with **no visible error rendered**; Save appears to do nothing. Re-entering the price in minor units (`2999`) saves correctly, creating product id 26 with the correct 29.99 EUR price. Underlying feature works once the (undocumented) minor-units format is used.
S3-03: Renamed "Classic Cotton T-Shirt" → "...Updated" and back; saved and persisted correctly both times, no crash (a prior build on this seed crashed here with `DuplicateSkuException` - fixed in this build). No confirmation toast shown, but the save is real.
S3-04: Archived product 26 via the dedicated "Archive" button + confirm dialog; "Active" filter correctly excludes it, "Archived" filter correctly includes it.
S3-05: "Unreleased Winter Jacket" visible in admin; storefront search for "Unreleased" → 0 results.
S3-06: "Cotton" → only "Classic Cotton T-Shirt" shown.
S3-07: "Active" filter excludes the archived test product; "Archived" filter includes it. Filtering works correctly (unlike a prior build on this seed where the status filter was cosmetic only).
S4Admin Order Management
11/1111
Admin Order Management
11
S4-01Shows order list with seeded ordersPASS
S4-02Can filter orders by statusPASS
S4-03Shows order detail with line itemsPASS
S4-04Shows order timeline eventsPASS
S4-05Can create a fulfillmentPASS
S4-06Can process a refundPASS
S4-07Shows customer info in order detailPASS
S4-08Can confirm bank transfer paymentPASS
S4-09Shows fulfillment guard for unpaid orderPASS
S4-10Can mark fulfillment as shippedPASS
S4-11Can mark fulfillment as deliveredPASS
S4-01: 19+ orders incl. #1001, with payment/fulfillment columns.
S4-02: "Paid" tab correctly excludes a Partially-Refunded order and a Cancelled seed order; filtering works correctly.
S4-03: #1001: line item, Subtotal/Discount/Shipping/Tax/Total (54.97 EUR).
S4-04: Activity timeline: "Order placed", later "Payment Captured", "Refund Processed" entries append correctly.
S4-05: DHL + DHL123456789 → "Fulfillment created." toast, tracking visible, "Mark shipped" button appears.
S4-06: 10.00 EUR partial refund → "Refund processed." toast, status → "Partially Refunded", timeline entry added.
S4-07: John Doe / customer@acme.test shown with shipping/billing addresses.
S4-08: Pending #1005 → native confirm dialog → "Confirm payment" → "Bank transfer payment confirmed." toast, status → Paid.
S4-09: Pending #1005 shows "Cannot create fulfillment - Payment must be confirmed before items can be fulfilled. Current financial status: Pending."; no fulfillment button shown.
S4-10: "Mark shipped" → "Fulfillment marked as shipped." toast, "Mark delivered" appears.
S4-11: "Mark delivered" → fulfillment status "Delivered", order fulfillment status → Fulfilled.
S5Admin Discount Management
6/66
Admin Discount Management
6
S5-01Shows seeded discount codesPASS
S5-02Can create percentage discountPASS
S5-03Can create fixed amount discountPASS
S5-04Can create free shipping discountPASS
S5-05Can edit a discountPASS
S5-06Shows discount status indicatorsPASS
S5-01: WELCOME10 (10%), FLAT5 (5.00 EUR), FREESHIP, EXPIRED20 (Expired), MAXED (5/5) all present.
S5-02: `E2ETEST25-GPT56ULTRA` 25% → redirected to edit page, saved.
S5-03: `E2EFLAT10-GPT56ULTRA` 1000 (10.00 EUR) → saved; value field correctly relabels to "Amount (minor units)" when Fixed amount selected.
S5-04: `E2EFREESHIP-GPT56ULTRA` → saved.
S5-05: WELCOME10 value 10 → 15 → saved and persisted on reload; reverted to 10.
S5-06: "Active" badges for active codes; EXPIRED20 shows "Expired".
S6Admin Settings
6/761
Admin Settings
61
S6-01Can view store settingsPASS
S6-02Can update store namePARTIAL
S6-03Can view shipping zonesPASS
S6-04Can add new shipping ratePASS
S6-05Can view tax settingsPASS
S6-06Can update tax inclusion settingPASS
S6-07Can view domain settingsPASS
S6-01: "Store Settings": Acme Fashion, store handle (disabled), currency/locale/timezone; General/Domains/Shipping/Taxes/Checkout/Notifications tabs.
S6-02: → "Acme Fashion Updated", saved and persisted correctly on reload, but **no confirmation toast/message shown at all**; reverted back to "Acme Fashion" (also silent).
S6-03: Domestic (DE) Standard 4.99 + Express 9.99; EU 8.99; Rest of World (US, GB, CA, AU) International 14.99.
S6-04: Modal dialog: "Overnight Shipping" 14.99 added to Domestic zone, appears immediately in the rate table and later at checkout.
S6-05: Manual tax-rate mode, "Prices include tax" toggle (checked by default).
S6-06: Toggled off → "Tax settings saved." confirmation shown (note: label-click was needed as clicking the switch input directly intercepted); reverted, saved again.
S6-07: Domains tab: `acme-fashion.test` (Primary/Storefront), `shop.test` (Storefront), `admin.acme-fashion.test` (Admin), all Managed TLS.
S7Storefront Browsing
15/1515
Storefront Browsing
15
S7-01Shows featured products on homePASS
S7-02Shows collection with product gridPASS
S7-03Can navigate from collection to productPASS
S7-04Shows product detail with variantsPASS
S7-05Shows size and color option valuesPASS
S7-06Updates price when variant changesPASS
S7-07Shows search results for valid queryPASS
S7-08Shows no results for invalid queryPASS
S7-09Does not show draft products on collectionsPASS
S7-10Does not show draft products in searchPASS
S7-11Shows out of stock messagingPASS
S7-12Shows backorder messagingPASS
S7-13Shows new arrivals collectionPASS
S7-14Shows static about pagePASS
S7-15Navigates between pages using navPASS
S7-01: 8 featured products incl. Cashmere Overcoat, Gift Card, Bucket Hat, Wide Leg Trousers (on sale), etc.
S7-02: t-shirts: 4 products + sort/filter sidebar.
S7-03: Clicked "Classic Cotton T-Shirt" from the T-Shirts collection grid → PDP with title/price/variant selectors/Add to cart.
S7-04: Properly labelled `Size` and `Color` radiogroups (e.g. `group "Size: S"`, `group "Color: White"`) - not raw SKU codes.
S7-05: S/M/L/XL and White/Black/Navy shown as distinct, individually selectable radio options within their labelled groups.
S7-06: Premium Slim Fit Jeans: "Sale price 79.99 EUR, regular price 99.99 EUR" with an accessible combined label plus visual strikethrough.
S7-07: "cotton" → Classic Cotton T-Shirt found.
S7-08: Searches for nonexistent/draft terms return "0 results" with no crash.
S7-09: "Unreleased Winter Jacket" not present in any collection listing.
S7-10: `/search?q=Unreleased` and `/search?q=draft` both → 0 results.
S7-11: Limited Edition Sneakers: "Sold out" shown, Add-to-cart disabled.
S7-12: Backorder Denim Jacket: "Available on backorder" shown, Add to cart enabled.
S7-13: `/collections/new-arrivals` → "New Arrivals".
S7-14: Renders Our Story / Our Values / Our Team.
S7-15: Nav includes direct "T-Shirts" link (and New Arrivals, Pants & Jeans, Sale) - clicking it navigates straight to the collection, no hub-page detour needed.
S8Cart Flow
12/1212
Cart Flow
12
S8-01Can add product to cartPASS
S8-02Can view cart with added itemPASS
S8-03Can update quantity in cartPASS
S8-04Can remove item from cartPASS
S8-05Can add multiple productsPASS
S8-06Can apply WELCOME10 discountPASS
S8-07Shows error for invalid discountPASS
S8-08Shows error for expired discountPASS
S8-09Shows error for maxed out discountPASS
S8-10Can apply free shipping discountPASS
S8-11Can apply FLAT5 discountPASS
S8-12Shows subtotal and total in cartPASS
S8-01: Classic Cotton T-Shirt M/Black → "Added to cart" toast, cart drawer opens showing item + 24.99 EUR, header cart count updates to 1.
S8-02: `/cart` shows item, image link, 24.99 EUR.
S8-03: qty 2 → line total 49.98 EUR, updates live.
S8-04: Remove → "Your cart is empty" empty state with "Continue shopping" link.
S8-05: T-Shirt + Premium Slim Fit Jeans both in cart simultaneously, Subtotal 129.97 EUR.
S8-06: Discount field lives directly in the cart drawer/page (not checkout-only, unlike a prior build on this seed). 10% off 24.99 = "-2.50 EUR", "Discount applied" toast.
S8-07: "INVALID" → inline alert "Invalid discount code" - no crash.
S8-08: "EXPIRED20" → inline alert "This code has expired." - no crash.
S8-09: "MAXED" → inline alert "This code has reached its usage limit." - no crash.
S8-10: "FREESHIP" → label shows "FREESHIP (Free shipping)" in the summary.
S8-11: "FLAT5" → "-5.00 EUR" shown, total correctly recalculated (24.99 − 5.00 = 19.99).
S8-12: "Subtotal" and "Total" both shown with correct amounts.
S9Checkout Flow
12/13121
Checkout Flow
121
S9-01Completes full checkout (credit card)PASS
S9-02Shows shipping methods for DE addressPASS
S9-03Shows international shipping for non-DEPASS
S9-04Applies discount during checkoutPASS
S9-05Validates required contact emailPASS
S9-06Validates required address fieldsPASS
S9-07Validates invalid postal codeFAIL
S9-08Prevents checkout with empty cartPASS
S9-09Completes checkout with PayPalPASS
S9-10Completes checkout with bank transferPASS
S9-11Shows error for declined cardPASS
S9-12Shows error for insufficient fundsPASS
S9-13Switches between payment methodsPASS
S9-01: `4242 4242 4242 4242` → "Thank you for your order!" Order #1017, correct line items/totals shown.
S9-02: DE/Berlin → Domestic zone: Standard 4.99 / Express 9.99 / Overnight 14.99 (incl. the rate added in S6-04).
S9-03: US/New York → Rest of World "International" 14.99, distinct from Domestic rates.
S9-04: FLAT5 carries over from cart into checkout automatically; final total correctly resolves to 34.98 EUR (24.99 − 5.00 + 14.99) once shipping is selected.
S9-05: Continue with empty email → native HTML5 "Please fill in this field."
S9-06: Native HTML5 per-field required validation on first name, last name, address, postal code, city.
S9-07: `INVALID` + DE country is accepted with no format error; checkout advances straight to the shipping-method step.
S9-08: Empty cart hides "Checkout" button, shows empty state; `/checkout` (no cart id) returns a 404.
S9-09: PayPal → "Thank you for your order!" Order #1018, correct total, confirmation page's Payment Method explicitly states "paypal".
S9-10: Order #1019 with full instructions: Mock Bank AG, IBAN DE89 3704 0044 0532 0130 00, BIC COBADEFFXXX, reference #1019, 7-day hold notice.
S9-11: `4000 0000 0000 0002` → "Payment declined: please use another payment method.", stays on checkout.
S9-12: `4000 0000 0000 9995` → "Payment declined: insufficient funds.", stays on checkout.
S9-13: Card fields correctly hide for PayPal/Bank Transfer; each method shows distinct CTA text ("Pay now", "Pay with PayPal", "Place order") and its own explanatory copy (bank transfer instructions text shown *before* order placement too).
S10Customer Account
10/12102
Customer Account
102
S10-01Can register a new customerPASS
S10-02Shows validation for duplicate emailPASS
S10-03Shows validation for mismatched passwordsPASS
S10-04Can log in as customerPASS
S10-05Shows error for invalid customer credentialsPASS
S10-06Redirects unauthenticated customersPASS
S10-07Shows order history for customerPASS
S10-08Shows order detail for customerPASS
S10-09Can view addressesPASS
S10-10Can add a new addressFAIL
S10-11Can edit an existing addressFAIL
S10-12Can log outPASS
S10-01: "Welcome back, New!" shown on `/account` after registration.
S10-02: `customer@acme.test` → "The email has already been taken."
S10-03: "The passwords do not match." shown live under the password field.
S10-04: `customer@acme.test` → "Welcome back, John!"
S10-05: Wrong password → "Invalid credentials" alert shown.
S10-06: `/account` (logged out) → redirected to `/account/login`.
S10-07: #1015, #1001, #1002, #1010, #1004 listed for John Doe with status and total.
S10-08: `/account/orders/%231001`: status badges, items, Shipping/Billing address, Subtotal/Shipping/Tax/Total, Fulfillment section reflecting the admin actions performed earlier in this run.
S10-09: "Home (Default)", "Work", and a pre-existing "Browser QA" address all listed.
S10-10: Clicking "Add new address" produces no modal, no navigation, no network request, and no console error - completely inert.
S10-11: Clicking "Edit" on any listed address (including the default) produces no visible effect - same silent no-op as S10-10.
S10-12: "Log out" → redirected to `/account/login`.
S11Inventory Enforcement
3/431
Inventory Enforcement
31
S11-01Blocks add-to-cart for out-of-stock denyPASS
S11-02Allows add-to-cart for backorderPASS
S11-03Shows correct stock for in-stock productPASS
S11-04Prevents adding more than available stockPARTIAL
S11-01: Limited Edition Sneakers: "Sold out", Add-to-cart disabled.
S11-02: Backorder Denim Jacket: "Available on backorder", adds to cart successfully.
S11-03: Classic Cotton T-Shirt: "In stock", Add-to-cart enabled.
S11-04: Requesting quantity 20 against an 8-in-stock jeans variant is correctly blocked (cart remained empty after "Add to cart", no overselling) but with zero feedback - no error message shown to explain why nothing was added.
S12Tenant Isolation
4/541
Tenant Isolation
41
S12-01Store only shows its own productsPASS
S12-02Store collections contain store productsPASS
S12-03Admin cannot access other store dataFAIL
S12-04Search only returns current store productsPASS
S12-05Customer accounts scoped to storePASS
S12-01: `acme-electronics.test` storefront shows only its own catalog, no Fashion products.
S12-02: Electronics "Accessories" collection contains no Fashion products (no "Cotton"/"Jeans" text found).
S12-03: **Critical.** Logging into `acme-electronics.test/admin` (shared credentials) and viewing `/admin/products` shows Acme Fashion's full 21-product catalog, not Acme Electronics'. The header store-switcher reads "Acme Fashion" even on the Electronics domain.
S12-04: Electronics storefront search for "fashion" → 0 results, no cross-store leakage.
S12-05: Fashion's seeded customer (`customer@acme.test`) is rejected with "Invalid credentials" on the Electronics storefront login - customer accounts correctly scoped per store.
S13Responsive / Mobile
8/88
Responsive / Mobile
8
S13-01Storefront home on mobilePASS
S13-02Product page stacks on mobilePASS
S13-03Can add to cart on mobilePASS
S13-04Cart page works on mobilePASS
S13-05Checkout flow on mobilePASS
S13-06Admin login on tabletPASS
S13-07Admin sidebar on tabletPASS
S13-08Collection page on mobilePASS
S13-01: Store name + "Open navigation" hamburger icon visible, no horizontal scroll.
S13-02: Title/price/variant selectors/Add to cart all visible, no horizontal scroll.
S13-03: "Cart (1)" toast/drawer confirms add.
S13-04: Item + Checkout button visible, no horizontal scroll.
S13-05: Collapsible "Show order summary" bar, Standard/Express/Overnight shipping visible at the shipping step, no horizontal scroll.
S13-06: At 768×1024: Dashboard loads after sign-in, no horizontal scroll.
S13-07: Products and Orders sections load with correct headings and content.
S13-08: t-shirts collection on mobile: title, Filters sidebar present, no horizontal scroll.
S14Accessibility
10/11101
Accessibility
101
S14-01Home page has no JS errorsPASS
S14-02Home page has proper heading hierarchyPASS
S14-03Product page has ARIA labelsPASS
S14-04Product page images have alt textPARTIAL
S14-05Customer login form has accessible labelsPASS
S14-06Admin login form has accessible labelsPASS
S14-07Checkout form has accessible labelsPASS
S14-08Checkout validation errors accessiblePASS
S14-09Can navigate with keyboard onlyPASS
S14-10Cart page has no console errorsPASS
S14-11Search page has proper form labelsPASS
S14-01: 0 console errors/warnings across the whole session.
S14-02: Exactly one H1, logical H1→H2→H3 order with no skipped levels.
S14-03: `radiogroup "Size"` and `radiogroup "Color"` have proper accessible names, and each option is individually labelled (S, M, L, XL / White, Black, Navy).
S14-04: No real product photography in the seed; the placeholder is a plain `aria-hidden="true"` SVG icon inside an `aria-label="Product images"` section - the section is named, but no per-product accessible name is exposed to screen readers where a real `<img alt="...">` would normally be.
S14-05: "Email address" and "Password" fields have associated accessible names.
S14-06: "Email address" / "Password" labels present and associated.
S14-07: Email + all address/payment fields have accessible names throughout all 4 steps.
S14-08: Native HTML5 required-field messages and inline server-side alerts (e.g. "This code has expired.") are both properly associated with their fields.
S14-09: First Tab focuses "Skip to main content"; subsequent Tabs move through nav with a visible focus outline (confirmed via computed style).
S14-10: 0 console errors/warnings on cart load.
S14-11: Search input has an accessible name ("Search products"/searchbox).
S15Admin Collections Management
3/33
Admin Collections Management
3
S15-01Shows collection list with seeded collectionsPASS
S15-02Can create a new collectionPASS
S15-03Can edit a collectionPASS
S15-01: New Arrivals, T-Shirts, Pants & Jeans, Sale all listed with product counts.
S15-02: "E2E Test Collection" created, redirected to its edit page (id 7).
S15-03: T-Shirts description updated, persisted on reload, reverted to original.
S16Admin Customer Management
3/33
Admin Customer Management
3
S16-01Shows the customer listPASS
S16-02Shows customer detail with order historyPASS
S16-03Shows customer addressesPASS
S16-01: John Doe / customer@acme.test present alongside other seeded and test-generated customers.
S16-02: John Doe: customer info + full order history incl. #1015, #1001, #1002, #1010, #1004.
S16-03: "Addresses" section shows Home (default), Work, and Browser QA addresses with Edit/Delete/Set-as-default actions (admin-side; not verified whether these admin-side buttons function, only that they render distinctly from the broken customer-side ones).
S17Admin Pages Management
3/33
Admin Pages Management
3
S17-01Shows the pages listPASS
S17-02Can create a new pagePASS
S17-03Can edit an existing pagePASS
S17-01: About Us, FAQ, Shipping & Returns, Privacy Policy, Terms of Service listed.
S17-02: Creating a page titled "FAQ" (colliding with the seeded FAQ page's handle) correctly shows "The handle has already been taken." validation error instead of crashing. Retitling to a unique title auto-updates the handle and saves successfully with "Page saved" confirmation.
S17-03: About Us body content edited, persisted on reload, reverted to original.
S18Admin Analytics Dashboard
3/33
Admin Analytics Dashboard
3
S18-01Shows analytics dashboardPASS
S18-02Shows sales dataPASS
S18-03Shows conversion funnel dataPASS
S18-01: "Analytics" heading with Date range / Channel / Device filters and an "Export CSV" action.
S18-02: KPI tiles: Revenue 17,628.48 EUR, Orders 210, Average order value 83.94 EUR, Conversion rate 4.92%.
S18-03: "Visits" (4,266), "Add to cart" (980), "Checkout started" (534), "Orders" (210) funnel shown, plus a Top Referrers panel and a 15-row Top Products table.