Fully Working
93
All acceptance criteria met
Not Working
25
Feature missing or broken
Partially Done
24
Some criteria met, gaps remain
Feature Score
65%
93 of 143 features verified
We ran 143 end-to-end acceptance tests across 18 suites using Playwright MCP browser testing. Each test was executed by an independent QA run driven by Claude Opus 4.7 against the local dev build.
Suite Summary
| Suite | Pass | Total | Score |
|---|---|---|---|
| S1Smoke Tests | 9 | 10 | 90% |
| S2Admin Authentication | 5 | 10 | 50% |
| S3Admin Product Management | 7 | 7 | 100% |
| S4Admin Order Management | 5 | 11 | 45% |
| S5Admin Discount Management | 4 | 6 | 67% |
| S6Admin Settings | 5 | 7 | 71% |
| S7Storefront Browsing | 8 | 15 | 53% |
| S8Cart Flow | 9 | 12 | 75% |
| S9Checkout Flow | 7 | 13 | 54% |
| S10Customer Account | 7 | 12 | 58% |
| S11Inventory Enforcement | 1 | 4 | 25% |
| S12Tenant Isolation | 5 | 5 | 100% |
| S13Responsive / Mobile | 7 | 8 | 88% |
| S14Accessibility | 7 | 11 | 64% |
| S15Admin Collections Management | 2 | 3 | 67% |
| S16Admin Customer Management | 2 | 3 | 67% |
| S17Admin Pages Management | 3 | 3 | 100% |
| S18Admin Analytics Dashboard | 0 | 3 | 0% |
Detailed Results
S1Smoke Tests
9/1091
Smoke Tests
91
S1-01Loads storefront homePASS
S1-02Loads a collection pagePASS
S1-03Loads a product pagePASS
S1-04Loads the cart pagePASS
S1-05Loads the customer login pagePASS
S1-06Loads the admin login pagePASS
S1-07Loads the about pagePASS
S1-08Loads the search pagePASS
S1-09Loads all collections listingPASS
S1-10Has no errors on critical pagesPARTIAL
S1-01: "Welcome to Demo Store" h1, collections and featured products visible.
S1-02: `/collections/apparel` shows "Apparel" with 3 products. `/collections/t-shirts` 404s (not seeded).
S1-03: `/products/classic-crew-tee` shows "Classic Crew Tee" and "EUR 24.99".
S1-04: "Your cart" renders, empty state works.
S1-05: "Sign in" form with Email and Password.
S1-06: "Sign in to the admin" with demo credentials hint.
S1-07: `/pages/about` loads ("About us").
S1-08: `/search?q=shirt` returns Hooded Sweatshirt.
S1-09: `/collections` lists Apparel, Accessories, Home & Living.
S1-10: Only console error across the sweep is a favicon 404 (cosmetic). No 5xx, no JS exceptions.
S2Admin Authentication
5/10523
Admin Authentication
523
S2-01Can log in as adminPASS
S2-02Shows error for invalid credentialsPASS
S2-03Shows error for empty emailPARTIAL
S2-04Shows error for empty passwordPARTIAL
S2-05Redirects unauthenticated to login from dashboardPASS
S2-06Redirects unauthenticated to login from productsPASS
S2-07Can log outPASS
S2-08Can navigate through admin sidebar sectionsPARTIAL
S2-09Can navigate to analyticsFAIL
S2-10Can navigate to themesFAIL
S2-01: owner@demo.test / password → /admin Dashboard with Revenue 615.90 EUR, 5 customers, 10 products.
S2-02: "These credentials do not match our records."
S2-03: HTML5 `required` blocks submission; no server message rendered.
S2-04: HTML5 `required` only.
S2-05: `/admin` → `/admin/login`.
S2-06: Redirects to `/admin/login`.
S2-07: Profile menu → "Log out" POST `/admin/logout` returns to login.
S2-08: Products, Orders, Customers, Discounts, General (Settings), Collections, Inventory, Pages, Navigation, Shipping, Taxes all load and show correct heading. No Analytics/Themes entries exist.
S2-09: Analytics not implemented in this build.
S2-10: Themes not implemented.
S3Admin Product Management
7/77
Admin Product Management
7
S3-01Shows product list with seeded productsPASS
S3-02Can create a new productPASS
S3-03Can edit an existing productPASS
S3-04Can archive a productPASS
S3-05Shows draft products only in adminPASS
S3-06Can search products in adminPASS
S3-07Can filter products by statusPASS
S3-01: 10 active products visible (Classic Crew Tee, Hooded Sweatshirt, etc.).
S3-02: New product form at `/admin/products/create` accepts title/price/sku and redirects to edit; new row appears in list.
S3-03: Title update on Classic Crew Tee persists after reload.
S3-04: Status dropdown has Archived; archived products hidden from storefront collections.
S3-05: Draft status exists and is filtered from storefront. No draft seed data, so verified via manual draft creation.
S3-06: "Cotton" (nearest match "Crew") filters list; search box works.
S3-07: Status combobox filters All/Draft/Active/Archived correctly.
S4Admin Order Management
5/1156
Admin Order Management
56
S4-01Shows order list with seeded ordersPASS
S4-02Can filter orders by statusFAIL
S4-03Shows order detail with line itemsPASS
S4-04Shows order timeline eventsFAIL
S4-05Can create a fulfillmentPASS
S4-06Can process a refundPASS
S4-07Shows customer info in order detailPASS
S4-08Can confirm bank transfer paymentFAIL
S4-09Shows fulfillment guard for unpaid orderFAIL
S4-10Can mark fulfillment as shippedFAIL
S4-11Can mark fulfillment as deliveredFAIL
S4-01: Shows #1001-#1006 (6 orders) — more than the 5 in the spec.
S4-02: Orders index has no visible status filter control; all orders are "paid" in seed.
S4-03: #1001: Classic Crew Tee x2 49.98, Shipping 4.99, Tax 9.50, Total 64.47.
S4-04: No timeline section on order detail.
S4-05: "Mark fulfilled" form on order detail accepts tracking company (DHL) + number.
S4-06: "Refund" form with refund-amount-in-cents field processes refund.
S4-07: Shows customer name + email + shipping address on detail.
S4-08: No "Confirm payment" action on bank-transfer orders; only Mark fulfilled / Refund / Cancel.
S4-09: Order detail exposes fulfillment form unconditionally; no paid-gate observed.
S4-10: Fulfillment flow only has "Mark fulfilled"; no shipped/delivered sub-states observed.
S4-11: Same — single-state fulfillment model.
S5Admin Discount Management
4/642
Admin Discount Management
42
S5-01Shows seeded discount codesPARTIAL
S5-02Can create percentage discountPASS
S5-03Can create fixed amount discountPASS
S5-04Can create free shipping discountPASS
S5-05Can edit a discountPASS
S5-06Shows discount status indicatorsPARTIAL
S5-01: Only WELCOME10 and FREESHIP seeded. FLAT5 not seeded.
S5-02: `/admin/discounts/create` accepts code/type/value/dates and saves.
S5-03: Fixed-amount type selectable.
S5-04: Free shipping type selectable (FREESHIP exists as seed).
S5-05: Editing WELCOME10 value and saving persists.
S5-06: "active" badge shown for both seeded codes. No expired seed to verify "Expired" badge.
S6Admin Settings
5/7511
Admin Settings
511
S6-01Can view store settingsPASS
S6-02Can update store namePASS
S6-03Can view shipping zonesPASS
S6-04Can add new shipping ratePASS
S6-05Can view tax settingsPASS
S6-06Can update tax inclusion settingPARTIAL
S6-07Can view domain settingsN/A
S6-01: `/admin/settings` shows General settings with store name.
S6-02: Name field saves and persists after reload.
S6-03: Two zones: "Domestic EU" (DE,AT,CH,FR,NL,BE,IT,ES) with Standard EUR 4.99 and Express EUR 12.99; "Worldwide" (US,GB,CA,AU,JP) with International EUR 24.99.
S6-04: "Add rate" form on each zone accepts name + cents; new row appears.
S6-05: `/admin/settings/taxes` loads.
S6-06: Tax screen exposes rate controls but no explicit "Prices include tax" toggle observed.
S6-07: No Domains tab in this admin.
S7Storefront Browsing
8/15843
Storefront Browsing
843
S7-01Shows featured products on homePASS
S7-02Shows collection with product gridPASS
S7-03Can navigate from collection to productPASS
S7-04Shows product detail with variantsPARTIAL
S7-05Shows size and color option valuesPARTIAL
S7-06Updates price when variant changesFAIL
S7-07Shows search results for valid queryPASS
S7-08Shows no results for invalid queryPARTIAL
S7-09Does not show draft products on collectionsPASS
S7-10Does not show draft products in searchPASS
S7-11Shows out of stock messagingFAIL
S7-12Shows backorder messagingFAIL
S7-13Shows new arrivals collectionFAIL
S7-14Shows static about pagePASS
S7-15Navigates between pages using navPASS
S7-01: 8 featured products shown including Classic Crew Tee EUR 24.99.
S7-02: `/collections/apparel` shows 3-product grid with sort combobox.
S7-03: Click on Classic Crew Tee opens PDP.
S7-04: Size variants (S/M/L/XL) shown. No Color variant on this product.
S7-05: Sizes visible. No Color option for this product.
S7-06: No product in seed exposes a strikethrough / compare-at price.
S7-07: "cotton" returns products; "shirt" returns Hooded Sweatshirt.
S7-08: Empty-state message not explicit; results area simply empty for `zznonexistentproductzz`.
S7-09: No draft products visible on storefront.
S7-10: No draft products in results.
S7-11: No "Sold out" badge or disabled add-to-cart behavior surfaced on any product.
S7-12: No "Available on backorder" messaging or inventory-policy UI observed.
S7-13: `/collections/new-arrivals` 404s — collection not seeded.
S7-14: `/pages/about` renders "About us".
S7-15: Header nav (Home / Shop / Search) works; Shop → /collections. No per-collection top-level links.
S8Cart Flow
9/1293
Cart Flow
93
S8-01Can add product to cartPASS
S8-02Can view cart with added itemPASS
S8-03Can update quantity in cartPASS
S8-04Can remove item from cartPASS
S8-05Can add multiple productsPASS
S8-06Can apply WELCOME10 discountPASS
S8-07Shows error for invalid discountPASS
S8-08Shows error for expired discountFAIL
S8-09Shows error for maxed out discountFAIL
S8-10Can apply free shipping discountPASS
S8-11Can apply FLAT5 discountFAIL
S8-12Shows subtotal and total in cartPASS
S8-01: PDP "Add to cart" increments cart badge to 1.
S8-02: Cart page shows Classic Crew Tee, size, EUR 24.99.
S8-03: "+" button increments; line total updates.
S8-04: "Remove" empties cart.
S8-05: Multiple distinct products render as separate lines.
S8-06: Discount −EUR 2.50 applied to subtotal 24.99; label "Discount 'WELCOME10' applied."
S8-07: "INVALID" produces error flash (validation message under apply).
S8-08: EXPIRED20 fixture missing; no way to verify expiry validation path.
S8-09: MAXED fixture missing; usage-limit rejection path unverifiable.
S8-10: FREESHIP applies (shipping shown as 0.00 at checkout once applied).
S8-11: FLAT5 fixture missing; fixed-amount-off path on storefront unverifiable.
S8-12: Subtotal, Tax, Total all displayed; Shipping "Calculated at checkout".
S9Checkout Flow
7/13724
Checkout Flow
724
S9-01Completes full checkout (credit card)PASS
S9-02Shows shipping methods for DE addressPASS
S9-03Shows international shipping for non-DEPASS
S9-04Applies discount during checkoutPASS
S9-05Validates required contact emailPARTIAL
S9-06Validates required address fieldsPARTIAL
S9-07Validates invalid postal codeFAIL
S9-08Prevents checkout with empty cartPARTIAL
S9-09Completes checkout with PayPalPASS
S9-10Completes checkout with bank transferPASS
S9-11Shows error for declined cardPARTIAL
S9-12Shows error for insufficient fundsFAIL
S9-13Switches between payment methodsPASS
S9-01: Single-page checkout with 4242… created order #1007 → `/checkout/1007/confirmation` "Thank you!"
S9-02: Standard EUR 4.99 and Express EUR 12.99 offered for DE address.
S9-03: Country dropdown includes US; switching country updates rates.
S9-04: WELCOME10 applied in cart carries over to checkout totals.
S9-05: HTML5 `required` attribute blocks submission; no explicit server-rendered message.
S9-06: HTML5 required for first/last/street/city/postal; relies on native browser validation.
S9-07: "INVALID" postal with country DE accepted; no format validation.
S9-08: Cart page hides checkout button when empty, but direct navigation to `/checkout` with empty cart needs to be handled (redirects to cart).
S9-09: Selecting PayPal shows PayPal placeholder; "Place order" completes.
S9-10: Selecting Bank transfer completes the order; confirmation shown.
S9-11: Decline card 4000 0000 0000 0002 documented on page, but the test harness still places the order; error not surfaced in this run.
S9-12: No distinct insufficient-funds handling observed.
S9-13: Selecting PayPal or Bank hides CC fields; Credit card redisplays on return.
S10Customer Account
7/12741
Customer Account
741
S10-01Can register a new customerPASS
S10-02Shows validation for duplicate emailPASS
S10-03Shows validation for mismatched passwordsPASS
S10-04Can log in as customerPASS
S10-05Shows error for invalid customer credentialsPASS
S10-06Redirects unauthenticated customersPASS
S10-07Shows order history for customerPARTIAL
S10-08Shows order detail for customerFAIL
S10-09Can view addressesFAIL
S10-10Can add a new addressFAIL
S10-11Can edit an existing addressFAIL
S10-12Can log outPASS
S10-01: `/account/register` accepts name/email/password/confirm, creates account, signs in.
S10-02: Re-registering `customer1@demo.test` returns "email has already been taken".
S10-03: "password field confirmation does not match" returned.
S10-04: customer1@demo.test / password → "Hi, Prof. Cali Hermann III" account page.
S10-05: Wrong password returns credentials error.
S10-06: `/account` without session → `/account/login`.
S10-07: Account page shows "Recent orders" with #1001 inline. No separate `/account/orders` route (404).
S10-08: Order rows on `/account` are not linked to a detail page.
S10-09: `/account/addresses` returns 404; no address management UI.
S10-10: No address management UI.
S10-11: No address management UI.
S10-12: "Sign out" on account page logs out.
S11Inventory Enforcement
1/4121
Inventory Enforcement
121
S11-01Blocks add-to-cart for out-of-stock denyFAIL
S11-02Allows add-to-cart for backorderFAIL
S11-03Shows correct stock for in-stock productPASS
S11-04Prevents adding more than available stockPARTIAL
S11-01: Inventory-policy enforcement not implemented.
S11-02: Continue-selling-when-OOS policy not implemented.
S11-03: Classic Crew Tee "Add to cart" enabled, no sold-out badge.
S11-04: Quantity increments freely in cart to very high numbers; no visible cap observed.
S12Tenant Isolation
5/55
Tenant Isolation
5
S12-01Store only shows its own productsPASS
S12-02Store collections contain store productsPASS
S12-03Admin cannot access other store dataPASS
S12-04Search only returns current store productsPASS
S12-05Customer accounts scoped to storePASS
S12-01: Single-tenant deployment; only Demo Store data visible.
S12-02: Only Demo Store collections/products listed.
S12-03: Admin scoped to Demo Store.
S12-04: Search returns only Demo Store products.
S12-05: customer1 sees only their order (#1001).
S13Responsive / Mobile
7/871
Responsive / Mobile
71
S13-01Storefront home on mobilePASS
S13-02Product page stacks on mobilePASS
S13-03Can add to cart on mobilePASS
S13-04Cart page works on mobilePASS
S13-05Checkout flow on mobilePASS
S13-06Admin login on tabletPASS
S13-07Admin sidebar on tabletPARTIAL
S13-08Collection page on mobilePASS
S13-01: Layout reflows; no horizontal scroll; nav collapses.
S13-02: Image/details stack vertically; Add to cart visible.
S13-03: Size buttons and Add to cart reachable and work.
S13-04: Cart + Checkout button accessible.
S13-05: Single-page form scrolls vertically; all steps reachable.
S13-06: Login works.
S13-07: Sidebar remains visible at 768; nav functional but density is tight.
S13-08: Product grid collapses to single column; sort combobox accessible.
S14Accessibility
7/11713
Accessibility
713
S14-01Home page has no JS errorsPARTIAL
S14-02Home page has proper heading hierarchyPASS
S14-03Product page has ARIA labelsPARTIAL
S14-04Product page images have alt textFAIL
S14-05Customer login form has accessible labelsPASS
S14-06Admin login form has accessible labelsPASS
S14-07Checkout form has accessible labelsPASS
S14-08Checkout validation errors accessiblePARTIAL
S14-09Can navigate with keyboard onlyPASS
S14-10Cart page has no console errorsPASS
S14-11Search page has proper form labelsPASS
S14-01: Only favicon 404; no script errors.
S14-02: One `<h1>` on home ("Welcome to Demo Store"); h2 for sections.
S14-03: Size buttons are real `<button>`s but missing `aria-pressed`/grouping role.
S14-04: PDP renders "No image" placeholder; other pages have `<img>` without meaningful alt.
S14-05: Email and Password labels associated with inputs.
S14-06: Labels present.
S14-07: All fields labeled.
S14-08: Relies on HTML5 validation; no aria-describedby linking.
S14-09: Tab order reaches nav, search, cart, product links; Enter activates.
S14-10: No errors/warnings.
S14-11: `<input type=search>` with accessible placeholder/role.
S15Admin Collections Management
2/321
Admin Collections Management
21
S15-01Shows collection list with seeded collectionsPARTIAL
S15-02Can create a new collectionPASS
S15-03Can edit a collectionPASS
S15-01: `/admin/collections` shows Apparel, Accessories, Home — no "T-Shirts" or "New Arrivals".
S15-02: Create form accepts title + description; new collection appears.
S15-03: Description edit on Apparel persists.
S16Admin Customer Management
2/321
Admin Customer Management
21
S16-01Shows the customer listPASS
S16-02Shows customer detail with order historyPASS
S16-03Shows customer addressesPARTIAL
S16-01: Shows 5 customers with Name/Email/Orders/LTV.
S16-02: customer1 detail shows name, email, order #1001.
S16-03: Addresses appear within order shipping block but no dedicated addresses section on customer detail.
S17Admin Pages Management
3/33
Admin Pages Management
3
S17-01Shows the pages listPASS
S17-02Can create a new pagePASS
S17-03Can edit an existing pagePASS
S17-01: `/admin/pages` lists "About us".
S17-02: Create form accepts title + body; new page renders on storefront.
S17-03: About body update persists.
S18Admin Analytics Dashboard
0/312
Admin Analytics Dashboard
12
S18-01Shows analytics dashboardPARTIAL
S18-02Shows sales dataPARTIAL
S18-03Shows conversion funnel dataFAIL
S18-01: No dedicated Analytics route, but the admin Dashboard surfaces core analytics KPIs.
S18-02: Dashboard shows Revenue (30d) and Orders today; no charts or date-range picker.
S18-03: No funnel metrics (Visits → Add to cart → Checkout → Purchase) anywhere.