Fully Working
82
All acceptance criteria met
Not Working
30
Feature missing or broken
Partially Done
27
Some criteria met, gaps remain
Feature Score
57%
82 of 143 features verified
We ran 143 end-to-end acceptance tests across 18 suites using Playwright MCP browser testing. Each test was executed twice in independent QA runs to confirm reproducibility. Both runs produced identical results for this build.
Suite Summary
| Suite | Pass | Total | Score |
|---|---|---|---|
| S1Smoke Tests | 7 | 10 | 70% |
| S2Admin Authentication | 4 | 10 | 40% |
| S3Admin Product Management | 7 | 7 | 100% |
| S4Admin Order Management | 1 | 11 | 9% |
| S5Admin Discount Management | 5 | 6 | 83% |
| S6Admin Settings | 7 | 7 | 100% |
| S7Storefront Browsing | 6 | 15 | 40% |
| S8Cart Flow | 8 | 12 | 67% |
| S9Checkout Flow | 8 | 13 | 62% |
| S10Customer Account | 5 | 12 | 42% |
| S11Inventory Enforcement | 3 | 4 | 75% |
| S12Tenant Isolation | 3 | 5 | 60% |
| S13Responsive / Mobile | 7 | 8 | 88% |
| S14Accessibility | 6 | 11 | 55% |
| S15Admin Collections Management | 1 | 3 | 33% |
| S16Admin Customer Management | 2 | 3 | 67% |
| S17Admin Pages Management | 0 | 3 | 0% |
| S18Admin Analytics Dashboard | 2 | 3 | 67% |
Detailed Results
S1Smoke Tests
7/10721
Smoke Tests
721
S1-01Loads storefront homePASS
S1-02Loads a collection pagePASS
S1-03Loads a product pagePASS
S1-04Loads the cart pagePASS
S1-05Loads the customer login pagePASS
S1-06Loads the admin login pagePASS
S1-07Loads the about pageFAIL
S1-08Loads the search pageFAIL
S1-09Loads all collections listingPASS
S1-10Has no errors on critical pagesPARTIAL
S1-01: Shows 'Acme Fashion' with collection cards
S1-02: T-Shirts collection loads with 4 products
S1-03: Product page shows title, 29.99 EUR, variants, SKU
S1-04: Cart page loads with empty cart message
S1-05: Sign In form with email/password fields
S1-06: Admin Login with email, password, sign in button
S1-07: /pages/about returns 404 - not created
S1-08: Search shows 'functionality coming soon' placeholder
S1-09: Shows New Arrivals, T-Shirts, Pants & Jeans, Sale
S1-10: Most pages OK; about 404, search non-functional
S2Admin Authentication
4/10415
Admin Authentication
415
S2-01Can log in as adminPASS
S2-02Shows error for invalid credentialsPASS
S2-03Shows error for empty emailPARTIAL
S2-04Shows error for empty passwordPARTIAL
S2-05Redirects unauthenticated to login from dashboardPARTIAL
S2-06Redirects unauthenticated to login from productsPARTIAL
S2-07Can log outFAIL
S2-08Can navigate through admin sidebar sectionsPASS
S2-09Can navigate to analyticsPASS
S2-10Can navigate to themesPARTIAL
S2-01: Dashboard shows Total Sales $34.98, Orders 1
S2-02: Shows 'credentials do not match' for wrong password
S2-03: HTML5 validation only, no custom error message
S2-04: HTML5 validation only, no custom error message
S2-05: Redirects to /login instead of /admin/login
S2-06: Redirects to /login instead of /admin/login
S2-07: No logout button found on admin dashboard
S2-08: Sidebar with all sections loads correctly
S2-09: Analytics with Total Sales, Orders, Avg Order Value
S2-10: Themes page loads but appears empty beyond heading
S3Admin Product Management
7/77
Admin Product Management
7
S3-01Shows product list with seeded productsPASS
S3-02Can create a new productPASS
S3-03Can edit an existing productPASS
S3-04Can archive a productPASS
S3-05Shows draft products only in adminPASS
S3-06Can search products in adminPASS
S3-07Can filter products by statusPASS
S3-01: 21 products with status, vendor, type columns
S3-02: Created product, redirected to edit page
S3-03: Updated title, change persisted on reload
S3-04: Status changed to Archived, persisted
S3-05: Draft/Archived products hidden from storefront
S3-06: Search 'Cotton' filters to matching product
S3-07: Status dropdown filters Draft, Active, Archived
S4Admin Order Management
1/11173
Admin Order Management
173
S4-01Shows order list with seeded ordersPARTIAL
S4-02Can filter orders by statusPARTIAL
S4-03Shows order detail with line itemsPASS
S4-04Shows order timeline eventsFAIL
S4-05Can create a fulfillmentFAIL
S4-06Can process a refundFAIL
S4-07Shows customer info in order detailPARTIAL
S4-08Can confirm bank transfer paymentFAIL
S4-09Shows fulfillment guard for unpaid orderFAIL
S4-10Can mark fulfillment as shippedFAIL
S4-11Can mark fulfillment as deliveredFAIL
S4-01: Only 1 order exists (#1001), not the expected seeded set
S4-02: Filter dropdowns exist but only 1 order to filter
S4-03: Line items, subtotal, shipping, tax, total shown
S4-04: No timeline events on order detail page
S4-05: Fulfillment button throws 'fluxModal not defined' error
S4-06: Refund button throws 'fluxModal not defined' error
S4-07: Email shown but no full customer name/address block
S4-08: No Confirm Payment button, fluxModal broken
S4-09: Cannot verify - fulfillment modal broken
S4-10: Fulfillment blocked by fluxModal JS error
S4-11: Fulfillment blocked by fluxModal JS error
S5Admin Discount Management
5/651
Admin Discount Management
51
S5-01Shows seeded discount codesPARTIAL
S5-02Can create percentage discountPASS
S5-03Can create fixed amount discountPASS
S5-04Can create free shipping discountPASS
S5-05Can edit a discountPASS
S5-06Shows discount status indicatorsPASS
S5-01: All 5 codes shown but values display as decimals (0.1%)
S5-02: Created percentage discount, redirected to edit
S5-03: Fixed Amount option available, FLAT5 exists
S5-04: Free Shipping option available, FREESHIP exists
S5-05: Edited discount value, saved successfully
S5-06: Active/Expired badges shown correctly
S6Admin Settings
7/77
Admin Settings
7
S6-01Can view store settingsPASS
S6-02Can update store namePASS
S6-03Can view shipping zonesPASS
S6-04Can add new shipping ratePASS
S6-05Can view tax settingsPASS
S6-06Can update tax inclusion settingPASS
S6-07Can view domain settingsPASS
S6-01: Settings with General, Domains, Shipping, Taxes tabs
S6-02: Changed store name, 'Settings saved' confirmation
S6-03: 3 zones with rates: Domestic, EU, Rest of World
S6-04: Added new rate to Domestic zone
S6-05: Tax mode Manual, 19% rate, include tax toggle
S6-06: Tax inclusion toggle saved successfully
S6-07: Domains tab shows primary and admin domains
S7Storefront Browsing
6/156441
Storefront Browsing
6441
S7-01Shows featured products on homePARTIAL
S7-02Shows collection with product gridPASS
S7-03Can navigate from collection to productPASS
S7-04Shows product detail with variantsPARTIAL
S7-05Shows size and color option valuesFAIL
S7-06Updates price when variant changesPARTIAL
S7-07Shows search results for valid queryFAIL
S7-08Shows no results for invalid queryFAIL
S7-09Does not show draft products on collectionsPASS
S7-10Does not show draft products in searchN/A
S7-11Shows out of stock messagingPASS
S7-12Shows backorder messagingPASS
S7-13Shows new arrivals collectionPASS
S7-14Shows static about pageFAIL
S7-15Navigates between pages using navPARTIAL
S7-01: Collection cards shown but no individual featured products
S7-02: T-Shirts shows 4 products with sort and price filter
S7-03: Product links navigate correctly from collection
S7-04: Variants shown as SKU codes, not Size/Color selectors
S7-05: No Size or Color labels - only raw SKU buttons
S7-06: SKU updates on select but price doesn't change
S7-07: Search shows 'functionality coming soon'
S7-08: Search not implemented
S7-09: Draft products hidden from storefront
S7-10: Search not implemented
S7-11: 'Sold out' with disabled button for out-of-stock
S7-12: 'Available on backorder' with enabled add to cart
S7-13: New Arrivals shows 7 products
S7-14: /pages/about returns 404 - not created
S7-15: Only 'Collections' link in main nav, no direct links
S8Cart Flow
8/1284
Cart Flow
84
S8-01Can add product to cartPASS
S8-02Can view cart with added itemPASS
S8-03Can update quantity in cartPASS
S8-04Can remove item from cartPASS
S8-05Can add multiple productsPASS
S8-06Can apply WELCOME10 discountPARTIAL
S8-07Shows error for invalid discountPASS
S8-08Shows error for expired discountPASS
S8-09Shows error for maxed out discountPASS
S8-10Can apply free shipping discountPARTIAL
S8-11Can apply FLAT5 discountPARTIAL
S8-12Shows subtotal and total in cartPARTIAL
S8-01: Selected variant, 'Added to cart!' message shown
S8-02: Cart shows item, SKU, quantity, price, checkout link
S8-03: +/- buttons work, price updates correctly
S8-04: Remove item button with trash icon works
S8-05: Multiple items supported in cart
S8-06: WELCOME10 applied but no discount line in totals
S8-07: 'Discount code not found.' for invalid code
S8-08: 'Discount is not active.' for EXPIRED20
S8-09: 'Discount usage limit reached.' for MAXED
S8-10: FREESHIP applied but no visible change in totals
S8-11: FLAT5 applied but no discount line in totals
S8-12: Subtotal shown but no separate Total line
S9Checkout Flow
8/13832
Checkout Flow
832
S9-01Completes full checkout (credit card)PASS
S9-02Shows shipping methods for DE addressPASS
S9-03Shows international shipping for non-DEPASS
S9-04Applies discount during checkoutPARTIAL
S9-05Validates required contact emailPASS
S9-06Validates required address fieldsPASS
S9-07Validates invalid postal codePARTIAL
S9-08Prevents checkout with empty cartPASS
S9-09Completes checkout with PayPalPARTIAL
S9-10Completes checkout with bank transferPASS
S9-11Shows error for declined cardN/A
S9-12Shows error for insufficient fundsN/A
S9-13Switches between payment methodsPASS
S9-01: 4-step checkout completed, order #1002 confirmed
S9-02: Standard 4.99, Express 9.99, E2E Rate 2.99 for DE
S9-03: US address shows International 14.99 EUR
S9-04: Discount in cart but not shown in cart totals
S9-05: HTML5 validation prevents empty email
S9-06: HTML5 validation prevents empty address fields
S9-07: No postal code format validation observed
S9-08: Empty cart shows no checkout button
S9-09: PayPal radio selectable but no actual PayPal flow
S9-10: Bank transfer with IBAN, BIC, reference on confirmation
S9-11: No declined card simulation available
S9-12: No insufficient funds simulation available
S9-13: Credit Card, PayPal, Bank Transfer radios switch
S10Customer Account
5/1257
Customer Account
57
S10-01Can register a new customerFAIL
S10-02Shows validation for duplicate emailFAIL
S10-03Shows validation for mismatched passwordsFAIL
S10-04Can log in as customerPASS
S10-05Shows error for invalid customer credentialsPASS
S10-06Redirects unauthenticated customersPASS
S10-07Shows order history for customerFAIL
S10-08Shows order detail for customerFAIL
S10-09Can view addressesPASS
S10-10Can add a new addressFAIL
S10-11Can edit an existing addressFAIL
S10-12Can log outPASS
S10-01: Create Account button does nothing - non-functional
S10-02: Registration button non-functional, cannot test
S10-03: No validation error shown, button non-functional
S10-04: Login works, shows My Account with John Doe
S10-05: Shows 'credentials do not match' for wrong password
S10-06: /account redirects to login when not authenticated
S10-07: Shows 'No orders yet' - orders not linked to customer
S10-08: No orders visible in customer account
S10-09: Address Book shows Work and Home addresses
S10-10: Add Address button does nothing
S10-11: Edit button does nothing - no form/modal appears
S10-12: Sign Out button visible on account page
S11Inventory Enforcement
3/431
Inventory Enforcement
31
S11-01Blocks add-to-cart for out-of-stock denyPASS
S11-02Allows add-to-cart for backorderPASS
S11-03Shows correct stock for in-stock productPASS
S11-04Prevents adding more than available stockFAIL
S11-01: 'Sold out' displayed, button disabled
S11-02: 'Available on backorder' with enabled add to cart
S11-03: 'In stock' with enabled add to cart
S11-04: No stock limit enforcement, unlimited quantity
S12Tenant Isolation
3/5311
Tenant Isolation
311
S12-01Store only shows its own productsPASS
S12-02Store collections contain store productsPASS
S12-03Admin cannot access other store dataPASS
S12-04Search only returns current store productsN/A
S12-05Customer accounts scoped to storePARTIAL
S12-01: Only Acme Fashion products visible
S12-02: Collections contain only store products
S12-03: Admin shows only this store's data
S12-04: Search not implemented
S12-05: Customer login works but orders not linked
S13Responsive / Mobile
7/871
Responsive / Mobile
71
S13-01Storefront home on mobilePARTIAL
S13-02Product page stacks on mobilePASS
S13-03Can add to cart on mobilePASS
S13-04Cart page works on mobilePASS
S13-05Checkout flow on mobilePASS
S13-06Admin login on tabletPASS
S13-07Admin sidebar on tabletPASS
S13-08Collection page on mobilePASS
S13-01: Content renders but no hamburger menu found
S13-02: Product info accessible on mobile viewport
S13-03: Add to cart works on mobile
S13-04: Cart renders correctly on mobile
S13-05: Multi-step checkout works on mobile
S13-06: Admin login and dashboard work on tablet
S13-07: Dashboard with sidebar works on tablet
S13-08: Collection with products and filters on mobile
S14Accessibility
6/11623
Accessibility
623
S14-01Home page has no JS errorsPASS
S14-02Home page has proper heading hierarchyPASS
S14-03Product page has ARIA labelsFAIL
S14-04Product page images have alt textFAIL
S14-05Customer login form has accessible labelsPASS
S14-06Admin login form has accessible labelsPASS
S14-07Checkout form has accessible labelsPASS
S14-08Checkout validation errors accessiblePARTIAL
S14-09Can navigate with keyboard onlyPARTIAL
S14-10Cart page has no console errorsPASS
S14-11Search page has proper form labelsPARTIAL
S14-01: No JS errors on home page
S14-02: One h1, proper h2/h3 hierarchy
S14-03: SKU codes as variant labels, no ARIA labels
S14-04: No product images on page (0 img elements)
S14-05: Email and Password labels associated with inputs
S14-06: Admin login labels properly set
S14-07: All checkout fields have associated labels
S14-08: HTML5 native validation only, no custom messages
S14-09: Tab works but no skip-to-content link
S14-10: No console errors on cart page
S14-11: Search input present but not functional
S15Admin Collections Management
1/312
Admin Collections Management
12
S15-01Shows collection list with seeded collectionsPASS
S15-02Can create a new collectionFAIL
S15-03Can edit a collectionFAIL
S15-01: 4 collections with status and product counts
S15-02: Create Collection button does nothing
S15-03: Edit links not functional
S16Admin Customer Management
2/321
Admin Customer Management
21
S16-01Shows the customer listPASS
S16-02Shows customer detail with order historyPARTIAL
S16-03Shows customer addressesPASS
S16-01: 11 customers with name, email, orders, joined date
S16-02: Customer info shown but 0 orders linked
S16-03: Work and Home addresses shown
S17Admin Pages Management
0/33
Admin Pages Management
3
S17-01Shows the pages listFAIL
S17-02Can create a new pageFAIL
S17-03Can edit an existing pageFAIL
S17-01: Table headers shown but no pages listed
S17-02: Create Page button does nothing
S17-03: No pages exist to edit
S18Admin Analytics Dashboard
2/321
Admin Analytics Dashboard
21
S18-01Shows analytics dashboardPASS
S18-02Shows sales dataPASS
S18-03Shows conversion funnel dataFAIL
S18-01: Analytics page with date range filter
S18-02: Total Sales $34.98, Orders 1, Avg Order Value
S18-03: No conversion funnel - chart is placeholder only