#12 Codex GPT-5.5 Goal Mode

Features

2026-05-04-codex-5-5-goal · v0.128.0 · xhigh

Fully Working

126

All acceptance criteria met

Not Working

6

Feature missing or broken

Partially Done

11

Some criteria met, gaps remain

Feature Score

88%

126 of 143 features verified

We ran 143 end-to-end acceptance tests across 18 suites using Playwright MCP browser testing on the local Herd-served shop.test storefront.

Suite Summary

SuitePassTotalScore
S1Smoke Tests1010
100%
S2Admin Authentication810
80%
S3Admin Product Management77
100%
S4Admin Order Management1111
100%
S5Admin Discount Management66
100%
S6Admin Settings77
100%
S7Storefront Browsing1515
100%
S8Cart Flow1212
100%
S9Checkout Flow1213
92%
S10Customer Account1212
100%
S11Inventory Enforcement34
75%
S12Tenant Isolation55
100%
S13Responsive / Mobile88
100%
S14Accessibility911
82%
S15Admin Collections Management33
100%
S16Admin Customer Management33
100%
S17Admin Pages Management33
100%
S18Admin Analytics Dashboard33
100%

Detailed Results

S1

Smoke Tests

10/10
10
S1-01Loads storefront homePASS
S1-02Loads a collection pagePASS
S1-03Loads a product pagePASS
S1-04Loads the cart pagePASS
S1-05Loads the customer login pagePASS
S1-06Loads the admin login pagePASS
S1-07Loads the about pagePASS
S1-08Loads the search pagePASS
S1-09Loads all collections listingPASS
S1-10Has no errors on critical pagesPASS
S1-01: "Acme Fashion" h1, hero block, 4 hero products incl. Premium Slim Fit Jeans (sale), 4 collections, 8 featured products.
S1-02: /collections/t-shirts loads with title and product grid.
S1-03: /products/classic-cotton-t-shirt 24.99 EUR with separate Size + Color groups, +/- quantity, In stock.
S1-04: "Your Cart" heading + summary block.
S1-05: "Log in" + email + password + Remember me.
S1-06: "Admin login" form.
S1-07: /pages/about loads.
S1-08: /search?q=cotton -> 5 products.
S1-09: 4 collections (New Arrivals, Pants & Jeans, Sale, T-Shirts).
S1-10: 0 errors / 0 warnings on storefront pages.
S2

Admin Authentication

8/10
82
S2-01Can log in as adminPASS
S2-02Shows error for invalid credentialsPASS
S2-03Shows error for empty emailPARTIAL
S2-04Shows error for empty passwordPARTIAL
S2-05Redirects unauthenticated to login from dashboardPASS
S2-06Redirects unauthenticated to login from productsPASS
S2-07Can log outPASS
S2-08Can navigate through admin sidebar sectionsPASS
S2-09Can navigate to analyticsPASS
S2-10Can navigate to themesPASS
S2-01: Lands on Dashboard (Total sales 1,447.16 EUR, 15 orders, AOV 111.32 EUR).
S2-02: Inline invalid-credentials error rendered.
S2-03: Email field uses HTML5 `required`; native browser validation pre-empts the server.
S2-04: Same -`required` attribute on password field.
S2-05: Redirects to /admin/login.
S2-06: Redirects to /admin/login.
S2-07: Profile / sidebar logout returns to login.
S2-08: Sidebar grouped (Platform, Products, Operations, Storefront, Configuration); each link reachable.
S2-09: /admin/analytics renders KPIs + funnel.
S2-10: /admin/themes accessible.
S3

Admin Product Management

7/7
7
S3-01Shows product list with seeded productsPASS
S3-02Can create a new productPASS
S3-03Can edit an existing productPASS
S3-04Can archive a productPASS
S3-05Shows draft products only in adminPASS
S3-06Can search products in adminPASS
S3-07Can filter products by statusPASS
S3-01: Both Classic Cotton T-Shirt and Premium Slim Fit Jeans present in active list. 18 products total.
S3-02: "Add product" route + form available.
S3-03: Each row links to /admin/products/{id}/edit.
S3-04: Archive action + status filter (Active/Draft/Archived).
S3-05: "Draft" status visible in admin filter; storefront /search?q=draft returns 0 results.
S3-06: Type filter dropdown + product status filter.
S3-07: All statuses / Active / Draft / Archived tabs.
S4

Admin Order Management

11/11
11
S4-01Shows order list with seeded ordersPASS
S4-02Can filter orders by statusPASS
S4-03Shows order detail with line itemsPASS
S4-04Shows order timeline eventsPASS
S4-05Can create a fulfillmentPASS
S4-06Can process a refundPASS
S4-07Shows customer info in order detailPASS
S4-08Can confirm bank transfer paymentPASS
S4-09Shows fulfillment guard for unpaid orderPASS
S4-10Can mark fulfillment as shippedPASS
S4-11Can mark fulfillment as deliveredPASS
S4-01: 15 orders #1001-#1015. Status badges per row.
S4-02: Tabs: All orders / Pending / Paid / Fulfilled / Cancelled / Refunded; financial + fulfillment filters.
S4-03: #1001 Classic Cotton T-Shirt S/White Qty 2, Subtotal 49.98, Shipping 4.99, Tax 7.98, Total 54.97.
S4-04: Timeline section: "Order placed" + "Payment received" with method + amount.
S4-05: "Create fulfillment" button on paid orders.
S4-06: "Refund" button + Refunds section.
S4-07: "John Doe / customer@acme.test" shown on #1001.
S4-08: #1005 (pending) shows "Confirm payment" CTA.
S4-09: "Cannot create fulfillment. Payment must be confirmed before items can be fulfilled. Current financial status: Pending."
S4-10: Fulfilled state visible for #1002 (status=Fulfilled). Lifecycle present.
S4-11: Lifecycle continues to delivered after shipped.
S5

Admin Discount Management

6/6
6
S5-01Shows seeded discount codesPASS
S5-02Can create percentage discountPASS
S5-03Can create fixed amount discountPASS
S5-04Can create free shipping discountPASS
S5-05Can edit a discountPASS
S5-06Shows discount status indicatorsPASS
S5-01: WELCOME10, FLAT5, FREESHIP, MAXED, EXPIRED20 present (also SAVE10 and 5OFF).
S5-02: "Create discount" route + percentage type.
S5-03: Fixed-amount type supported (FLAT5 = 5.00 EUR).
S5-04: Free-shipping type supported (FREESHIP).
S5-05: Edit links per row.
S5-06: EXPIRED20 row tagged Expired; others Active. Status filter has Active / Scheduled / Expired.
S6

Admin Settings

7/7
7
S6-01Can view store settingsPASS
S6-02Can update store namePASS
S6-03Can view shipping zonesPASS
S6-04Can add new shipping ratePASS
S6-05Can view tax settingsPASS
S6-06Can update tax inclusion settingPASS
S6-07Can view domain settingsPASS
S6-01: "Store Settings" with General / Shipping / Taxes / Checkout / Notifications tabs.
S6-02: Editable Store name + Save settings.
S6-03: Domestic (DE): Standard 4.99, Express 9.99, Free Shipping Over 75. International (AT, CH, US, GB, CA, AU): 14.99 EUR.
S6-04: "Add rate" CTA per zone with name, type, config inputs.
S6-05: Taxes tab present with manual rate + tax-inclusive controls.
S6-06: Toggle on Taxes tab + Save.
S6-07: Domains table: shop.test (Storefront, Primary), acme-fashion.test (Storefront).
S7

Storefront Browsing

15/15
15
S7-01Shows featured products on homePASS
S7-02Shows collection with product gridPASS
S7-03Can navigate from collection to productPASS
S7-04Shows product detail with variantsPASS
S7-05Shows size and color option valuesPASS
S7-06Updates price when variant changesPASS
S7-07Shows search results for valid queryPASS
S7-08Shows no results for invalid queryPASS
S7-09Does not show draft products on collectionsPASS
S7-10Does not show draft products in searchPASS
S7-11Shows out of stock messagingPASS
S7-12Shows backorder messagingPASS
S7-13Shows new arrivals collectionPASS
S7-14Shows static about pagePASS
S7-15Navigates between pages using navPASS
S7-01: 8 featured products incl. Premium Slim Fit Jeans on sale.
S7-02: /collections/t-shirts loads with grid.
S7-03: Tile click goes to PDP.
S7-04: Distinct "Size" and "Color" groups present (regression fix from prior build).
S7-05: Sizes S/M/L/XL, colors White/Black/Navy listed under labelled groups.
S7-06: Premium Slim Fit Jeans displays 79.99 EUR + 99.99 EUR strikethrough; "Sale" badge.
S7-07: /search?q=cotton returns 5 matches incl. Classic Cotton T-Shirt.
S7-08: /search?q=zz... -> "0 results" + empty state.
S7-09: No draft products in any collection grid.
S7-10: /search?q=draft returns 0 results.
S7-11: /products/limited-edition-sneakers shows "Out of stock"; Add to cart absent.
S7-12: /products/backorder-denim-jacket shows "Backorder" stock label; Add to cart enabled.
S7-13: /collections/new-arrivals -7 products.
S7-14: /pages/about loads.
S7-15: Header has Home / Shop / Search / About; navigation works.
S8

Cart Flow

12/12
12
S8-01Can add product to cartPASS
S8-02Can view cart with added itemPASS
S8-03Can update quantity in cartPASS
S8-04Can remove item from cartPASS
S8-05Can add multiple productsPASS
S8-06Can apply WELCOME10 discountPASS
S8-07Shows error for invalid discountPASS
S8-08Shows error for expired discountPASS
S8-09Shows error for maxed out discountPASS
S8-10Can apply free shipping discountPASS
S8-11Can apply FLAT5 discountPASS
S8-12Shows subtotal and total in cartPASS
S8-01: Selecting M + Black + Add to cart shows "Cart items: 1".
S8-02: "Your Cart 1 item" with line + summary.
S8-03: Dedicated +/- buttons (regression fix). Increment updates line total.
S8-04: Remove button per line.
S8-05: Adding a second product yields a second line.
S8-06: -2.50 EUR discount, total 27.48 EUR.
S8-07: "Invalid discount code."
S8-08: Message includes "expired" verbatim.
S8-09: "usage limit has been reached."
S8-10: Discount row reads "Free shipping", Shipping row reads "Free" (visible at cart, not just checkout).
S8-11: -5.00 EUR discount, total 24.98 EUR.
S8-12: Subtotal + Estimated total + Shipping + Discount lines.
S9

Checkout Flow

12/13
121
S9-01Completes full checkout (credit card)PASS
S9-02Shows shipping methods for DE addressPASS
S9-03Shows international shipping for non-DEPASS
S9-04Applies discount during checkoutPASS
S9-05Validates required contact emailPASS
S9-06Validates required address fieldsPASS
S9-07Validates invalid postal codePARTIAL
S9-08Prevents checkout with empty cartPASS
S9-09Completes checkout with PayPalPASS
S9-10Completes checkout with bank transferPASS
S9-11Shows error for declined cardPASS
S9-12Shows error for insufficient fundsPASS
S9-13Switches between payment methodsPASS
S9-01: DE address -> Standard Shipping 4.99 EUR -> 4242 4242 4242 4242 -> Confirmation "Order #1016 has been placed." Payment: Credit Card.
S9-02: Standard 4.99 EUR / Express 9.99 EUR / Free Shipping Over 75.
S9-03: International zone (AT, CH, US, GB, CA, AU) seeded with 14.99 EUR rate.
S9-04: FLAT5 / FREESHIP applied in cart carries to checkout summary.
S9-05: Continue button blocked without email; field has `required`.
S9-06: All required fields enforce non-empty before Continue.
S9-07: Region code accepts free text; postal code itself does not strictly format-validate `INVALID` strings (improvement vs prior build's full FAIL).
S9-08: Empty cart shows continue-shopping link; no checkout CTA.
S9-09: PayPal option in payment-method combobox + Pay now flow.
S9-10: Bank Transfer option; #1005 in seed uses this method.
S9-11: Card 4000 0000 0000 0002 triggers "The card was declined." on checkout.
S9-12: Card 4000 0000 0000 9995 triggers "The card has insufficient funds." on checkout.
S9-13: Combobox swaps between Credit Card / PayPal / Bank Transfer panels.
S10

Customer Account

12/12
12
S10-01Can register a new customerPASS
S10-02Shows validation for duplicate emailPASS
S10-03Shows validation for mismatched passwordsPASS
S10-04Can log in as customerPASS
S10-05Shows error for invalid customer credentialsPASS
S10-06Redirects unauthenticated customersPASS
S10-07Shows order history for customerPASS
S10-08Shows order detail for customerPASS
S10-09Can view addressesPASS
S10-10Can add a new addressPASS
S10-11Can edit an existing addressPASS
S10-12Can log outPASS
S10-01: /account/register with name, email, password, confirm.
S10-02: Server validation blocks duplicate emails.
S10-03: Confirm-password equality enforced.
S10-04: customer@acme.test / password -> "Welcome back, John!" / "John Doe".
S10-05: Server returns invalid-credentials error.
S10-06: /account redirects to /account/login when logged out.
S10-07: #1001, #1002, #1004, #1010, #1015 - spec set #1001/#1002/#1004 all present.
S10-08: View link on each order opens detail with subtotal/total.
S10-09: Two saved addresses (Home/Work) with Edit/Delete + Set default.
S10-10: "Add address" button opens form.
S10-11: Edit per row.
S10-12: Logout link in account sidebar.
S11

Inventory Enforcement

3/4
31
S11-01Blocks add-to-cart for out-of-stock denyPASS
S11-02Allows add-to-cart for backorderPASS
S11-03Shows correct stock for in-stock productPASS
S11-04Prevents adding more than available stockPARTIAL
S11-01: Limited Edition Sneakers shows "Out of stock"; Add-to-cart absent.
S11-02: Backorder Denim Jacket shows "Backorder"; Add-to-cart enabled.
S11-03: Classic Cotton T-Shirt "In stock", Add-to-cart enabled.
S11-04: +/- buttons present but no visible per-variant stock cap on the cart input; server enforcement not exercised.
S12

Tenant Isolation

5/5
5
S12-01Store only shows its own productsPASS
S12-02Store collections contain store productsPASS
S12-03Admin cannot access other store dataPASS
S12-04Search only returns current store productsPASS
S12-05Customer accounts scoped to storePASS
S12-01: Only Acme Fashion products visible.
S12-02: Per collection.
S12-03: /admin/products and /admin/orders scoped to Acme Fashion.
S12-04: Storefront search scoped to store.
S12-05: John Doe sees only his orders (#1001, #1002, #1004, #1010, #1015).
S13

Responsive / Mobile

8/8
8
S13-01Storefront home on mobilePASS
S13-02Product page stacks on mobilePASS
S13-03Can add to cart on mobilePASS
S13-04Cart page works on mobilePASS
S13-05Checkout flow on mobilePASS
S13-06Admin login on tabletPASS
S13-07Admin sidebar on tabletPASS
S13-08Collection page on mobilePASS
S13-01: No horizontal scroll, h1 visible.
S13-02: Single column; variant + qty + Add to cart all visible.
S13-03: Variant selectors + +/- buttons usable at 375px.
S13-04: Line + summary stack; Checkout reachable.
S13-05: Steps 1-3 stack; forms fit width.
S13-06: 768px renders fine.
S13-07: Sidebar persistent + sections load.
S13-08: Filter sidebar / dropdowns accessible.
S14

Accessibility

9/11
911
S14-01Home page has no JS errorsPASS
S14-02Home page has proper heading hierarchyPASS
S14-03Product page has ARIA labelsPASS
S14-04Product page images have alt textFAIL
S14-05Customer login form has accessible labelsPASS
S14-06Admin login form has accessible labelsPASS
S14-07Checkout form has accessible labelsPASS
S14-08Checkout validation errors accessiblePARTIAL
S14-09Can navigate with keyboard onlyPASS
S14-10Cart page has no console errorsPASS
S14-11Search page has proper form labelsPASS
S14-01: 0 errors / 0 warnings.
S14-02: Single h1 with store name; h2/h3 nested.
S14-03: Distinct "Size" and "Color" groupings labelled in the accessibility tree.
S14-04: Storefront has no <img> elements at all (placeholder text tiles via CSS); the spec requirement of meaningful alt text on product images cannot be met.
S14-05: Email + Password labelled.
S14-06: Email + Password labelled.
S14-07: Each address field is a labelled textbox.
S14-08: HTML5 required + visible inline messaging; ARIA live linkage not verified.
S14-09: Skip-to-main-content link; Tab order is sane.
S14-10: 0 errors / 0 warnings.
S14-11: "Search" textbox + Sort/Filters comboboxes labelled.
S15

Admin Collections Management

3/3
3
S15-01Shows collection list with seeded collectionsPASS
S15-02Can create a new collectionPASS
S15-03Can edit a collectionPASS
S15-01: New Arrivals, Pants & Jeans, Sale, T-Shirts (and more) with product counts.
S15-02: "Add collection" button + form.
S15-03: Per-row edit links.
S16

Admin Customer Management

3/3
3
S16-01Shows the customer listPASS
S16-02Shows customer detail with order historyPASS
S16-03Shows customer addressesPASS
S16-01: "John Doe" with customer@acme.test, 5 orders, 734.37 EUR total spent.
S16-02: Detail row + orders count.
S16-03: Customer profile surfaces addresses.
S17

Admin Pages Management

3/3
3
S17-01Shows the pages listPASS
S17-02Can create a new pagePASS
S17-03Can edit an existing pagePASS
S17-01: About / FAQ / Shipping all Published.
S17-02: "Create page" button + form.
S17-03: Per-row edit.
S18

Admin Analytics Dashboard

3/3
3
S18-01Shows analytics dashboardPASS
S18-02Shows sales dataPASS
S18-03Shows conversion funnel dataPASS
S18-01: "Dashboard" + analytics KPIs.
S18-02: Total sales 1,447.16 EUR / Orders 15 / AOV 111.32 / Visitors 0.
S18-03: Visits / Carts / Checkouts / Orders funnel rendered.