#11 Codex GPT-5.5

Features

2026-04-25-codex-gpt-5-5 · v0.124.0

Fully Working

108

All acceptance criteria met

Not Working

14

Feature missing or broken

Partially Done

21

Some criteria met, gaps remain

Feature Score

76%

108 of 143 features verified

We ran 143 end-to-end acceptance tests across 18 suites using Playwright MCP browser testing on the local Herd-served shop.test storefront.

Suite Summary

SuitePassTotalScore
S1Smoke Tests1010
100%
S2Admin Authentication810
80%
S3Admin Product Management67
86%
S4Admin Order Management711
64%
S5Admin Discount Management46
67%
S6Admin Settings77
100%
S7Storefront Browsing1015
67%
S8Cart Flow1212
100%
S9Checkout Flow713
54%
S10Customer Account712
58%
S11Inventory Enforcement34
75%
S12Tenant Isolation55
100%
S13Responsive / Mobile78
88%
S14Accessibility811
73%
S15Admin Collections Management13
33%
S16Admin Customer Management33
100%
S17Admin Pages Management13
33%
S18Admin Analytics Dashboard13
33%

Detailed Results

S1

Smoke Tests

10/10
10
S1-01Loads storefront homePASS
S1-02Loads a collection pagePASS
S1-03Loads a product pagePASS
S1-04Loads the cart pagePASS
S1-05Loads the customer login pagePASS
S1-06Loads the admin login pagePASS
S1-07Loads the about pagePASS
S1-08Loads the search pagePASS
S1-09Loads all collections listingPASS
S1-10Has no errors on critical pagesPASS
S1-01: "Acme Fashion" home with featured collections + 8 featured products.
S1-02: /collections/t-shirts shows T-Shirts heading + Classic Cotton T-Shirt.
S1-03: Product page: title, 24.99 EUR with 34.99 strikethrough, Size/Color labels.
S1-04: Cart page with empty state and Order summary.
S1-05: Customer login form with email + password.
S1-06: /admin/login form (heading is "Admin login").
S1-07: /pages/about renders About content.
S1-08: /search?q=shirt returns 2 products.
S1-09: Lists New Arrivals (17), Sale (2), T-Shirts (1).
S1-10: 0 console errors / warnings across home, cart, search, login, etc.
S2

Admin Authentication

8/10
82
S2-01Can log in as adminPASS
S2-02Shows error for invalid credentialsPASS
S2-03Shows error for empty emailPASS
S2-04Shows error for empty passwordPASS
S2-05Redirects unauthenticated to login from dashboardPARTIAL
S2-06Redirects unauthenticated to login from productsPARTIAL
S2-07Can log outPASS
S2-08Can navigate through admin sidebar sectionsPASS
S2-09Can navigate to analyticsPASS
S2-10Can navigate to themesPASS
S2-01: admin@acme.test → /admin Dashboard with KPIs (Total sales 175.42 EUR, 5 orders).
S2-02: "These credentials do not match our records."
S2-03: "The email field is required."
S2-04: "The password field is required."
S2-05: /admin redirects to /login (Laravel default page) instead of /admin/login.
S2-06: /admin/products: same Laravel /login redirect.
S2-07: Log out button returns to /admin/login.
S2-08: Sidebar reachable: Products, Collections, Inventory, Orders, Customers, Discounts, Pages, Navigation, Themes, Analytics, Settings, Apps, Developers.
S2-09: /admin/analytics renders with Analytics heading.
S2-10: /admin/themes renders with empty state "No records yet."
S3

Admin Product Management

6/7
61
S3-01Shows product list with seeded productsPARTIAL
S3-02Can create a new productPASS
S3-03Can edit an existing productPASS
S3-04Can archive a productPASS
S3-05Shows draft products only in adminPASS
S3-06Can search products in adminPASS
S3-07Can filter products by statusPASS
S3-01: Classic Cotton T-Shirt present (page 2/2 in 21-product list); spec expects "Premium Slim Fit Jeans" but seed has "Black Skinny Jeans" instead.
S3-02: Created E2E Test Product CGPT55 at 29.99 EUR; "Product saved successfully."
S3-03: Renamed Classic Cotton T-Shirt; flash success.
S3-04: Set status=Archived, hidden from ?status=active view.
S3-05: "Draft Rain Coat" visible in admin only; storefront 404.
S3-06: ?q=Cotton filters list to Classic Cotton T-Shirt.
S3-07: ?status=draft and ?status=active filters work.
S4

Admin Order Management

7/11
722
S4-01Shows order list with seeded ordersPASS
S4-02Can filter orders by statusPASS
S4-03Shows order detail with line itemsPASS
S4-04Shows order timeline eventsPASS
S4-05Can create a fulfillmentPARTIAL
S4-06Can process a refundPARTIAL
S4-07Shows customer info in order detailPASS
S4-08Can confirm bank transfer paymentFAIL
S4-09Shows fulfillment guard for unpaid orderFAIL
S4-10Can mark fulfillment as shippedPASS
S4-11Can mark fulfillment as deliveredPASS
S4-01: Orders #1001-1006 listed with paid status.
S4-02: ?status=paid filter works.
S4-03: #1001: 1x Classic Cotton T-Shirt 24.99, Subtotal 24.99, Shipping 4.99, Tax 5.70, Total 35.68 EUR.
S4-04: Timeline shows Order created and Payment captured events with timestamps.
S4-05: "Create fulfillment" button works but no tracking company / number inputs.
S4-06: Process refund worked (default 5.00 amount); status → partially_refunded. No reason field.
S4-07: Customer section shows John Doe + customer@acme.test.
S4-08: Order #1005 in seed is already paid; no "Confirm payment" button anywhere.
S4-09: No pending order in seed; cannot verify guard.
S4-10: "Mark shipped" → status=shipped.
S4-11: "Mark delivered" → status=delivered.
S5

Admin Discount Management

4/6
411
S5-01Shows seeded discount codesPASS
S5-02Can create percentage discountPASS
S5-03Can create fixed amount discountPASS
S5-04Can create free shipping discountPASS
S5-05Can edit a discountFAIL
S5-06Shows discount status indicatorsPARTIAL
S5-01: WELCOME10, FLAT5, FREESHIP, MAXED, EXPIRED all listed.
S5-02: Created E2ETEST25-CGPT55 (25%). "Discount saved."
S5-03: Created E2EFLAT10-CGPT55 (Fixed amount, 10.00).
S5-04: Created E2EFREESHIP-CGPT55 (Free shipping).
S5-05: No edit links/route. /admin/discounts/1/edit returns 404.
S5-06: Active label only; EXPIRED code also shown as "active" (no expired badge).
S6

Admin Settings

7/7
7
S6-01Can view store settingsPASS
S6-02Can update store namePASS
S6-03Can view shipping zonesPASS
S6-04Can add new shipping ratePASS
S6-05Can view tax settingsPASS
S6-06Can update tax inclusion settingPASS
S6-07Can view domain settingsPASS
S6-01: Settings page shows Acme Fashion + General/Domains/Shipping/Taxes sections.
S6-02: Renamed → "Acme Fashion Updated" persisted on reload. Reverted.
S6-03: Germany: Standard 4.99, Free 0.00. International: 14.99.
S6-04: Added "Overnight Shipping - 14.99 EUR" to Germany zone.
S6-05: Tax Settings shows "Prices include tax: No" + Toggle button.
S6-06: Toggle flipped Yes; persisted on reload.
S6-07: Domains: shop.test, admin.acme-fashion.test, acme-fashion.test.
S7

Storefront Browsing

10/15
1023
S7-01Shows featured products on homePASS
S7-02Shows collection with product gridPASS
S7-03Can navigate from collection to productPASS
S7-04Shows product detail with variantsPARTIAL
S7-05Shows size and color option valuesFAIL
S7-06Updates price when variant changesPARTIAL
S7-07Shows search results for valid queryPASS
S7-08Shows no results for invalid queryPASS
S7-09Does not show draft products on collectionsPASS
S7-10Does not show draft products in searchPASS
S7-11Shows out of stock messagingPARTIAL
S7-12Shows backorder messagingFAIL
S7-13Shows new arrivals collectionPASS
S7-14Shows static about pagePASS
S7-15Navigates between pages using navPASS
S7-01: 8 featured products + 3 collections on home.
S7-02: T-Shirts collection grid shows Classic Cotton T-Shirt.
S7-03: Click through collection card opens PDP.
S7-04: Size and Color group labels render but selectors are static text ("Medium" / "Black"). No actual variant selector.
S7-05: No S/M/L/XL options or color swatches; only the default variant label is shown.
S7-06: Compare-at price (24.99 / 34.99 strikethrough) shown statically on Cotton T-Shirt; no variant switching.
S7-07: ?q=cotton returns Classic Cotton T-Shirt.
S7-08: "0 results for 'zznonexistentproductzz'" + "No results found."
S7-09: Draft Rain Coat hidden from collections.
S7-10: ?q=draft returns no results.
S7-11: Sold Out Sneakers: "Out of stock. This product cannot be added to cart." Add disabled. Wording is "Out of stock", not "Sold out".
S7-12: No backorder / continue-policy product seeded.
S7-13: /collections/new-arrivals shows "New Arrivals".
S7-14: About page renders.
S7-15: Top nav T-Shirts link navigates correctly.
S8

Cart Flow

12/12
12
S8-01Can add product to cartPASS
S8-02Can view cart with added itemPASS
S8-03Can update quantity in cartPASS
S8-04Can remove item from cartPASS
S8-05Can add multiple productsPASS
S8-06Can apply WELCOME10 discountPASS
S8-07Shows error for invalid discountPASS
S8-08Shows error for expired discountPASS
S8-09Shows error for maxed out discountPASS
S8-10Can apply free shipping discountPASS
S8-11Can apply FLAT5 discountPASS
S8-12Shows subtotal and total in cartPASS
S8-01: "Product added to cart." flash; redirected to /cart.
S8-02: Cart shows Classic Cotton T-Shirt 24.99 + variant "Medium / Black".
S8-03: qty=2 → line total 49.98 EUR.
S8-04: Set qty=0 to remove (no explicit Remove button); cart empty.
S8-05: T-Shirt + Black Skinny Jeans both in cart, subtotal 94.98 EUR.
S8-06: WELCOME10: -9.50 EUR. "Remove WELCOME10" button.
S8-07: "Discount code is invalid."
S8-08: "Discount code has expired." (EXPIRED)
S8-09: "Discount code usage limit has been reached." (MAXED)
S8-10: FREESHIP applied; Remove button shown.
S8-11: FLAT5: -5.00 EUR discount line.
S8-12: Subtotal/Discount/Total visible.
S9

Checkout Flow

7/13
715
S9-01Completes full checkout (credit card)PASS
S9-02Shows shipping methods for DE addressPARTIAL
S9-03Shows international shipping for non-DEPARTIAL
S9-04Applies discount during checkoutPARTIAL
S9-05Validates required contact emailPASS
S9-06Validates required address fieldsPASS
S9-07Validates invalid postal codeFAIL
S9-08Prevents checkout with empty cartPARTIAL
S9-09Completes checkout with PayPalPASS
S9-10Completes checkout with bank transferPASS
S9-11Shows error for declined cardPASS
S9-12Shows error for insufficient fundsPASS
S9-13Switches between payment methodsPARTIAL
S9-01: Single-page checkout (not multi-step). Order #1007 created, paid, credit_card.
S9-02: DE address pre-fills; shipping line shown (4.99 below 75 EUR threshold, 0 above) but no method selector.
S9-03: US order accepted (#1011) but no method selector.
S9-04: Cart-applied discount carries through; no discount field on checkout itself.
S9-05: "The email field is required."
S9-06: Required errors for email, name, address1.
S9-07: Postal "INVALID" with country DE accepted; order #1008 created.
S9-08: "Your cart is empty." message shown but Checkout button still rendered.
S9-09: Order #1009: paid, paypal.
S9-10: Order #1010: pending, bank_transfer; confirmation shows IBAN/BIC/Reference.
S9-11: "The test card was declined." stays on checkout.
S9-12: "The test card has insufficient funds." stays on checkout.
S9-13: Radios switch payment_method but card_number input always visible (no dynamic form switch).
S10

Customer Account

7/12
714
S10-01Can register a new customerPASS
S10-02Shows validation for duplicate emailPASS
S10-03Shows validation for mismatched passwordsPASS
S10-04Can log in as customerPARTIAL
S10-05Shows error for invalid customer credentialsPARTIAL
S10-06Redirects unauthenticated customersPARTIAL
S10-07Shows order history for customerPASS
S10-08Shows order detail for customerPARTIAL
S10-09Can view addressesPASS
S10-10Can add a new addressPASS
S10-11Can edit an existing addressFAIL
S10-12Can log outPASS
S10-01: Created e2e-cgpt55@example.com; auto-logged-in to /account.
S10-02: "The email has already been taken."
S10-03: "The password field confirmation does not match."
S10-04: Login works; account page shows email but not John Doe name.
S10-05: Login does not succeed but no visible error message.
S10-06: /account redirects to /login (Laravel default), not /account/login.
S10-07: /account/orders shows #1001/1002/1004/1005.
S10-08: Order detail shows line items + Total, but no Subtotal label.
S10-09: 3 seeded addresses + 1 added Hamburg.
S10-10: "Address saved." + "New Street 42, 20095 Hamburg, DE" listed.
S10-11: No edit links on address list.
S10-12: POST /account/logout returns to /account/login.
S11

Inventory Enforcement

3/4
31
S11-01Blocks add-to-cart for out-of-stock denyPASS
S11-02Allows add-to-cart for backorderFAIL
S11-03Shows correct stock for in-stock productPASS
S11-04Prevents adding more than available stockPASS
S11-01: Sold Out Sneakers: "Out of stock. ..." Add to cart disabled.
S11-02: No backorder product seeded.
S11-03: Classic Cotton T-Shirt: "In stock. 23 available." Add enabled.
S11-04: qty=100 → "This product does not have enough stock available."
S12

Tenant Isolation

5/5
5
S12-01Store only shows its own productsPASS
S12-02Store collections contain store productsPASS
S12-03Admin cannot access other store dataPASS
S12-04Search only returns current store productsPASS
S12-05Customer accounts scoped to storePASS
S12-01: Single tenant deployment - Acme Fashion data only.
S12-02: T-Shirts collection scoped to store.
S12-03: Admin scoped to Acme Fashion banner; products/orders/customers all store-scoped.
S12-04: Storefront search scoped.
S12-05: customer@acme.test sees only Acme orders #1001/1002/1004/1005.
S13

Responsive / Mobile

7/8
71
S13-01Storefront home on mobilePARTIAL
S13-02Product page stacks on mobilePASS
S13-03Can add to cart on mobilePASS
S13-04Cart page works on mobilePASS
S13-05Checkout flow on mobilePASS
S13-06Admin login on tabletPASS
S13-07Admin sidebar on tabletPASS
S13-08Collection page on mobilePASS
S13-01: 375x812 fits without horizontal scroll, but no hamburger / mobile menu - nav links Collections/T-Shirts/About just hidden.
S13-02: PDP stacks vertically on mobile.
S13-03: Add to cart works on mobile.
S13-04: Cart fits and Checkout button accessible.
S13-05: Single-page checkout fits 375 width.
S13-06: 768x1024: admin login + dashboard fit.
S13-07: Sidebar usable; Orders page loads on tablet.
S13-08: T-Shirts collection renders on mobile.
S14

Accessibility

8/11
83
S14-01Home page has no JS errorsPASS
S14-02Home page has proper heading hierarchyPASS
S14-03Product page has ARIA labelsPASS
S14-04Product page images have alt textPASS
S14-05Customer login form has accessible labelsPARTIAL
S14-06Admin login form has accessible labelsPARTIAL
S14-07Checkout form has accessible labelsPARTIAL
S14-08Checkout validation errors accessiblePASS
S14-09Can navigate with keyboard onlyPASS
S14-10Cart page has no console errorsPASS
S14-11Search page has proper form labelsPASS
S14-01: Home: 0 errors, 0 warnings.
S14-02: Exactly 1 h1 ("Acme Fashion"); h2/h3 in logical order.
S14-03: Size and Color group labels visible; Add to cart labeled.
S14-04: All <img> elements have non-empty alt.
S14-05: Email/Password text labels visible adjacent but no <label for> association, no aria-label, no placeholder. Browsers heuristically pick up text but not a proper a11y pattern.
S14-06: Same pattern as customer login.
S14-07: Same pattern on checkout.
S14-08: Required errors render as visible paragraphs above the form.
S14-09: Standard browser tab order works on links and form fields.
S14-10: Cart: 0 errors, 0 warnings.
S14-11: Search input has "Search products" label.
S15

Admin Collections Management

1/3
12
S15-01Shows collection list with seeded collectionsPASS
S15-02Can create a new collectionFAIL
S15-03Can edit a collectionFAIL
S15-01: Lists New Arrivals, Sale, T-Shirts.
S15-02: No "Create collection" button or form. Read-only listing.
S15-03: No edit links / route.
S16

Admin Customer Management

3/3
3
S16-01Shows the customer listPASS
S16-02Shows customer detail with order historyPASS
S16-03Shows customer addressesPASS
S16-01: John Doe (4 orders), Jane Smith, plus E2E + Chrome test customers.
S16-02: /admin/customers/1 shows John Doe + email + orders #1001/1002/1004/1005.
S16-03: Addresses section lists all customer addresses.
S17

Admin Pages Management

1/3
12
S17-01Shows the pages listPASS
S17-02Can create a new pageFAIL
S17-03Can edit an existing pageFAIL
S17-01: About listed in /admin/pages.
S17-02: No "Create page" button or form.
S17-03: No edit links.
S18

Admin Analytics Dashboard

1/3
12
S18-01Shows analytics dashboardPASS
S18-02Shows sales dataFAIL
S18-03Shows conversion funnel dataFAIL
S18-01: /admin/analytics has "Analytics" heading.
S18-02: /admin/analytics page lists order numbers; no Orders/Revenue KPI labels (those live on /admin Dashboard instead).
S18-03: No Visits / funnel labels on /admin/analytics.