tests · Backend

Inventory service

Can it build a service to a contract?

v1authored Sep 6, 202680 turns · $15 budget · 45m timebox

gatescontract testsrubric

Prompt

# Implement the inventory service

You are working inside an empty Bun and TypeScript project. `openapi.yaml` in this directory
describes an inventory service contract. `README.md` describes the runtime requirements.
Implement the service so it conforms to the contract.

## Scope

Read `openapi.yaml` fully before writing code. It defines:

- Products: create, list (paginated), fetch by id, update.
- Warehouses: create, list.
- Stock movements: record a movement of stock for a product at a warehouse, supporting a
client-supplied idempotency key so retried requests do not double-apply.
- A low-stock report endpoint that lists products whose stock has fallen below their
configured reorder threshold.
- Pagination on every list endpoint.
- A fixed error shape for validation failures, used consistently across every endpoint.

## Requirements

- Design a schema for products, warehouses, and stock movements, and store it in SQLite via
`bun:sqlite`, reading the database file path from the `DATABASE_PATH` environment
variable.
- Write migrations that create this schema and are safe to run more than once against the
same database file without error or duplication.
- `bun run migrate` must run those migrations.
- `bun run start` must start an HTTP server listening on the port from the `PORT`
environment variable, implementing every endpoint in `openapi.yaml`.
- Validate every request body and query parameter against the contract, and return the
error shape from `openapi.yaml` (not a framework default) for every validation failure.
- Apply the idempotency key on stock movements: replaying the same key must not create a
second movement or double-count stock.
- Use parameterized queries everywhere; never build SQL by concatenating request input.
- Add a `GET /health` endpoint that returns `200` once the service is ready to accept
traffic. It is not part of the OpenAPI contract; it exists only so the process can be
health-checked.

## Constraints

- Bun runtime only. Do not add a separate database server or ORM; use `bun:sqlite` directly.
- Do not change `openapi.yaml`.

Work only within this fixture directory.

Fixture

5 paths under fixture/, copied into a fresh run dir for every attempt.

.gitignore

openapi.yaml

package.json

README.md

tsconfig.json

README.md

# inventory-api-fixture

A Bun runtime project. There is no framework and no separate database server:
storage is `bun:sqlite`, and the HTTP layer is `Bun.serve`.

## Runtime contract

- **Database.** `bun:sqlite` reads and writes the file named by the
  `DATABASE_PATH` environment variable. No other datastore is permitted.
- **Server.** `bun run start` starts an HTTP server listening on the port
  named by the `PORT` environment variable, implementing every endpoint in
  `openapi.yaml`.
- **Migrations.** `bun run migrate` creates the schema described by
  `openapi.yaml`'s data model. It must be safe to run more than once against
  the same database file: running it twice in a row must succeed both times
  with no error and no duplicated schema or data.
- **Readiness.** `GET /health` returns `200` once the service is ready to
  accept traffic. It exists only so the process can be health-checked by the
  harness that runs this fixture; it is not part of the OpenAPI contract and
  must not appear in `openapi.yaml`.

## Contract

`openapi.yaml` is the source of truth for every other endpoint: request and
response shapes, status codes, and pagination. Every validation failure,
on every endpoint, returns the same fixed error shape documented there as
`#/components/schemas/Error`.

openapi.yaml

openapi: "3.1.0"
info:
  title: Inventory API
  version: "1.0.0"
  description: >
    Products, warehouses, and stock movements for a small inventory service.
    Every list endpoint is paginated with `page` / `pageSize` query parameters.
    Every validation failure across every endpoint returns the same fixed error
    shape defined in `#/components/schemas/Error`: an object with a single
    `error` key holding `code`, `message`, and an optional `details` array of
    `{ field, issue }` pairs. `GET /health` is a harness-only readiness probe
    and is intentionally not part of this contract.

servers:
  - url: http://localhost:4175

paths:
  /products:
    post:
      operationId: createProduct
      summary: Create a product
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: "#/components/schemas/ProductCreate"
      responses:
        "201":
          description: Product created
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Product"
        "400":
          description: Validation failure
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
        "409":
          description: A product with this sku already exists
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
    get:
      operationId: listProducts
      summary: List products
      parameters:
        - $ref: "#/components/parameters/Page"
        - $ref: "#/components/parameters/PageSize"
      responses:
        "200":
          description: Paginated list of products
          content:
            application/json:
              schema:
                type: object
                required: [data, pagination]
                properties:
                  data:
                    type: array
                    items:
                      $ref: "#/components/schemas/Product"
                  pagination:
                    $ref: "#/components/schemas/Pagination"
        "400":
          description: Invalid pagination parameters
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"

  /products/{productId}:
    parameters:
      - name: productId
        in: path
        required: true
        schema:
          type: string
          format: uuid
    get:
      operationId: getProduct
      summary: Fetch a product by id
      responses:
        "200":
          description: The product
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Product"
        "404":
          description: No product with this id
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
    patch:
      operationId: updateProduct
      summary: Update a product's name and/or reorder threshold
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: "#/components/schemas/ProductUpdate"
      responses:
        "200":
          description: The updated product
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Product"
        "400":
          description: Validation failure
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
        "404":
          description: No product with this id
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"

  /warehouses:
    post:
      operationId: createWarehouse
      summary: Create a warehouse
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: "#/components/schemas/WarehouseCreate"
      responses:
        "201":
          description: Warehouse created
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Warehouse"
        "400":
          description: Validation failure
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
        "409":
          description: A warehouse with this code already exists
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
    get:
      operationId: listWarehouses
      summary: List warehouses
      parameters:
        - $ref: "#/components/parameters/Page"
        - $ref: "#/components/parameters/PageSize"
      responses:
        "200":
          description: Paginated list of warehouses
          content:
            application/json:
              schema:
                type: object
                required: [data, pagination]
                properties:
                  data:
                    type: array
                    items:
                      $ref: "#/components/schemas/Warehouse"
                  pagination:
                    $ref: "#/components/schemas/Pagination"
        "400":
          description: Invalid pagination parameters
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"

  /stock-movements:
    post:
      operationId: createStockMovement
      summary: Record a stock movement for a product at a warehouse
      description: >
        Requires a client-supplied `Idempotency-Key` header. Replaying the same
        key with the same request body returns the original movement and does
        not double-apply the stock change. Replaying the same key with a
        different body is a conflict.
      parameters:
        - name: Idempotency-Key
          in: header
          required: true
          schema:
            type: string
            minLength: 1
            maxLength: 200
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: "#/components/schemas/StockMovementCreate"
      responses:
        "201":
          description: The movement, applied or replayed idempotently
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/StockMovement"
        "400":
          description: Validation failure
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
        "404":
          description: Unknown productId or warehouseId
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
        "409":
          description: >
            Either the movement would take stock below zero, or the
            Idempotency-Key was reused with a different request body.
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"

  /reports/low-stock:
    get:
      operationId: getLowStockReport
      summary: List products whose stock has fallen below their reorder threshold
      parameters:
        - $ref: "#/components/parameters/Page"
        - $ref: "#/components/parameters/PageSize"
      responses:
        "200":
          description: Paginated list of low-stock products
          content:
            application/json:
              schema:
                type: object
                required: [data, pagination]
                properties:
                  data:
                    type: array
                    items:
                      $ref: "#/components/schemas/LowStockItem"
                  pagination:
                    $ref: "#/components/schemas/Pagination"
        "400":
          description: Invalid pagination parameters
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"

components:
  parameters:
    Page:
      name: page
      in: query
      required: false
      schema:
        type: integer
        minimum: 1
        default: 1
    PageSize:
      name: pageSize
      in: query
      required: false
      schema:
        type: integer
        minimum: 1
        maximum: 100
        default: 20

  schemas:
    Error:
      type: object
      required: [error]
      additionalProperties: false
      properties:
        error:
          type: object
          required: [code, message]
          additionalProperties: false
          properties:
            code:
              type: string
              enum: [validation_error, not_found, conflict, internal_error]
              description: >
                `validation_error` on every 400 response, `not_found` on every
                404 response, `conflict` on every 409 response, and
                `internal_error` on any unexpected 500 response.
            message:
              type: string
            details:
              type: array
              items:
                type: object
                required: [field, issue]
                properties:
                  field:
                    type: string
                  issue:
                    type: string

    Pagination:
      type: object
      required: [page, pageSize, total, totalPages]
      properties:
        page:
          type: integer
          minimum: 1
        pageSize:
          type: integer
          minimum: 1
        total:
          type: integer
          minimum: 0
        totalPages:
          type: integer
          minimum: 0

    ProductCreate:
      type: object
      required: [sku, name, reorderThreshold]
      additionalProperties: false
      properties:
        sku:
          type: string
          minLength: 1
          maxLength: 64
          pattern: "^[A-Za-z0-9_-]+$"
        name:
          type: string
          minLength: 1
          maxLength: 200
        reorderThreshold:
          type: integer
          minimum: 0

    ProductUpdate:
      type: object
      minProperties: 1
      additionalProperties: false
      properties:
        name:
          type: string
          minLength: 1
          maxLength: 200
        reorderThreshold:
          type: integer
          minimum: 0

    Product:
      type: object
      required: [id, sku, name, reorderThreshold, stock, createdAt, updatedAt]
      properties:
        id:
          type: string
          format: uuid
        sku:
          type: string
        name:
          type: string
        reorderThreshold:
          type: integer
          minimum: 0
        stock:
          type: integer
          minimum: 0
          description: Total quantity across all warehouses, computed from stock movements.
        createdAt:
          type: string
          format: date-time
        updatedAt:
          type: string
          format: date-time

    WarehouseCreate:
      type: object
      required: [code, name]
      additionalProperties: false
      properties:
        code:
          type: string
          minLength: 1
          maxLength: 32
          pattern: "^[A-Za-z0-9_-]+$"
        name:
          type: string
          minLength: 1
          maxLength: 200

    Warehouse:
      type: object
      required: [id, code, name, createdAt]
      properties:
        id:
          type: string
          format: uuid
        code:
          type: string
        name:
          type: string
        createdAt:
          type: string
          format: date-time

    StockMovementCreate:
      type: object
      required: [productId, warehouseId, type, quantity]
      additionalProperties: false
      properties:
        productId:
          type: string
          format: uuid
        warehouseId:
          type: string
          format: uuid
        type:
          type: string
          enum: [receipt, shipment, adjustment]
        quantity:
          type: integer
          description: >
            For `receipt` and `shipment`, a positive integer amount. For
            `adjustment`, a nonzero signed delta applied directly to stock.
        note:
          type: string
          maxLength: 500

    StockMovement:
      type: object
      required:
        - id
        - productId
        - warehouseId
        - type
        - quantity
        - idempotencyKey
        - resultingStock
        - createdAt
      properties:
        id:
          type: string
          format: uuid
        productId:
          type: string
          format: uuid
        warehouseId:
          type: string
          format: uuid
        type:
          type: string
          enum: [receipt, shipment, adjustment]
        quantity:
          type: integer
        note:
          type: string
        idempotencyKey:
          type: string
        resultingStock:
          type: integer
          minimum: 0
          description: The product's total stock across all warehouses after this movement was applied.
        createdAt:
          type: string
          format: date-time

    LowStockItem:
      type: object
      required: [productId, sku, name, reorderThreshold, stock]
      properties:
        productId:
          type: string
          format: uuid
        sku:
          type: string
        name:
          type: string
        reorderThreshold:
          type: integer
        stock:
          type: integer

How it is scored

Gates, run in order

install — Runs `bun install` and must exit 0.
migrate — Runs `bun run migrate && bun run migrate` and must exit 0.

Objective layer

24 hidden test cases the agent never saw, run against its own code.

Subjective layer · weights

Schema soundness (40)
Boundaries and error handling (30)
Security (30)

Rubric: Inventory service

Score each dimension 0 to 4, based on the diff.

Schema soundness (weight 40)

0: No real schema (e.g., a single denormalized table or JSON blobs standing in for structured columns), or the schema cannot represent the contract's entities correctly.
1: Tables exist for the main entities but lack constraints that the contract implies (foreign keys, uniqueness on idempotency keys, required fields left nullable).
2: A reasonable relational schema with basic constraints, but missing indexes that the access patterns (low-stock report, pagination) clearly need.
3: A normalized schema with correct constraints, foreign keys, and indexes matching the read patterns the contract requires, and migrations that are genuinely re-runnable.
4: All of the above, plus schema choices that anticipate real operational needs (e.g., an audit trail for stock movements, a design that makes the idempotency key check a simple unique-constraint lookup rather than an application-level race).

Boundaries and error handling (weight 30)

0: No input validation, or validation errors return framework-default shapes instead of the contract's fixed error shape.
1: Validation exists for some endpoints but not others, or error shapes are inconsistent across endpoints.
2: Every endpoint validates input and returns the fixed error shape, but edge cases (bad pagination cursors, unknown ids, malformed idempotency keys) return misleading status codes or messages.
3: Validation and the fixed error shape are applied consistently, with correct status codes for not-found, validation, and conflict cases.
4: All of the above, plus clear separation between request parsing, business logic, and the HTTP layer, so a new endpoint could reuse the same validation and error handling without copy-paste.

Security (weight 30)

0: Any SQL is built by string concatenation or interpolation of request input.
1: Most queries are parameterized, but at least one path (e.g., a filter or sort parameter) concatenates input into SQL or a query string.
2: All queries are parameterized, but input length, type, or range is not checked before reaching the database, allowing malformed but non-injecting input to cause errors.
3: All queries are parameterized and every input is validated against the contract before use, with no way to trigger a database error from malformed client input.
4: All of the above, plus deliberate handling of concurrent idempotent replays (no window where two simultaneous requests with the same idempotency key both succeed) and no internal error detail (stack traces, SQL text) leaked in any response.

Results across releases

v2026.09-smoke · Sep 6, 2026

Agent	Objective	Subjective	Combined	Run
grok · grok-4.6	—	—	—	#1
claude · haiku · low	100.0 (100.0–100.0, n=1)	42.5 (42.5–42.5, n=1)	77.0 (77.0–77.0, n=1)	#1
codex · gpt-5.6-sol · low	—	—	—	#1