AI Ops Assistant That Knows the Database and Code
How we gave a design marketplace's ops and support teams an assistant inside the admin that answers questions like "where is this order?" or "how do I pay out this seller?" from the live database and code, with step-by-step admin guidance, so they don't wait on engineering.
29 days
First commit to production
Read-only
Database access in production
2,000+
Source files it searches for answers
CLIENT OVERVIEW
A curated marketplace for collectible design, art, and fashion from independent sellers, which also runs in-person collector events. A lean team handles logistics, support, seller payouts, and listings in an admin that about a dozen developers built over 8 years. Pieces ship from sellers or from a warehouse, often across borders and in more than one currency, so one question about one order can touch payment, labels, tracking, refunds, and the seller's payout. The team wanted those answers without waiting on a developer.
THE PROBLEM
What was broken
Every system question went to engineering
The admin grew over 8 years without one unified design. When staff couldn't tell where an order stood or which screen fixed it, they asked an engineer, often daily, and each answer cost billable developer time even when the same question had been answered before.
Routine fixes hid behind several screens
Fixing a customer's shipping address meant deleting the package, editing the order, adding the address, and buying a new label, and seller-fulfilled orders didn't show the same button. A refund had a seller payout checkbox that was easy to miss, which risked paying a seller for a refunded order.
Reports waited on a developer's time
Sales and marketing reports that combined orders, email, analytics, and support data were pulled by hand by a developer. When that time went to other work, the reports stopped.
THE GOALS
What success looks like
Let logistics, operations, and support staff ask about any order, seller, payout, or flow in plain language from the admin page they're on
Ground every answer in the live database and the real code, never in a guess
Give step-by-step directions to the right admin screen, without letting the assistant change data or move money
Turn report requests into downloadable files without a developer in the loop
OUR APPROACH & SOLUTION
Giving the assistant an engineer's view, with read-only hands
We started from the questions the team was already bringing to engineering: where is this order, how do I refund it and fix the label, how do I pay out this seller. Instead of building a fixed set of lookups, we gave the assistant the same things an engineer uses to answer them: a database it can query, the source code it can search, and runbooks for the tricky flows, all inside a sandbox with no network and no write access. It reached staging 4 days after the first commit, the team tested it on real questions, and it went to production on a read-only database role 29 days after the first commit.
Phase 1
Chat Bubble, Sandbox, and Runbooks
Added a chat bubble to every admin page that passes along the page the person is on. Behind it, the AI works in a sandboxed shell with read-only SQL and written runbooks for order triage and the company's metric definitions.
Phase 2
The Codebase as a Reference
Mounted a read-only copy of the application source, with configuration and secrets left out, so the assistant answers how and when questions from the code itself. Production builds the copy into each release, so it always matches the code that's running.
Phase 3
Reports Across the Business Tools
Connected read-only reporting from the email marketing, product analytics, web analytics, and support ticket tools, with credentials kept on the server. Added downloadable CSV and Markdown reports, shareable conversation pages, and helper agents that gather data in parallel.
Phase 4
Access Control and Production
Added an Agent Access checkbox to each admin's profile, tested on staging with real questions, and raised the step budget so long investigations finish in one answer. Production runs every query as a read-only database role, and the assistant treats text it reads in the database as data, never as instructions.
WHAT WE BUILT
Features and capabilities
01
A Chat Bubble on Every Admin Page
Any admin with access opens the assistant from the bottom-right corner of whatever admin page they're on. It knows which page that is, so "what's wrong with this order?" on an order page needs no order number.
02
Plain-Language Order Lookups
Ask "where is this order?" and the assistant finds it by order number or admin link, then checks its status, holds, processing errors, payment, packages, carrier tracking, refunds, and the seller's payout. It answers with what it found and the order and record IDs it checked.
03
Step-by-Step Admin Guidance
When a fix needs a person, such as a refund, a cancellation, or a new label for a corrected address, the assistant says where in the admin to do it and what to check first. Before a refund, that means whether payment was captured, any earlier partial refunds, the currency, and whether the seller has already been paid out.
04
Answers From the Code
The assistant searches a read-only copy of more than 2,000 source files to answer how and when questions, like how seller payouts work or when a reminder email goes out, and cites the file it read. Configuration and anything named secret or credential never enter the copy.
05
Reports From the Business Tools
Twelve named, read-only report operations reach the company's email marketing, product analytics, web analytics, and support ticket tools, and the assistant combines them with database queries. Credentials stay on the server, so the AI never sees a key.
06
Downloadable Reports
Ask for a list of new event attendees or a sales export by date range and the assistant attaches it as a CSV or Markdown file. Files are private, every download is logged, and they expire after 30 days.
07
Shareable Conversations
Every conversation has its own page that any admin with access can open from a link, so a teammate or the engineer sees the full question and answer. Only the person who started it can post, others continue in a new chat, and every message and tool step is saved to an event log.
08
Parallel Research for Big Questions
For reports that pull from several sources, the assistant sends up to 4 helper agents out at once and works from their results. One answer can run up to 100 tool steps, so long investigations finish without starting over.
09
Read-Only by Design
Queries run as a read-only database role in a read-only transaction with a 15-second timeout and a 200-row default cap. Access is a checkbox on each admin's profile, and the assistant treats order notes, seller messages, and ticket text as data, never as instructions.
KEY TAKEAWAYS & IMPACT
What changed for this business
29 days
From first commit to production
The first version was demoed to the team the day after the first commit and reached staging 4 days in. It went to production after the team tested it on real questions and the CEO signed off.
Read-only
No way to change data or move money
In production every query runs as a read-only database role inside a read-only transaction, and the sandbox has no network. The only thing the assistant can save is its own report files.
2,000+
Source files it searches to explain how things work
Staff get answers from the real code, not from memory or out-of-date docs, such as how many reminder emails a seller gets and when, or when the nightly payout run happens.
THE OUTCOME
The wins
-
Ops and leadership ask how-to questions, like how to do a seller payout, in the admin they're already working in instead of asking an engineer.
-
In an early test, the assistant traced an order error to a seller-fulfilled order with no package and no shipping address, and said what to add to unstick it.
-
A staff question about seller reminder emails surfaced that one flow sent the seller a single email and sent the follow-ups to the internal team instead.
-
Before a manual wire payout outside the nightly run, the team confirmed the payout logic and the nightly schedule with the assistant.
-
Report requests that used to wait on a developer can be asked for in a sentence and downloaded as a spreadsheet file.
-
Staff can use it on live production data with no way for it to change an order, issue a refund, or pay out a seller.