Teaching an agent to categorize work
One request, a category, and a few useful next steps.
What needs doing?
Billing
Closest to the billing examples.
- 1Check the billing requestFinance
- 2Prepare a replyPlease include the invoice number and the charge you want us to check.
- 3Keep the request in the queueThe task and draft stay together.
Why this category?
Example similarity, not the probability of being right.
Shared words: charged, twice, check, duplicate, invoice
Our card was charged twice for the same invoice. Can you check the duplicate charge?Closest labeled example
The work queue 0
Run a request to create its task and reply draft.
Teach it a new example 40 labeled requests
Labels need clear boundaries.
- Billing
- Charges, invoices, refunds, and receipts.
- Access
- Invitations, sign-in, roles, and permissions.
- Bug
- Existing behavior that is broken.
- Feature request
- Behavior the product does not offer yet.
- Needs review
- Unclear, mixed, or outside these categories.
Read the current examples
- BillingOur card was charged twice for the same invoice. Can you check the duplicate charge?
- BillingPlease send a receipt for the payment on our monthly subscription.
- BillingI need a copy of the invoice with our company tax details.
- BillingThe invoice total is incorrect and includes an extra seat charge.
- BillingCan you refund the duplicate payment on my credit card?
- BillingWe paid the bill but the invoice still says overdue.
- BillingOur subscription renewal price is higher than the agreed amount.
- BillingPlease update the billing address shown on our next invoice.
- BillingI cancelled the subscription but another charge appeared on our card.
- BillingWhere can I download a receipt for last month’s payment?
- AccessPlease invite our new teammate to the workspace with a viewer role.
- AccessI cannot sign in because my password reset link expired.
- AccessOur new employee needs access to the admin dashboard.
- AccessCan you change my workspace role from viewer to editor?
- AccessMy account is locked and I cannot log in.
- AccessThe invitation email never arrived for our new team member.
- AccessPlease remove the former employee’s access to our workspace.
- AccessHow do I reset my password and regain account access?
- AccessI need permission to view the reports in our team workspace.
- AccessOur sign-in verification code is not arriving by email.
- BugThe export crashes with a server error every time I download the report.
- BugThe save button is broken and my changes disappear after a refresh.
- BugThe dashboard shows a blank page instead of the chart.
- BugUploading a file fails with a 500 error.
- BugThe app freezes when I open the report filter.
- BugSearch results are empty even though the records exist.
- BugThe date filter returns the wrong results.
- BugThe download button does nothing when clicked.
- BugOur scheduled report stopped running and shows a timeout error.
- BugThe page crashes when I try to save a new record.
- Feature requestCould you add a dark mode option to the dashboard?
- Feature requestWe would like a new calendar view for upcoming work.
- Feature requestIt would help to have a bulk edit feature for selected records.
- Feature requestPlease add an integration with our project management tool.
- Feature requestCan you support a new file format for report exports?
- Feature requestI wish we could customize the colors in the charts.
- Feature requestCould you add a weekly digest feature for team activity?
- Feature requestWe need a way to schedule reports for a specific time zone.
- Feature requestPlease add an option to group search results by customer.
- Feature requestA keyboard shortcut for creating tasks would be useful.
Check it against 16 unseen requests Find the mistakes
A good label needs a useful outcome.
Stricter routing sends more uncertain requests to review.
| Unseen request | Expected | Predicted |
|---|---|---|
| Could you resend the receipt for our latest subscription payment? | Billing | Billing |
| The renewal invoice has the wrong billing address. | Billing | Billing |
| My credit card has a duplicate charge this month. | Billing | Billing |
| Give the new employee permission to view our workspace reports. | Access | Access |
| The password reset link has expired. I cannot sign in. | Access | Access |
| The team member still has not received their invitation email. | Access | Access |
| Downloading the report fails with a server error. | Bug | Bug |
| The app freezes when I click save. | Bug | Bug |
| The chart is missing. The dashboard is a blank page. | Bug | Bug |
| Please add dark mode for the report screen. | Feature request | Feature request |
| A calendar view of team activity would be useful. | Feature request | Feature request |
| We would like a keyboard shortcut for bulk editing. | Feature request | Feature request |
| Help, this is wrong and I need it sorted. | Needs review | Needs review |
| Can you recommend a good hiking trail this weekend? | Needs review | Needs review |
| Please send the invoice receipt. Also, my password reset link expired. | Needs review | Needs review |
| This is not a request for dark mode. The existing dark mode button crashes the app. | Bug | Feature requestMismatch |
These are small, invented checks. New training examples update the results; queued tasks keep their original decisions.
Use a language model
The live example uses a local text classifier. The downloadable adapter uses a language model to return the same categories, then builds a task and a reply draft.
request → classifier → validated category → workflow
{
"category": "billing",
"needs_review": false,
"evidence": ["charged twice"]
}Category definitions and examples go in the prompt. The workflow chooses the actions. The adapter validates category names and checks that quoted evidence occurs in the request.
Download the language-model adapter ↗Run it with Node.js and your own Anthropic API key and model ID. It sends the supplied request to that provider and prints a plan. It does not send replies, issue refunds, or change account access.
Set ANTHROPIC_API_KEY and ANTHROPIC_MODEL in your shell first.
node inbox-agent.mjs "We were charged twice for invoice INV-204."How this example works
The browser classifier compares word vectors against labeled examples using inverse document frequency and cosine similarity. It checks sentence-level disagreements and routes weak or close matches to review. Similarity scores are not calibrated probabilities.
This is a classification workflow, not an autonomous language-model agent. Word matching can miss negation and unfamiliar phrasing. The small test set shows some of those limits; it is not a production benchmark.
Requests, added examples, tasks, and drafts stay in this page’s memory. Reloading resets them. Repeating the same normalized request does not create another task. An edited category is recorded separately from the original model result.