Skip to content

Why the LLM Doesn’t Make Every Decision

· 1 min read · 0 views

LLM EngineeringAgentic AISoftware ArchitectureResponsible AI

TimeSense uses an LLM, but the LLM is not the entire architecture.

The system first gathers structured context: calendar commitments, tasks, routines, sleep, meals, commute signals, deadlines and feedback.

Deterministic services calculate usable time, apply trust rules, filter candidates and score possible actions.

The LLM is then used where probabilistic reasoning is useful—interpreting unstructured information and generating a natural explanation for the recommendation.

This division matters.

Asking an LLM to control every decision would make the system harder to test, more expensive to operate and less predictable. Removing it entirely would make the product less capable of interpreting real-world context.

The tradeoff is maintaining two reasoning layers instead of one.

The benefit is a system that combines testable rules with flexible language intelligence.

The goal was never to put an LLM everywhere. It was to place it where it improves the system without weakening trust.

Why the LLM doesn’t make every decision: structured context feeds deterministic logic (rules, filters, scoring), and LLM reasoning adds interpretation and explanation. The layered approach is predictable, flexible, lower in cost and latency, and easier to test. The key question: where should deterministic logic end and probabilistic AI begin?

Originally posted on LinkedIn.