Skip to content

Building TimeSense: An Agentic AI Assistant in Two Weeks

· 4 min read · 0 views

Agentic AIAI EngineeringSystem ArchitectureLove AI

I took a two-week break from research, got bored, and decided to build a fun agentic AI project before getting back to work.

Lately, I have been inspired by the Kryptonian robots in Man of Steel.

In one scene, Jor-El is being led away when his robot comes to speak with him. Jor-El barely has to communicate what he wants. The robot understands his intent, knows what action to take, can contact Lara Lor-Van, and operates almost like an extension of him.

A robot like that is physically embodied agentic AI.

So I decided to build an agentic AI assistant that I like to imagine could someday become one small part of the brain behind a robot like that.

I called it TimeSense.

TimeSense connects to the tools and signals that shape my day: calendars, health data, location, Slack, tasks, email, routines, and anything else I decide to plug into it.

I can add tasks as I would in any productivity tool. The difference is that managing them should not become a second job.

TimeSense continuously evaluates where I am, the time of day, how much usable time I actually have, how much I slept, my energy level, upcoming meetings, travel time, routines, priorities, and past behavior.

It then recommends the best next action.

It can also suggest calendar changes, help schedule tasks, and prepare communications such as meeting requests or late-arrival messages—with my approval before anything is sent or changed.

Most importantly, it should continually learn how I actually operate.

Building all of this perfectly in two weeks would be impossible. But building a working version was a lot of fun.

I also used an agentic AI development workflow to build it. Agents created Jira tickets and moved work across a Kanban board. Others handled parts of the GitHub workflow, wrote code, reviewed issues, and ran tests.

My role was to coordinate the system, review major changes, make the architectural decisions, and follow the golden rule of AI-assisted engineering:

We are still responsible for the code we put into the world.

Two weeks later, TimeSense is not perfect. It was never going to be. But it works, and I am ready to start sharing it.

Here is the current system architecture. Video demos start next week.

This is not a startup launch or an attempt to sell anything. It is simply me having fun, experimenting with technology, and enjoying the kind of geeky building that got many of us into this field in the first place.

For everyone who still enjoys disappearing into an ambitious side project during a two-week break, I give you:

TimeSense — an agentic AI productivity assistant.

TimeSense system architecture: iOS, Android, and web apps talk to a FastAPI backend; a context ingestion layer normalizes calendar, health, task, Slack, location, voice, and routine signals; a recommendation engine runs deterministic scoring plus an LLM reasoning layer to produce the best next action; results are delivered through screens, widgets, and notifications, all backed by PostgreSQL, Redis, and Celery.

TimeSense system architecture overview in six stages: user interfaces, context and signals, platform and services, recommendation engine, user delivery, and an agentic decision loop, on a data layer of PostgreSQL, Redis, Celery, logs, push notifications, and maps. Mission: recommend the best next action for your real day.

Context and signals: data sources such as calendars, health and sleep, location, tasks, Slack, voice input, email metadata, and routines are normalized into features like availability windows, usable time, energy, travel time, and task urgency, then stored as a learned user profile.

Platform and services: core services (FastAPI API gateway, Firebase Auth, user profiles, integration connectors, file and data service) built on PostgreSQL, Redis, Celery workers, and blob storage, with logging, metrics, tracing, and alerts.

Recommendation engine: candidate generation, constraint and feasibility checks, scoring and ranking, a provider-agnostic LLM reasoning layer, explanation and confidence, then the best next action. Scoring factors include priority, impact, energy match, time available, location, travel time, dependencies, habits, and past behavior.

User delivery: recommendation screen, why-this explanations, proactive notifications, and calendar and message drafts that need approval. A phone mockup shows “Good morning, Ekele” with the best next action “Finish SCOUT++ paper results section” and the reason: 50 minutes of deep work available, high energy, and a close deadline.

Agentic decision loop: observe context, normalize signals, generate candidates, score and rank, select the best action, explain and deliver, then re-evaluate and adapt. Feedback improves every recommendation, and the user stays in control.

LLM providers and security: swappable providers (OpenAI, Anthropic, Google Gemini, Mistral AI) behind an orchestrator, with encryption in transit and at rest, user consent and approvals, data minimization, and SOC 2–ready practices.

Built to learn, built for you: TimeSense connects your tools, understands your context, recommends the best next action, helps you act with your approval, and learns who you are, on iOS, Android, and web.

Originally posted on LinkedIn.