Dearly
Mara is a character you talk to out loud. She answers with a cloned voice and an animated face, and she remembers what you've told her. Everything is designed around replying within about a second of you finishing, which is hard with three large models in the loop.
Latency
Three models run for every reply. A language model writes it, a voice model speaks it and a face model animates it. One after the other, that's several seconds, so they overlap. The reply is cut into clauses while the language model is still writing, and each clause goes to the voice model as soon as it's complete. The first clause is deliberately cut short at its first comma, since its length is how long you wait before she makes a sound. Audio streams to the player as it's generated, and the face model works from that audio faster than real time.
Syncing happens on the player's machine, with the audio as the master clock. Anything synced earlier would be undone by network jitter. If the face falls behind, playback pauses until it catches up. I'd take a short pause over lips that don't match.
Conversation design
Out of the box, language models are bad at conversation. A study comparing model chat with real phone calls found model turns running up to six times longer than people's, and the models copied the other speaker's wording more and more as the conversation went on. Prompting for natural speech backfires as well. Told to use fillers, one model started 77% of its replies with "oh".
So the language model doesn't decide what Mara says. A conversation system plans each turn as a few moves (answer, ask something back, bring up something from her day, change the subject, correct a wrong claim) and the model just writes the words. Your lines are classified into dialogue acts drawn from Switchboard, ISO 24617-2 and Gottman's coding of couples' conversations. That's how it tells teasing from contempt, or agreeing from actually validating someone. When a move mentions someone, it carries who they are and whether you've heard of them.
Each turn makes one main point. A longer story starts as a headline and only unfolds if you ask about it. In testing, that cut the average reply from 40 words to 21 without the replies feeling clipped. She can also drop a hint ("Jen texted me something today, though") and leave it to you to ask.
Mara's own life
People are easier to talk to when they've got their own stuff going on. Mara has a job, friends, and days where things happen while you're away, so she usually has something to bring up. It matters for the relationship too. In a sixteen-year study of couples, boredom predicted lower satisfaction years later, and new things shared together kept closeness up. Hearing about her day counts.
She has wants of her own, so a conversation can let her down or cheer her up for reasons that are hers. Only a short core of her bio is always in the prompt; the rest of her backstory is pulled in when it's relevant. Anything she improvises, like a sister or a flat she used to live in, gets saved as fact.
What Mara thinks of you
Mara keeps her own model of you as beliefs, each with a source and a confidence. They can be wrong. A couple of cancelled plans can turn into "they're unreliable", which then changes how she takes the next cancellation, until you show her otherwise or tell her she's got it wrong.
What you know is tracked separately from what she knows. The old version kept mentioning people and events the player had never been told about, and nothing broke the illusion faster. Now anything new gets introduced first. Her curiosity follows the research on information gaps: she asks most about things she knows a little about, not things she knows nothing about.
Promises you make stay pinned until they're kept or they lapse. Unfinished subjects become threads she comes back to later (people return to interrupted topics about two-thirds of the time). If something is bothering her and she keeps it to herself, it fades more slowly and starts to show in how she talks, which is what suppression does to people too.
Emotions and the relationship
Emotions are appraisal-based. Events are judged against what Mara wants, and each feeling keeps its cause, so the prompt can say what she's upset about as well as that she's upset. Feelings fade over hours and moods over days. Recalling an old memory stirs the matching feeling for a while without changing her underlying mood; otherwise her mood would swing with every search result.
Trust and closeness change slowly, and negative events count about five times as much as positive ones, in line with the long-term studies of couples. Repair depends on what went wrong. An apology fixes a broken promise but does nothing for a lie, and deception leaves a dent that never fully goes. Making up reduces a hurt without erasing it, so an old grievance can come back with a new one.
The relationship moves through Knapp's stages. One that's going downhill narrows, with fewer topics and less said, instead of resetting to small talk. How much Mara opens up tracks how much you do, exchange by exchange.
Memory
Everything that happens goes into an event log. Mood, memories and what she knows are all derived from it, so I can change a rule, replay the log and see how she'd turn out. The log is also the save file.
Memories are stored as dated facts in the third person, using names instead of pronouns, with who did what to whom stated outright. Plain vector search can't tell "he hates it" from "she hates it". They're tagged by person, place and topic. Recall weighs recency, importance and relevance, and rare tags count for more than common ones (the fan effect from ACT-R). About a dozen defining memories keep their full detail while the rest get compressed into summaries over time. Facts expire instead of being deleted, so "used to like" can still be found.
Nothing is written to memory before a reply. Lookups are plain search with no model call, and new memories are filed afterwards, while the GPU would otherwise be idle.
Building the prompt
Nobody writes Mara's prompt by hand. Code assembles it from blocks: who she is, how she talks and how that shifts with her mood, the relationship, the situation tonight, the conversation so far, and the plan for this turn. The blocks are ordered by how often they change. vLLM reuses a prefix it has already seen, so a 9,000-word prompt with an unchanged start takes 48 ms to read, against 1.1 s from scratch.
The model never sees numbers or rules. State is written as prose ("She's still a bit stung that you cancelled on Friday") and there are no "don't say X" lists, since naming a phrase makes the model more likely to use it. Example dialogue comes from real recorded conversations between people at home.
A second pass checks each line with yes/no questions the model answers in a single token, and strips stage directions like *laughs*, which survived every prompt change I tried. Overused phrases are suppressed during generation with an antislop sampler, so they never have to be named in the prompt.
Listening
Mara reacts while you're talking. Small nods land on your stressed words, there's a quiet "mm" in your pauses, she keeps eye contact with the odd glance away and blinks at the ends of clauses, and her eyebrows lift just before her own stressed words. All of it is timed from the audio, with no speech recognition. The face is rendered in Godot.
Architecture
Dearly is four programs. The front end plays audio and animation and makes no decisions. The engine is the game, one per session, holding Mara's state and running the chain of model calls for each reply. The router schedules GPU work and never handles any of the data. To use a model, the engine asks the router for a slot, waits however long that takes, and gets back a signed grant for a specific pod and model that expires after a few seconds. The pods run the models and nothing else, one job per GPU at a time. In my tests, running jobs side by side on a card made everything slower overall.
None of it assumes a layout. The same code works with all three models on one GPU or spread over many, and with the engine and router on my laptop or in the cloud next to the GPUs. The test suite runs in three layouts against fake pods that simulate jobs at real speeds, so queueing, cancelling and scaling get tested without renting a GPU, and the same suite then runs against real pods.
Scaling works off spare capacity. A new GPU takes tens of seconds to come up, so if you wait for requests to queue, somebody has already been kept waiting. The router starts pods early and treats a growing queue as an alarm.
Interruptions
If you talk over her, she stops mid-word. You can see it in the video at the seagull. Cancelling was the fiddly part, because a line can be in four places at once: still queued at the router, generating on the pod, buffered in the engine waiting for its face, or already sitting in the player's buffer. A cancel has to clear all of them. The same goes for a player who disconnects. Their jobs get killed straight away so the GPU isn't left talking to nobody.
GPU hosting
For development, or a single player, everything fits on one GPU: Ministral 3 14B on vLLM, Chatterbox with a cloned voice, and NVIDIA's Audio2Face-3D through its own SDK, behind a face server I wrote in C++. That's a 32 GB Blackwell card (an RTX PRO 4500, the same class as a 5090) on RunPod, or a 48 GB L40S on Cerebrium. Getting all three onto one CUDA version took some doing; vLLM's published CUDA 12.9 image turned out to ship a torch built for CUDA 13. The pod is snapshotted with every model loaded, so it comes up from zero in 35 to 47 seconds instead of several minutes. The face model needs 0.14 s of GPU time per second of speech.
The architecture is built for many players, though, and the single GPU is just its smallest layout. In production each model gets its own pods and a session uses whichever ones are free. Slots are granted per clause and held only while there's work, so one voice GPU can serve several conversations in turn, and the language model can be writing one player's next clause while another player's audio is being generated. The router sends each request to the pod with the most free slots, starts new pods before the existing ones fill up, and places them in datacenters close to the players, with the model weights already cached on the hosts. The snapshots are what make that workable, since a pod that takes a minute to arrive can't be started on demand.
Anything specific to a provider (starting machines, networking, storage, images) sits behind one adapter per host. Supporting a new provider means writing an adapter and nothing else.