Role
Designer & Full-Stack Developer
Client
Personal project
Duration
Most AI products in a portfolio are a chat window with a different colour scheme. This one is interesting because of what the model is asked to return: not text, but a declarative scene graph — a small JSON world with a background, positioned elements, entrance animations, a mood for the avatar, and the narration to speak. The frontend interprets that generically, so any question can produce a genuinely novel animated result rather than selecting from a fixed set of templates.
Ask it "where is Nepal and what's good around it?" and the avatar shifts mood, a stylised map and mountains fly in, fact cards pop up, and it narrates the answer aloud. Change the tone selector from funny to dramatic and both the wording and the visuals change, because tone is an input to the same generation rather than a wrapper applied afterwards.
The architecture is the point. `server/src/schema.ts` holds the contract, `prompt.ts` constrains the model to it, and `bedrock.ts` calls Claude through AWS Bedrock. On the web side, `SceneRenderer` walks the returned elements and animates them with Framer Motion, `Avatar` maps moods to expressions, and `useSpeech` handles voice input and spoken output through the browser's own Web Speech API — no cloud STT or TTS, deliberately, since the browser is enough to prove the loop.
The design docs are unusually explicit about scope: no accounts, no history, no 3D, no conversation memory. The stated success criterion is one sentence — speak a question, watch a relevant scene build itself, hear an on-tone answer. That discipline is why it works as a demo rather than sprawling into a half-finished product.
Tech Stack
Features
- Scene graph instead of proseThe model returns structured JSON describing a small animated world — background, positioned elements, entrance animations and mood — which the frontend renders generically.
- Tone as a first-class inputFunny, dramatic, professional and kid-friendly change the wording and the visuals together, because tone feeds the same generation rather than post-processing it.
- Voice in, voice outQuestions are spoken and answers are narrated using the browser's own Web Speech API, with a typed fallback when the microphone is denied.
- Schema-constrained generationA shared schema and a prompt built around it keep the model's output renderable, so a malformed response is a handled case rather than a broken screen.
- Mood-driven avatarThe avatar's expression is chosen by the model as part of the scene, so the character reacts to its own answer.