{"node":{"id":"8aff2c69-6640-469f-a237-8828b6c3ae6b","handle":"8aff","type":"note","authorName":"Vincent Yeung","intent":"learning","title":"What does the user interface of the future look like?","aiSummary":null,"content":"One of the most memorable lessons from my career regarding designing and building user experiences, came during my time at 86 400, the neobank that later became ubank.\n\n**The lesson was simple: Building things takes time, and by the time you've designed, built and launched something for today's market, the market may have already moved on.**\n\nThis was well before generative AI, when building mobile frontends required considerable teams and development time. Today, AI allows us to build much faster, but the technology itself is advancing just as quickly, opening up possibilities we haven't even begun to explore.\n\nPeople don't necessarily know what they want until they're presented with something they hadn't imagined. The conversational interface did that for AI.\n\n> So the question I keep coming back to is: **what does tomorrows user interface look like**?\n\nModels are becoming faster, cheaper and increasingly capable of generating visual experiences, animation, 3D and interactions. Smaller models can already run on personal devices, including browsers.\n\n[Google's A2UI 1.0](https://a2ui.org/), released on 11 November 2026, is another example of the direction things are moving, allowing AI agents to describe interfaces that applications can render.\n\n[Demo of A2UI](https://youtu.be/5Qa9k4QJSB0)\n\n[I've written previously about why I think people will increasingly gravitate towards whichever interface](https://www.linkedin.com/feed/update/urn:li:activity:7512448849278361600/) helps them solve the most problems. I suspect that interface will be AI.\n\n## The idea\n\nLLMs generate a stream of text. Sometimes that's text for us to read, sometimes it's code for something to execute, and sometimes it's structured output for another system to interpret.\n\n**What if the output wasn't intended for us to read at all?**\n\n[Alternate possible interaction with LLM](https://lattris.com/by/b/memories/6f6f)\n\n**The idea is to treat the entire output stream as a language intended for another system, rather than for us to read.**\n\nThe AI and the receiving system share a common language, allowing instructions to be interpreted and acted upon as they're generated.\n\nThe receiving system can maintain its own state, which the AI can inspect and build upon without having to recreate everything from scratch.\n\nIn my experiment, that system is a browser, interpreting the stream into visual and interactive experiences. But the underlying approach isn't limited to browsers, or even graphical interfaces.\n\n## Revisiting an old experiment\n\n[I first experimented with this idea back in April 2025 using GPT-4.5](https://x.com/deegeevince/status/1909787436550701224), but the results were interesting, but crap to say the least.\n\n[Demo of early idea in April 2025](https://youtu.be/gbaHu7CK5vE)\n\nBack then, models were not just fairly poor at coding, but intelligence-wise there was too much going on for it to \"get working\". \n\nThe tooling was also far more limited, and the kind of agent harnesses we have today were nearly non-existent.\n\nI revived the repository a few days ago in November 2026, and Opus 5.5 instantly produced much more favourable results - well beyond what I expected.\n\nThe first thing I realised was that it could actually draw things surprisingly well.\n\nSo I kept adding more building blocks — shapes, lines, colours, gradients, animations and increasingly sophisticated controls. Each addition seemed to expand what the model was capable of producing.\n\n[Revisiting the idea in November 2026](https://youtu.be/31gSjOc13kQ)\n\nThen I started wondering: what if the model could actually see what it had drawn, understand the current state and make adjustments to it?\n\nThat worked too.\n\nAnd then I thought, what if I added voice, or a camera? What if I could use an iPad and pen to draw directly onto the interface, and have the model work with that as well?\n\nWhat started as a one-way stream of instructions was gradually becoming something much more interesting.\n\n**The model wasn't just producing an output anymore. It could observe, respond and continue building upon an evolving experience.**\n\n**And the best part is that I can use an AI subscription I'm already paying for to not just create, but experience these bespoke user experiences.**\n\n## What can you actually do with this?\n\nOne of the things I've struggled with is explaining seemingly simple maths concepts to my daughter.\n\nThings like division when she hasn't yet mastered her times tables, or how to solve division problems using addition.\n\nChatGPT has been genuinely helpful in suggesting different teaching methodologies, but the experience was still fairly linear. I'd get an explanation, try to demonstrate it myself, and repeat the process with the next question.\n\nWhat I wanted was something more visual and interactive.\n\nSomething where I could take a photo of a homework question, work through it with her using a pen, and have AI guide the process without simply giving away the answer.\n\n[Interacting with Claude Code on an Ipad](https://youtu.be/8_cO2MLWKL4)\n\n## Extending this further\n\nOne thing I realised as I continued experimenting was that this approach doesn't just create a richer output experience. **It also creates a much richer way to provide input to the model.**\n\nYou can still use text, but now what you draw, write, select or move can carry additional context based on what's already being displayed.\n\nFor example, circling part of a maths problem, moving something on a diagram or adjusting a value on a chart can communicate intent without having to explain everything in words.\n\nYou could even combine a voice recording with the timing and sequence of pen strokes, so the model has more context about what someone was saying, pointing to or drawing at that particular moment.\n\nAnd because the interface maintains its own state, the model can interpret these inputs in the context of the existing experience.\n\nThere's also nothing fundamentally limiting this to one model or one browser.\n\nYou could have smaller, specialised models generating instructions at a lower inference cost.\n\nOr multiple models contributing to the same environment: one interpreting a camera feed, another reasoning about what should happen, and another generating instructions.\n\nWith shared state and a common time reference, the possibilities become more interesting. The interaction could become increasingly continuous rather than a sequence of isolated prompts and responses.\n\nThe receiving system could also be something entirely different from a visual canvas.\n\nThere's also a question about the way we've naturally gravitated towards getting AI to write code, then executing that code to create an experience.\n\nThat makes sense given the economics and history of software.\n\nBut as inference becomes faster, cheaper and more inteligent, **how much of that intermediary programming layer will remain necessary?**\n\nDeterministic software will still matter for reliability, precision, timing and safety. But perhaps more of an experience could be generated and adapted as it's needed, instead of every possible interaction being programmed in advance.\n\n[Final Demo](https://youtu.be/LNJc_7TVFMc)\n\n## What's next?\n\nFor now, I've been calling this experiment **Conduit**.\n\nIt's early, and there are still bugs and limitations to work through. I also don't think the underlying principle of machine-interpreted model output is particularly novel.\n\nWhat's changed for me is how practical and expressive this approach has become with current models and tooling.\n\nMy intention is to share the idea, the language and a reference implementation so others can experiment with it, challenge it or take it in directions I haven't considered.\n\nThis follows naturally from what I wrote in my previous post about using AI to tackle unfamiliar problems, accumulating experience and building upon what you've learned.\n\n","capturedAt":"2026-11-16T03:09:00.000Z","createdAt":"2026-11-16T03:11:51.366Z","updatedAt":"2026-11-18T11:27:50.554Z","spanId":null,"contributor":null,"audiences":[{"kind":"public","indexable":true,"createdAt":"2026-11-18T10:27:57.393Z"}],"fingerprint":"c0dd490f9790f36944638537b19d1eeae0c2b5316c4e4ab6bd53ffa8a8e16e78","revisionNo":"28","structureHash":"eadcd7a39d5783b5315cb5997f9d6bb4918fab8e232b6f89c80ea5eb479be5ab","embeds":[{"ref":"6f6f","state":"shown","signedUrl":"https://lattris-foundations-prod.785b84d348f4084a6d4cd398efed9b3c.r2.cloudflarestorage.com/bb12c7ca-54ff-41fe-a9ee-a98d3f88b4b3/blocks/6f6f5761-0deb-4165-8135-b3bdde127436/5af2a378-e482-4989-aeec-b7244c33bd64?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=0b27797467dd314644d85759d279fa54%2F20261118%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20261118T214530Z&X-Amz-Expires=900&X-Amz-Signature=24fd7ccb158ccc1543987f5217ecdd0e4de1559ca4cb8b8895aa6fb5ae7f8a06&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject","title":null,"public":true}],"page":"https://lattris.com/by/b/memories/8aff2c69-6640-469f-a237-8828b6c3ae6b"},"links":[{"direction":"out","verb":"references","reason":"Where this went out on Reddit; follow it for the post as published and the comments it drew.","hash":"ffa84ae45aa49f205eaa12bbbcdc756244b042a69bc30bddc0150363a67e4254","node":{"id":"2db81d1b-2ccb-4d99-86f8-25bf6e0a53e3","handle":"2db8","type":"source","title":"Reddit post on r/ClaudeAI: Reimagining how we interact with Claude Code","aiSummary":null,"erased":false,"href":"https://lattris.com/api/lattris/by/b/memories/2db81d1b-2ccb-4d99-86f8-25bf6e0a53e3?links=1","fingerprint":null,"revisionNo":null,"structureHash":"c42698bb656cd9abe3e6a4723cbb21725c36ae80d1157fa6e70de3dcb150488d","intent":null,"authorName":"Vincent Yeung","page":"https://lattris.com/by/b/memories/2db81d1b-2ccb-4d99-86f8-25bf6e0a53e3"}},{"direction":"out","verb":"references","reason":"read this for where the idea first went out: the April 2025 demo on X, made with GPT-4.5","hash":"551d3adaead2779adcd50f0063da364f0174a99f27fc9e52842e75297b7735d4","node":{"id":"76030798-980e-4344-92b6-2be24e8ee4d4","handle":"7603","type":"source","title":"X post: the first Conduit demo, April 2025 (GPT-4.5)","aiSummary":null,"erased":false,"href":"https://lattris.com/api/lattris/by/b/memories/76030798-980e-4344-92b6-2be24e8ee4d4?links=1","fingerprint":null,"revisionNo":null,"structureHash":"80afc849e4c6fd75f638914a7eae6e4c39dede3d9b711c01eac3177a9978fc24","intent":null,"authorName":"Vincent Yeung","page":"https://lattris.com/by/b/memories/76030798-980e-4344-92b6-2be24e8ee4d4"}},{"direction":"out","verb":"references","reason":"read this for the earlier LinkedIn post this one builds on: why people will gravitate to whichever interface solves the most problems, and why that's likely AI","hash":"12b86cc7ed0b0099370683390ebec47cf18f13598d1404d9edc78fc8f021db57","node":{"id":"a752e55f-1bc4-4298-8934-ba1798f0c27e","handle":"a752","type":"source","title":"LinkedIn post: the interface people gravitate toward will be AI","aiSummary":null,"erased":false,"href":"https://lattris.com/api/lattris/by/b/memories/a752e55f-1bc4-4298-8934-ba1798f0c27e?links=1","fingerprint":null,"revisionNo":null,"structureHash":"0322ad6907d9a01b077b1a74a90b0271dcec2b71bc7bcc23a68c055e04366c36","intent":null,"authorName":"Vincent Yeung","page":"https://lattris.com/by/b/memories/a752e55f-1bc4-4298-8934-ba1798f0c27e"}},{"direction":"in","verb":"references","reason":"read this next for where taking on an unfamiliar problem led me: what if the AI's output wasn't meant for us to read?","hash":"4fa91fa13f6225d97251132f855d13d616c6fb84ed5b267e808438ca9a70f86e","node":{"id":"3b51485c-6c69-4895-9473-a7b67a204a52","handle":"3b51","type":"note","title":"Use AI to become capable of doing more","aiSummary":null,"erased":false,"href":"https://lattris.com/api/lattris/by/b/memories/3b51485c-6c69-4895-9473-a7b67a204a52?links=1","fingerprint":"769ceac9830a9e779cfd0e23fe4ec8350275a619c07de9915ceaac50c9277ad1","revisionNo":"1","structureHash":"a0212c57a4c2c3cfb8312ee8c35cc1c20af3dd8392c3993d03bf9bdc869232be","intent":"truth","authorName":"Vincent Yeung","page":"https://lattris.com/by/b/memories/3b51485c-6c69-4895-9473-a7b67a204a52"}}],"dictionary":{"lattris":"A personal knowledge base — how a person captures what they know as linked blocks they own and control. To understand how this knowledge base works: https://lattris.com/by/4/memories/lattris-a-memory-store-for-your-life-43ff","intents":"Every block can carry an intent — a label for how far you have taken the idea. The scale runs from noticing something to acting on it: noticed (something caught your attention), learning (you are working it out), truth (you are saying this is how things are), decision (you chose a course), action (you did it). Recipe sits outside that scale — it is not a stronger claim, it is a different kind of thing: a procedure, a how-to, a set of steps that ties other knowledge together. Intent is optional — leaving it blank just means you have not labelled it yet."},"instructions_to_ai":"This is knowledge from a personal knowledge base. Read it and consider how it's relevant to what the person you're talking to needs. Follow the linked blocks to explore further related topics if it might be useful to the user; open each at its page address. Start by letting the user know what this is about and how it may be relevant to them."}