I've been experimenting with AI-driven design for the past three months.
Here are my takes so far: four ways I use agents in my design work, the tools behind them and where I still step in. https://x.com/i/article/2116910814548336641
Should we still design by hand?
I've always enjoyed UI exploration, from figuring out the visual direction to polishing the smallest details. I thought it would take a long time for AI to reach the level of craft you get from tweaking something intentionally and thinking about how it fits into the whole system.
But lately I've had a hard time falling asleep. Both from excitement and anxiety.
I've spent the last three months experimenting heavily with agent-driven design. I started by bringing component states from code onto a canvas, iterating there, and asking the agent to implement the changes back in code. Then I got into interactive websites, landing pages, and redesigning a product of mine that had thousands of monthly users but still looked like total AI slop.
It's made me take the question seriously: should we still design by hand?
Here are four examples of how designing with agents has changed the way I work.
1. Get stuff from code onto a canvas
One of my first aha moments was asking an agent to lay out what already existed in the code.
I've been working on a personal project called Contexx, although it's mostly been paused over the last year. Think of it as Obsidian in the cloud, with collaboration and an AI that can work with your documents. I still use it occasionally to explore how interfaces for AI tools could work, and I wasn't happy with the document-editing experience.
I'd been asking agents to explain how things worked in the codebase for a while. This time, I asked Cursor to show me the AI states on a canvas, from the status messages to how suggested edits work.
From there, I could edit by hand or with an agent, then ask Cursor to implement the changes. Something that would have been annoying and time consuming to tweak only through code became much easier to inspect and polish on a canvas.
This is a useful starting point if you're working on a product that lacks design files or documentation. You already have something to work from. Ask the agent to lay out the relevant flow and its states, then check them against the product before you start changing things.
Another example is Kdykdy, a mobile first booking system I'm working on with my friends. We had dozens of transactional emails with no Figma file. I designed a template by hand, asked an agent to lay out the emails next to it and apply the styling, then tweaked them and handed them over to a developer. A task that would previously have been quite annoying took less than an hour.
I still designed the template and refined the result. Having the agent apply that direction across the collection made the repetitive part much easier.
2. Explore more directions
When I showed this workflow to a friend, he said it looked like I had a team of designers and was directing them. That's how it feels, especially during visual exploration. The quality of some of the directions has surprised me as much as the speed.
I've been having a lot of fun exploring interactive landing pages in Pen.dev, including concepts built with Three.js. Being able to explore movement and interaction alongside the visual direction makes these experiments particularly interesting to me.
Another way I like to explore is by thinking in scales. I often think of design in adjectives: how expressive is this screen? What happens if you turn that all the way up? What about all the way down?
You can take the same screen and ask for both extremes while keeping the content and purpose the same. Seeing them next to each other gives you something concrete to react to. You can decide which parts of the expressive version are worth keeping and where it becomes too much.
The same approach works when you zoom in on a component. I've always enjoyed exploring different ways to design a card. Doing that manually takes time, especially when you're trying to make something that works across different content and screen sizes. Now I can ask for ten variations and have several directions to compare.
Those variations still need checking. Which one handles a longer title? What happens on mobile? Does it fit the rest of the interface? Getting the options is much faster. Choosing and refining one still takes attention.
3. Work with your design system
My recent experiments connecting an agent to Figma through MCP have been some of the biggest time savers of the lot. MCP is the connection that lets the agent work with the design tool. In this workflow, I can ask it to use the existing design system when making changes.
That matters when you work with a team. If your colleagues review work in Figma and your shared components live there, you need the result to fit into that process. Moving everything to another tool is a much bigger decision than choosing where to do a personal experiment.
In my experiments, working through Figma's MCP connection has felt slower for open-ended exploration. I've enjoyed Paper's HTML canvas and Pen's interactive pages more for that kind of work. Figma has been particularly useful for everyday tasks within an existing system.
That includes adapting a mobile design to other screen sizes, iterating on an existing screen, or creating several versions of a page using the same components. There's plenty of design work where the visual language is already established and the job is to apply it well.
I also tried creating a design-system skill: a reusable set of instructions with design guidelines and references to the components. In my experiments, the agent already uses the components, tokens and spacing, and even names layers properly. The skill gives it more guidance on how those pieces should work together.
With a mature design system, I'd say the agent gets me roughly 80% of the way to the result I want. Adding the skill pushes that closer to 90%. Those are my own rough estimates, but the difference has been noticeable in everyday UI work.
If you're trying this, start with the guidance you find yourself repeating: which component and variant to use, how to handle spacing, and what to call the layers. Include references to good examples from your system. You still need to review the result, especially when the work calls for something distinctive, but you don't have to explain the same conventions every time.
4. Sweat the details in a working component
There's another part of this that gets me excited: design engineering. Getting into how a component behaves, how it responds to your input, and all the tiny details you can spend an unreasonable amount of time on.
For someone like me, who doesn't write code, AI makes it possible to build and refine much more of that myself. I can get a component running, try it, describe what feels off, and keep adjusting it.
Think about a menu opening. The final layout is only part of it. How quickly does it respond? Does the movement feel too bouncy? What happens if you change your mind halfway through and close it again? These are things you want to feel while using it.
I made a model-switcher interaction study because I got tired of opening the same dropdown. It's a tiny part of an interface, but one you might use constantly. Having a working version gives you something to judge: can you switch quickly, is the active model obvious, and does the interaction still feel good after you've used it twenty times?
You can take the same approach with one component from your own product. Get its basic behaviour working, then focus on one detail at a time: the opening motion, the feedback after a click, or the transition between loading and finished. Try each change in context. A nice animation still needs to let you get on with what you're doing.
This is a big part of designing with AI for me. Some of the time it saves can go straight back into craft. I can explore details I would previously have struggled to build myself, and keep designing while the component is running.
So, should we still design by hand?
It depends. I'd say for about 80% of the UI work I do, there's little reason to design everything by hand anymore. When the patterns are familiar and the design system is there, I'm happy to let an agent do much of that work and refine the result.
The other 20% is where I still want to spend time sweating the details. Sometimes you need to come up with something special, and getting there takes exploration, judgment, and a lot of tweaking. AI can help with all of that too. I can compare more directions, make changes by hand when it's easier, and try an interaction that I wouldn't have been able to build myself.
I want to use the time AI saves me to be more ambitious with that 20%. To try the idea I'd normally drop because I couldn't build it, and keep tweaking it until it feels right.
That being said, all of this is about making interfaces, which is still only a small part of design work. I think we'll still get to keep our job for a while.