FIGR AI • AI-NATIVE CANVAS
Turning a fixed gallery into shared working memory for people and AI increasing attachment adoption from 51% to 73%
My role
Product Designer
Years
May 2026
Platform
Web Application
Scope
Team
4 Engineers & CTO
TL;DR
The Canvas became shared working memory for teams and AI
Figr’s post-launch Canvas displayed AI outputs but gave people little direct control. Slack requests, Mixpanel recordings and query-box workarounds showed the same mismatch: people saw a canvas and expected to work spatially. I built a functional prototype connected to our live agent to test direct manipulation and canvas-state changes before engineering committed to the larger interaction model.
43%
increase in attachment adoption
4.6×
queries from visual-context users
3.8×
upgrade rate among visual-context users
Why now
We improved the output before investing in a much larger interaction model
We had deliberately prioritised the quality of Figr’s AI output before expanding Canvas interactions. There was little value in making weak output more editable.
After public launch, the constraint shifted. Teams were bringing existing screens and generated work into the Canvas, then trying to move, resize, organise and reference it. The output had become useful enough to work with; the interaction model was now getting in the way.
problem discovery
Users wanted to compare and organise. The product only accumulated.
The interface borrowed the visual language of a whiteboard without the direct manipulation people expected. Three signals kept pointing to the same gap:
Slack requests: teams asked to organise screens, connect flows, add headings and mark exact regions.
Mixpanel recordings: People repeatedly tried to drag and resize artifacts even though those interactions did not exist.
Query-box workarounds: Users asked the agent to delete, move or resize objects because conversation was their only way to control the Canvas.
This was more than a missing-feature list. People saw a canvas and expected to think spatially, but the product made them translate basic visual actions back into words.
Decision 1
Validate the system before detailing every screen: A Figma prototype could show the layout. It could not test whether direct edits would become useful agent context.
The product hypothesis crossed two systems. People needed familiar Canvas interactions, while the agent needed a structured record of what had changed. Mocked screens could communicate the interface, but not whether the loop held together.
Using Claude Code, I built a functional POC connected to Figr’s live agent. It supported movable nodes, text, sticky notes, connectors, sections and marked areas. The agent received the starting Canvas as structured JSON; each subsequent change travelled with the next query as a diff.
That prototype gave product engineering and AI/ML one shared artifact to test and scope. We could evaluate direct manipulation, state changes and agent responses together before committing to the larger production change.
Human-Canvas-AI Interaction Loop
Annotations as Sticky notes in POC
POC Testing - Chat, Agent Inputs and Canvas Activity
Decision 2
Compare context combinations instead of trusting one successful output. I built a blind comparison UI to isolate which context actually improved the agent’s answer
The functional POC proved that the system could work. It did not tell us whether Canvas, chat, activity—or a particular combination—was responsible for a better response.
I built an evaluation view that ran the same prompt against four configurations: chat only, chat + activity, chat + Canvas, and chat + Canvas + activity. Responses appeared as A–D options without revealing their configurations, so we could select the strongest answer before knowing which mode produced it.
The comparison clarified each layer’s role. Chat carried intent. Activity could tell the agent that images had been added, but not interpret them. Canvas state let it understand the actual screens and spatial work. This helped validate the Canvas as current working state, with activity as supporting history.
POC Compare - With/Without Canvas and Activity
Decision 3
Keep the conventions people understood; remove the complexity they did not need. The POC validated familiar Canvas behaviour and exposed two collisions before production
On calls, users needed little guidance to move, resize, connect and arrange objects. Familiar conventions reduced the learning burden. The important findings came from the places where those conventions collided with Figr’s agent and annotation model.
What broke in the POC
What shipped
Why
Every marked region created a floating sticky note that had to stay anchored, avoid overlaps and resize with its attachment.
A persistent marker opened a lightweight annotation. Standalone sticky notes remained; team comments stayed collaboration-only
Preserved precise context for the agent without introducing an unstable layout system
A bottom toolbar became unstable as tools grew and competed with chat.
Familiar Canvas tools moved left, work stayed central and the agent moved right
Created a clearer spatial hierarchy that could scale with more tools
Simplifying these ideas reduced implementation scope without weakening the product model. Chat carried the conversation; the Canvas carried the evolving work.
Decision 4
Treat every direct edit as context. Every Canvas change became a diff, so users could show the agent what changed instead of explaining it again
The full Canvas preserved the current state of the work. Moving a screen, writing a note, connecting nodes or marking a region created a smaller structured update for the next query.
Spatial changes helped people think; diffs made those changes legible to the agent. Direct manipulation was no longer only a UI convenience—it became a way to add product context naturally while keeping that context visible and controllable. Users who sent Canvas edits back to the agent did so 3.2 times on average.
Final interaction and craft
The final model gave each surface a clear job
Tools on the left. Work in the centre. Agent on the right. Selection established scope, contextual actions appeared only when relevant, and every imported or generated artifact remained available as visible product context.
Canvas Final
impact
After the 28 May release, use became deeper; not broader
We compared four complete weeks before the release with four complete weeks after it, excluding rollout week.
9%
30%
63%
73%
51%
34%
5%
1.3%
Added attachments
Acted on an artifact
Artifact full-screen
Used Select & Edit
% active users
0%
20%
40%
60%
80%
Canvas engagement deepened after launch
Reflection
I used code to test the product hypothesis; design judgment decided what should ship
AI-assisted coding let me test the complete interaction model before detailing every state or asking engineering to commit. It surfaced system-level edge cases—annotation overlap, toolbar growth and Canvas-state changes—that a Figma prototype would have hidden.
The most important learning was not that the Canvas needed more tools. It needed a clearer contract: people directly shape the work, the Canvas keeps that state visible, and the agent reads those changes as context. The prototype expanded what we could test. Judgment kept the shipped product focused.