Case Study
Joy AI
Would people trust an AI to help them trade in their device?
A concept evaluation study exploring how customers respond to AI voice guidance and AR scanning in a device trade-in experience, and what it would take for them to actually trust it.
Role
Lead UX Researcher
Team
Product and Design partners
Platform
Mobile Concept
Duration
8 Weeks
Study Type
Concept Evaluation
Methods
Talk-Aloud Study
Participants
15 Participants
Research Outcome
Trust Design Direction
The opportunity
Joy AI was a product concept, not yet built, that imagined a new kind of device trade-in experience. The concept combined AR scanning, conversational AI voice guidance, and a clean commerce UI to help customers assess and trade in their devices without needing to visit a store.
The product team needed to know whether this concept was worth building. Specifically: would customers trust it? Would the AI and AR features feel helpful, or would they create more questions than they answered? This study was designed to find out before any significant development investment was made.
What we needed to learn
The team had a concept but not yet evidence. Before investing in development, we needed to understand how real people would react, not just whether they could complete tasks, but whether they would actually trust this kind of experience.
Research questions
- 1How do customers react to AR device identification, and what concerns come up around camera access and accuracy?
- 2What does a trade-in process need to do to feel trustworthy? What information do people need, and when?
- 3How do people navigate switching between AR, voice, buttons, and chat input in a single flow?
- 4Where does the experience feel intuitive, and where does it create friction or confusion?
Hypotheses going in
- People would hesitate around the AR scanner, both the act of granting camera access and doubts about whether it would identify their device correctly.
- Existing expectations about device trade-in value would shape how participants judged the concept's fairness, regardless of the actual offer.
What we learned
The UI foundation held up
The clean structure, browse-by-category, and suggested upgrades all worked. The gaps were about explanation, guidance, and reassurance, not the layout.
AR scanning had real potential
Participants responded well to the camera scan, but they needed a manual fallback, clearer accuracy feedback, and some reassurance about privacy.
AI voice missed the mark
Voice narration felt more like a demo feature than a useful guide. It needed to be optional, and tied to something participants actually wanted help with.
How I approached the research
15 participants completed a remote talk-aloud study via Dscout, working through 6 prompts covering the full trade-in journey from home screen to checkout confirmation.
Research decision
This study focused on trust, expectations, and perceptions rather than task completion rates or usability metrics alone. The reason: a feature can be perfectly usable and still not be trusted. For an AI concept involving camera access and automated device valuation, we needed to understand the emotional and cognitive response, not just whether people could get through the flow.
Inclusive research consideration
Participants came in with very different levels of familiarity with AI tools. Some had used voice assistants and AR features before; others hadn't. That gap mattered when interpreting reactions to the AI voice and AR scanner, what felt intuitive to one person felt unfamiliar or uncertain to another. Findings were analyzed with that variation in mind rather than treating all hesitation as a usability failure.
Research findings
Findings organized by area. Each section includes what worked and what raised questions.
Design opportunities
These are the areas where the research pointed most clearly toward a change, and why.
UI & Home Screen
Keep the clean structure, browse by category, suggested upgrades, but clarify labels and device tags to reduce confusion about what's personalized versus generic.
AR Scanner
Add a manual fallback if the scan doesn't work or gets the wrong device. Ask for camera permission upfront and be explicit that photos are not stored.
AI Voice
Make narration optional with a clear toggle. Reframe the voice from a flashy intro to a contextual guide, something that explains compatibility, answers common questions, or walks through trade-in value.
Trade-In Clarity
Explicitly tell people that cross-category trades are allowed. Move 'Trade in a different device' higher in the flow. AI could surface eligible devices based on account history.
Sharing Feature
Let people share via link, SMS, or email. Give them control over what's included, value only versus full device details. Consider surfacing this contextually rather than as a fixed step.
Trade-In Value & Confirmation
Show taxes and shipping before payment, not after. Provide a clear confirmation screen and a receipt. If AI is explaining the trade-in value, make that explanation visible before the person commits.
Business Impact
What changed because of this research
Helped the team distinguish which AI features increased trust and which created hesitation, before any significant development investment was made.
Shifted design priorities from adding more AI functionality to improving transparency, user control, and confidence throughout the trade-in experience.
Provided clear direction for future iterations, including optional AI guidance, stronger camera permissions messaging, and clearer trade-in explanations.
What surprised us
Not everything landed the way we expected. A few findings shifted how we understood the concept.
01
The sharing feature resonated more than expected
We thought sharing might feel niche. Instead, several participants immediately connected it to trust, sharing the trade-in with someone else meant they couldn't be taken advantage of. That was a use case we hadn't fully anticipated.
02
The green check created new confusion
The AR scanner's success state, a green checkmark, was supposed to signal that the scan worked. But participants read it differently. Some thought it meant the trade-in itself had been approved. A small visual element was carrying more weight than intended.
03
AI voice divided the room clearly
The split wasn't random. Participants who already used voice assistants regularly were more open to it. Those who didn't found it intrusive. That pattern made it clear the feature needed to be optional, not removed, but not assumed.
Looking back
This project reinforced something I think about a lot in AI concept research: novelty and trust are in tension. People can find a feature impressive and still not believe it. The "wow factor" of AR scanning got participants talking, but their real questions were about accuracy, privacy, and fairness, none of which the UI was answering clearly at that point. The study helped the team see that the work wasn't just adding polish. It was building the conditions for trust from the ground up.
Participant perspective
"Overall, the process felt modern and convenient, but lasting trust hinges on fair pricing, accurate scans, and clear logistics."
Many participants said they would use this in real life, mostly for the convenience. But willingness to use it came with conditions: fair valuations, transparent process, and clarity about what the AI was actually doing at each step.