Why moderated usability testing? I needed to observe real-time behavior as participants encountered the interface for the first time. Remote moderated sessions let me watch where they hesitated and struggled, ask follow-up questions when something wasn't clear, and understand their thought process — not just whether they completed tasks, but how and why.

Approach:

• Prepared a detailed discussion guide with three parts: background questions about participants' editing habits and tool preferences, the main review task with observation, and follow-up questions probing specific interface elements
• Created separate test articles tailored to each participant's area of expertise — so they could review as genuine subject matter experts, not just test users encountering unfamiliar content. Each article contained intentional errors (spelling mistakes, factual inaccuracies, formatting inconsistencies, problematic images) to prompt realistic reviewer behavior
• Wrote "system emails" simulating how a real content owner would invite a reviewer, setting up the scenario naturally to also test the flow outside of the tool
• Created pre-session instruction emails explaining the setup (Zoom, screen sharing, camera/mic requirements) and asking participants not to open the system invitation until the session began — to capture genuine first impressions
• Task-based testing with think-aloud protocol — participants narrated their thoughts while completing realistic review tasks
• Mixed-method: observation during tasks + follow-up questions after each session

Tasks tested:

• Receive the email invitation, read the instructions, open the article
• Locate and use formatting tools (bold, italic, underline, lists, links, removing formatting)
• Add comments on text
• Suggest in-text corrections
• Add or replace images (including header/preview image)
• Understand what header/preview image means
• Save progress and return later
• Download content for their records
• Switch between markup and final view

Screener survey: Built a screener survey in Tally with disqualification logic to filter participants. Disqualification criteria:

• Current or past Mediaforta employees — to avoid bias from internal knowledge
• Anyone who had used ContentRockr before — to capture genuine first-time impressions
• Unable to speak/read English or Dutch — session language requirement
• No access to laptop/desktop with webcam, microphone, and stable internet — technical requirements for remote moderated testing
• Unavailable for a 1-hour recorded session within 3 weeks — project timeline constraint
• Uncomfortable being recorded — sessions needed to be recorded for analysis

Segmentation: Participants who passed were segmented into two buckets based on their responses:

Technical comfort level:

• "I usually ask for help — I prefer simple tools"
• "I can handle basic tools but complex ones can be confusing"
• "I'm comfortable with most tools"
• "I pick up new software quickly"

Experience with content editing/reviewing:

• "I often review or edit written content in tools like Google Docs, WordPress, or Notion"
• "I rarely or never review or edit written content online"

This segmentation aimed to recruit two groups: Rudi-profile participants (limited technical comfort, little editing experience) and more experienced users. The experienced group was important to test existing mental models and habits — to see how ContentRockr compared to tools they already knew, and where their expectations were met or broken.

Consent forms: Designed consent forms covering recording permissions (audio, video, screen), confidentiality, voluntary participation, and right to withdraw. Participants signed before their scheduled session.

Scheduling: Qualified participants received a Calendly link to book their session, automating the process and reducing back-and-forth.

Recruitment outcome:

• 8 participants scheduled
• 6 completed sessions
• 2 cancelled (P7 and P8)
• 0 Rudi-profile participants recruited — all fell into the more technically comfortable, experienced group

Why this happened: The screener was distributed through personal networks only. As a solo researcher managing projects, research, design, and implementation simultaneously, I lacked the capacity to expand recruitment reach. A broader distribution (social media, professional networks, even paid recruitment) could have surfaced Rudi-profile participants, but would have added more sessions than I could realistically handle.

Format: 45–60 minute remote sessions via Zoom, recorded with Fathom for transcription.

Why this format: Remote sessions allowed me to reach participants across different locations and observe them in their natural work environment. The 45–60 minute timeframe balanced thoroughness with participant fatigue — enough time for background questions, the full review task, and follow-up probes without losing focus.

Session structure:

1. Introduction — Confirmed recording consent, explained the session format, emphasized there are no wrong answers and we're testing the software not them, encouraged thinking aloud
2. Background questions — Explored their job, tools they use daily, how they typically collaborate on content, how they give feedback, what they do when encountering unfamiliar tools
3. Main task — Participants opened the system email, followed the invitation link, and reviewed the article as they would in a real scenario. I turned off my video/audio to avoid influencing their behavior, but remained available for questions
4. Observation — Noted hesitations, confusion points, workarounds, and moments of success or frustration. Flagged follow-up questions to ask after the task
5. Follow-up questions — Probed specific interface elements: "Can you show me how to add a header image?", "What's the difference between Save Article and Submit Article?", "Can you turn off track changes?", "Where would you find the briefing?"
6. Closing questions — Overall impressions, what they liked/disliked, what they expected to see but didn't, what surprised them

Before each session, I verified the test environment, set up the participant's tailored article, confirmed consent forms were signed, and tested all links. After each session, I saved and labeled recordings, wrote summary notes immediately while memory was fresh, and identified issues to probe in subsequent sessions.

‣

Screenshots from usability sessions

Participants reviewing tailored articles on topics matching their expertise — comics, interactive displays, philosophy, freelancing in Belgium, and LEGO. Each session captured real-time interactions with the comments panel, formatting tools, and image handling.

Stage 1 — Tagging in Dovetail: Uploaded session recordings to Fathom for transcription, then imported transcripts into Dovetail. I watched each recording, tagged observations, and categorized them: comments and feedback, track changes, formatting, images and media, header image, navigation and discoverability, terminology confusion, positive moments, and bugs.

Stage 2 — Clustering and theme identification: Did a first pass through all transcripts to identify patterns across participants, looking for repetition (same problem across multiple users) and noting severity based on impact and frequency. This initial analysis surfaced the main themes and allowed high-priority issues to move into implementation while detailed analysis continues.

Stage 3 — Detailed video review and synthesis (in progress): Reviewing full session recordings to capture nuances missed in transcripts — hesitations, mouse movements, facial expressions, and moments where participants said one thing but did another. This deeper pass will refine the findings and uncover additional issues.

Main themes that emerged:

• Commenting functionality and limitations
• Track changes inconsistency
• Header/preview image confusion
• Briefing and SEO tabs discoverability
• Formatting and some UI element expectations vs. reality
• Mental models from other tools (Word, Google Docs, Adobe)
• Language inconsistencies (expectation vs. reality)
• Bugs

These themes became the foundation for the findings.

Severity: Critical

Users couldn't edit their own comments after posting — only reply, resolve, or delete. Multiple users searched extensively for an edit button, then had to delete and recreate comments to fix mistakes. Additionally, comment status couldn't be changed by the reviewer (stuck on "pending" even after addressing), comments were ordered chronologically rather than by document position, and users couldn't comment on images at all.

I cannot find the way how I can change my comment. For me it's like a mystery.”

— P3

They are pending maybe for you, but for me they're not pending anymore because I've replied."

— P1

I really wanted to have the annotation or comment option for the pictures."

— P5

Why it matters: Comments are the primary way reviewers provide feedback. When core functionality is missing or behaves unexpectedly, reviewers lose trust in the tool and resort to workarounds.

Severity: High

Users had expectations about text display (line height, text size) that didn't match the interface. The zoom slider was confused with font size control. Comment highlights in the text were sometimes confused with links or other formatting. The visual distinction between different types of markup wasn't always clear.

I found it strange there was no text size but then I guess it's this... put like this I wouldn't know that it's for the text and not for the entire layout."

— P1

Why is this underlined? Is it a link? Is there a comment?”

— P1

Why it matters: When users can't distinguish between different interface elements or don't understand what visual cues mean, they lose confidence in what they're looking at.

Severity: High

Users were confused about what the header/preview image was, why and if they needed to add metadata for an image that wasn't visible, and how to add or replace it. The "refresh" button was misunderstood as "reload" rather than "replace." Some users tried right-clicking for options (like other tools) but it didn't work.

I'm confused now, why did I have to put this metadata here... this is the metadata of the article preview... I wasn't sure why I had to add image metadata here when I didn't actually have an image."

— P1

If you ask me to add an image, I wouldn't know where to go."

— P6

Refresh. I would call it something else. Replace."

— P1

Why it matters: The header image affects how the article appears when published. If reviewers can't understand or interact with it, they can't provide feedback on this element

Severity: Critical

Users made corrections directly in the text, but track changes only recorded some of them. There was no clear pattern for what got tracked versus what didn't. The "final view" toggle was also misunderstood — users thought it meant "submit final version" rather than "view without markup."

I changed three, four things and it recorded only one thing in terms of the track change.”

— P4

When I saw final... I thought okay this is maybe when I'm finished with the article then I click to say this is the final version."

— P6

Why it matters: If reviewers don't trust the system to capture their edits, they'll resort to workarounds or disengage from the review process entirely.

Severity: High

Users were unclear on how to edit, replace, or delete images in the text. The undo function didn't work for image deletion. Embedded media (like YouTube videos) had different interaction patterns than images, causing confusion. Users wanted to add metadata to media as well as images.

I want to comment on the image, but I don't see how."

— P2

I went to actually the title of the article to talk about the picture, which was strange.”

— P5

Why it matters: Visual content needs review just like text. Inconsistent handling between images and media creates uncertainty about what actions are possible.

The following issues also emerged but were less frequent or lower severity:

• Briefing hidden — Critical context (purpose, audience, tone, deadline) was tucked under a small toggle; some users never found it
• SEO tab discoverability — Similar to briefing, users didn't notice or explore this panel
• Save vs. Submit confusion — Users weren't sure of the difference; some thought "save" meant download locally
• Autosave unclear — The "saved" indicator was too subtle (gray text); users weren't sure if their work was being saved
• Missing formatting options — Users expected justify, font size, font selection, text highlighting
• Undo inconsistency — Worked for some actions but not others (e.g., not for image deletion)
• Anonymous edits — Some edits showed as "anonymous" even when the user was identified elsewhere
• Spellcheck missing — No built-in spellcheck in different languages
• Right-click expectations — Users tried right-clicking for options; didn't work
• Language inconsistencies — Terminology didn't always match user expectations (e.g., "refresh" vs. "replace")

I created annotated screenshots in Figma documenting each issue with clear descriptions and recommendations. Issues communicated include:

UI and labeling:

• Toggle on/off state was confusing — clarified that ON (circle right, blue) should show markup, OFF should show clean text with no highlights
• "Save Article" label unclear — recommended changing to "Save for later"
• "Saved" indicator confusing — recommended changing to "Autosaved"
• CTA buttons stacking issue — should stay on one line when space allows
• Margins inconsistency — "Autosaved" text alignment with other elements

Icons and interactions:

• Wrong icon for "no formatting" — specified correct icon to use
• Link icon function mismatch — icon is for copying, not testing links; added inline validation indicator
• Active comment state — no comment should be selected/active on toolbar when no text with comment is clicked
• Canvas highlight when comment selected — yellow outline behavior needs refinement
• Comment icon lingers after clicking elsewhere

Bugs:

• H2 and H3 selected together triggered a bug
• User can't delete own comment (noticed multiple times)
• Word count shows 0 — only updates after clicking around
• Formatting bar doesn't disappear when user scrolls away

Content and display:

• Copyright field — keep © symbol visible even when user starts typing, add space after it
• Briefing panel font too bold — should use regular weight
• User icons — specified correct sizes and colors from design system
• Comments flow — users expect document order (top to bottom), not chronological
• "No feedback yet" message — should not be interactive, just informational
• Embedded media element appearing unexpectedly
• Screen resolution issues on entry screen — add scroll as interim fix

Hints and help:

• Add hints for Zoom In, Zoom Out, Print, Download As Word

‣

Annotated Figma frames showing developer communication

While quick fixes are being implemented, I'm working on the more complex issues that require new design solutions:

Comments redesign → Finding 1

• Adding edit functionality for posted comments
• Reviewing comment status logic (pending/resolved) and who can change it
• Exploring options for commenting on images

‣

Re-design work in progress

Track changes and toggle → Finding 4

• Working with developer to identify why some edits aren't captured
• Redesigning toggle behavior and labeling to match user expectations

Header image flow → Finding 3

• Redesigning the header image area to make purpose clearer
• Improving visibility and interaction patterns

The study aimed to include users with limited technical comfort — the "Rudi" persona the product was designed for. Despite screening, we recruited zero participants matching this profile. All 6 participants were comfortable with technology and experienced with similar tools. This means the findings reflect how experienced users interact with the interface, but may underestimate challenges that less technical users would face.

The screener survey was distributed through personal networks only. As a solo researcher balancing research, design, and implementation, I lacked capacity to expand recruitment through broader channels (social media, professional networks, paid recruitment). A wider reach might have surfaced Rudi-profile participants or revealed a more diverse range of issues.

Six participants completed sessions (two cancelled). While this is sufficient for identifying major usability issues, a larger sample would increase confidence in findings and potentially surface edge cases or less common problems.

The findings presented are based on a first pass through all transcripts. Detailed video review is ongoing and may reveal additional issues or nuances missed in the initial analysis. Some findings may be refined or re-prioritized as deeper analysis continues.

I designed the interface being tested, which introduces potential bias. I may have unconsciously guided participants toward certain interactions, or interpreted ambiguous moments more favorably.

Rather than waiting for complete analysis, I identified quick fixes that could move into development immediately. This allowed the team to address issues while I continued deeper analysis — reducing the gap between research and action.

I created detailed annotated screenshots documenting 30+ issues with clear descriptions and specific recommendations. This direct handoff approach — rather than formal reports — enabled faster implementation and fewer misunderstandings.

I developed a prioritization approach separating quick fixes (bugs, labels, icons) from issues requiring redesign (comments, track changes, header image). This allowed development work to continue without waiting for complex design solutions.

Research doesn't follow a neat sequence. This project had cancellations, parallel workstreams (quick fixes vs. redesign), and multiple analysis passes happening at different speeds. I've learned to embrace this and build structure around it. 

Next time: Plan for cancellations by recruiting 1-2 extra participants. Build buffer time for the unexpected.

A complete first pass through all transcripts surfaced the major themes and allowed prioritization. Detailed video review adds nuance, but waiting for it would have delayed action on clear issues. Separating "fast initial pass" from "deep review" meant quick fixes could move to development while I continued analysis. 

Next time: Plan for staged analysis from the start — it's not a shortcut, it's a practical approach.

I designed elements I thought were clear — the Save button, the zoom slider — but users interpreted them differently. "Save" wasn't obviously "save for later." Zoom was confused with font size. This was a humbling reminder that designer logic and user logic are not the same. 

Next time: Question every "obvious" element. If I think it's clear, that's exactly when I should test it.

One participant (P6) was in a very positive mood and focused more on complimenting the interface than engaging critically with the task. This was new for me — I had to politely redirect them to focus on their experience and finding issues, not giving praise. I later sought advice from UX practitioners on how to handle this. 

Next time: Add explicit wording to my script: "I didn't design this, so your positive or negative feedback won't affect me personally — please be honest."

Direct, visual communication with numbered annotations and specific recommendations worked better than lengthy documents. The developer could see exactly what needed to change and where. 

Next time: Default to annotated screenshots for quick fixes; save detailed documentation for complex design rationale.

This project pushed me to learn Notion and Tally. I used Tally for the screener survey with disqualification logic, and Notion for documentation toward the end of the project. I created my own templates and frameworks in Notion to use for future research. 

Next time: Keep exploring tools — each one opens new ways to work.

Personal networks weren't enough to find Rudi-profile participants. Even with limited capacity, I could have explored other channels — community groups, social media posts, or asking participants to refer others. 

Next time: Build recruitment reach into the planning phase, not as an afterthought.