Why moderated usability testing? I needed to observe real-time behavior and ask follow-up questions when users encountered friction. Remote sessions allowed me to observe experienced users in their natural work environment.

Approach:

• Task-based testing with think-aloud protocol—I wanted to understand decision-making, not just actions
• Mixed-method: observation + interview after each task
• Prepared interview scripts in two languages (English and Dutch), consent forms, and session instructions
• Practiced the script aloud (in English and Dutch)

‣

Planning document screenshots

Criteria:

• Active ContentRockr users with at least 3 months experience—to ensure they were familiar with the tool and could compare it to other editors they use
• Professional content creators, not managers or clients—because the focus was on people who actually write content, not just review or manage it

Why 2 participants? ContentRockr has a small user base. I recruited from existing content creators on the platform, and two were available within the timeline. Both participants revealed the same core insight independently, which suggested I had found a real pattern worth acting on.

Format: Sessions were conducted remotely because participants were located across the country, making in-person observation impractical. The 45-60 minute timeframe is optimal for usability testing—shorter sessions feel rushed and don't allow for proper exploration, while longer sessions lead to participant fatigue and reduced focus.

Structure: Background questions about their work and tool usage, followed by two tasks, each with follow-up questions.

Tasks tested:

1. Write content directly in ContentRockr — to observe friction points and understand why they don't use it as their primary tool
2. Write content in their preferred editor, then paste into ContentRockr — to understand their habits, what makes that tool their preference, and how ContentRockr compares

‣

Session screenshots

Tagging in Dovetail: I transcribed the recordings in Dovetail and tagged observations across categories: Editing (formatting, saving, layout, copy/paste), Text Editor preferences (Google Docs, Word, Vim), Content Writing process, Feelings (pain points, frustrations, positive moments), System issues, and SEO behavior.

‣

Tags screenshots

Affinity mapping in FigJam: I clustered related observations into themes to identify patterns. Main themes that emerged:

• Writing process and tool preferences
• Editing and formatting behaviors
• SEO field usage
• Copy/paste issues
• Keyboard shortcuts and right-click habits

‣

Clusters screenshots

Key observation: During the background interviews, both participants independently revealed they don't write content in ContentRockr—they write in their preferred tools (Vim, Google Docs, Word) and paste it in. This wasn't something they thought of as unusual; it was just "how they work."

All the time I have been writing my text outside of ContentRockr and just copying it and pasting it into ContentRockr... I have just been working like this for the last 20 years. So I have this editor, Vim, that I'm used to. It's really bare bones. I don't have to click around the screen. I just know the keyboard shortcuts by heart."

— P1

Nee, nee, want dat vind ik dus redelijk onhandig. Zeker qua layout... Google Docs vind ik wel handig, omdat je suggesties kan maken in een tekst."

— P2

Why this matters:

The original goal was to make ContentRockr the preferred writing environment. But users weren't avoiding ContentRockr because of fixable usability issues—they had established workflows built over years. They work with multiple clients who require different tools and formats, so they stick to their own preferred tool and adapt to whatever each client needs. No amount of editor improvements would change these deeply ingrained habits.

All the time I have been writing my text outside of ContentRockr and just copying it and pasting it into ContentRockr."

— P1

Nee, nee, want dat vind ik dus redelijk onhandig."

— P2

Why it matters: The entire product assumption was wrong. No amount of editor improvements would change deeply ingrained workflows. Users need their preferred tools (Google Docs, Vim, Word) for creation.

Evidence: Users refer back to the briefing multiple times to check requirements. Some copy the briefing into their working document to keep it visible while writing.

Why it matters: The entire product assumption was wrong. No amount of editor improvements would change deeply ingrained workflows. Users need their preferred tools (Google Docs, Vim, Word) for creation.

When I copy a text from LibreOffice... it adds some stylesheet codes."

— P1

Evidence: Bullet points didn't paste correctly—paragraphs got bullets they shouldn't have, and users couldn't remove them.

Why it matters: If users are going to paste content in, that experience needs to be frictionless. Instead of "fixing" the editor, improve paste handling and preserve formatting from external sources.

Evidence: Users switched back and forth between SEO and editor tabs to check things and copy content. They copy/paste the title for SEO title, the first paragraph for meta description.

Why it matters: This could be automated. Side-by-side view and pre-populated fields would reduce friction.

Evidence: Users were positively surprised keyboard shortcuts worked. They frequently right-clicked looking for solutions. They have habitual formatting structures (intro, H2, H3, closing paragraph).

Why it matters: Supporting familiar patterns (keyboard shortcuts, right-click menus, standard formatting options) builds trust and reduces learning curve.

Finding 1, 3 & 5 → The redesigned interface embraces users' habit of writing elsewhere rather than fighting it. Users can import text, paste with or without formatting through multiple access points: the empty state prompt, the Edit menu, or right-click context menu. Each method is supported by corresponding system keyboard shortcuts (Ctrl+V for paste, Ctrl+Shift+V for paste without formatting) to align with existing habits.

‣

Low-fidelity wireframes

Finding 2 → The redesigned interface moves all tabs (Briefing, Feedback, Chat, History) next to the canvas, supporting side-by-side view. The briefing can now be opened alongside the content when needed, eliminating the need to switch tabs or copy the briefing into a separate document.

‣

Low-fidelity wireframes

Finding 4 → Like the briefing, the SEO panel can be opened next to the editor when needed, allowing users to view and edit both simultaneously. The improved SEO panel automatically populates fields based on article content—the title becomes the SEO title, the first paragraph generates the meta description, and key terms are suggested as focus keywords. This eliminates the constant switching between tabs and manual copy-pasting users were doing before.

‣

Low-fidelity wireframes

Finding 5 → The redesigned interface reduces visual clutter and mirrors conventional text editors users are accustomed to. The canvas is now much larger, secondary navigation is collapsed when not relevant, and the menu bar is more prominent with clear, recognizable items—File, Edit, Format, Insert, and View. The Format menu provides access to text styling (Bold, Italic, Underline, Strikethrough) and heading structures (H1, H2, H3, Quote) that match users' habitual formatting patterns. All options include corresponding keyboard shortcuts.

‣

Low-fidelity wireframes

I recruited only 2 participants, both professional copywriters with 15-20 years of experience. While both independently revealed the same core insight (suggesting pattern validity), this narrow sample means I may have missed perspectives from other user types—content managers who review rather than write, or clients who commission content. A broader sample across roles could have revealed different needs or workflows.

Both participants were active ContentRockr users with at least 3 months experience. This was intentional—I needed people familiar enough with the tool to identify usability issues. However, this means I didn't capture the new user experience. New users might have different pain points during onboarding, or they might be more willing to adapt to ContentRockr's editor before establishing external tool habits.

I conducted all sessions alone—planning, moderating, note-taking, and analysis. This introduces possible moderator bias. An observer could have caught details I missed while facilitating, or challenged my interpretations during analysis. The lack of real-time collaboration meant I couldn't validate observations immediately.

I chose simple topics (weather in Belgium, favorite food) so participants could write without extensive preparation. The goal was to focus on tool usage, not content quality. However, this backfired—the tasks weren't representative of their real workflow. In actual work, content creators research topics, gather information in separate documents, create structures, and draft over time. The simplified tasks didn't capture this complexity, which means I may have missed friction points that only appear in realistic scenarios.

The research revealed that the core product assumption—making ContentRockr the preferred writing environment—was fundamentally flawed. This shifted the conversation from "how do we improve the editor" to "how do we support users' actual workflows." This prevented continued investment in editor features users would never adopt, allowing the team to focus development resources on workflow management and content validation.

I created low-fidelity wireframes demonstrating how the interface could better support paste-first workflows, side-by-side views for briefing and SEO, and familiar editing patterns. Most of these recommendations are being implemented in the current redesign.

Based on the research findings, the development team addressed multiple technical issues. Paste functionality was overhauled—stylesheet codes no longer appear when pasting from external editors, bullet point handling was corrected so lists paste properly without extra bullets, and a "paste without formatting" option was added. Formatting controls were also fixed, resolving unexpected text bolding and restoring keyboard shortcut functionality for indentation. These fixes directly addressed the friction points users encountered in their actual workflow.

While not part of my initial design recommendations, the new redesign I have been developed in 2025 addresses other friction points identified in the study. Notifications were redesigned to be non-intrusive and not cover content. The file upload system now prevents selection of unsupported formats. System notifications are being completely redesigned based on user feedback from the sessions.

Meeting with a content creation expert before designing the study would have helped me understand realistic content workflows. This would have prevented the simplified test tasks (weather, food topics) that didn't capture the complexity of real content creation. 

→ Next time: Interview internal stakeholders and subject matter experts before research planning.

I had a surface-level understanding of ContentRockr when I started—I knew it was a content platform but didn't fully grasp how it fit into users' broader workflows and all the design and tech complexities of it. 

→ Next time: Find a way to better understand the product before planning the research, for example shadow the product team for a day, review support tickets before planning, and use the tool myself for real tasks to understand the full context.

I was surprised by how much copy-pasting happened. Users had built entire workflows around getting content into the system, not creating it in the system. This taught me that when users develop complex workarounds, the problem isn't the feature—it's the product assumption. 

→ Next time: Watch for workaround patterns as signals to challenge core assumptions.

With only 2 participants available, I recruited experienced users to identify usability issues. However, new users might have revealed different friction points. 

→ Next time: Mix experienced and new users, and include observational sessions alongside task-based testing to capture natural workflows without researcher influence.