Review campaigns

A review campaign lets you evaluate an agent with the help of people you invite to it. Testers hold real conversations with the agent and answer questions about them; reviewers then read those sessions and give their own independent judgement. When the campaign closes, their feedback is aggregated into a report.

Review campaigns are opened from the Review campaigns card on the workspace overview, and belong to the workspace.

How a campaign works

A campaign moves through three states: Draft → Active → Closed. You build it as a draft (agent, questions, and — optionally — a description), then Activate it to start inviting participants and collecting feedback. Once activated, the configuration is locked so everyone evaluates the same thing. When you Close it, results are aggregated and the campaign becomes read-only, ready to read in the report.

Participants have one of two roles:

• Tester — chats with the agent and answers per-session and end-of-phase questions.
• Reviewer — reads a tester’s session and answers the reviewer questions. The tester’s rating stays hidden until the reviewer submits their own, so judgements stay independent.

The full flow at a glance

The walkthrough below replays every step in the real interface. Use Prev / Next to move at your own pace; each step highlights the button to click and the area to watch.

Step by step

1. Open Review campaigns

On the workspace overview, open the Review campaigns card. The page lists your campaigns; the first time, it shows No review campaigns yet.

2. Create a campaign

1. Click New campaign (top-right). The editor opens as a panel from the bottom of the screen, organised into tabs.
2. In General, give the campaign a Name and pick the Target agent (the agent being evaluated). A Description is optional.
3. Open the Questions tab and build the questionnaire (detailed below).
4. Open the Preview tab to check what testers will see — a read-only preview of the After each session feedback form and the End-of-phase survey, built from your questions.
5. Click Create campaign (top-right of the editor). It’s saved as a Draft.

3. Build the questions

The Questions tab has three groups, each with its own Add question button:

• Tester — per-session questions — asked after every session the tester completes.
• Tester — end-of-phase questions — asked once when the tester finishes.
• Reviewer questions — shown to reviewers while they review a session.

Click Add question in a group to append a question card. Each card has:

• Prompt — the question text (e.g. Was the response clear?).
• Type — a dropdown on the right of the prompt. This is how you change the question’s mode. Open it and choose one of:
  • Rating (1–5) — a 1-to-5 score.
  • Single choice — pick one option. Choosing this reveals an extra Options field below the prompt — enter the choices separated by commas (e.g. Yes, No, Not sure).
  • Free text — a written answer.
• Required — a toggle; turn it on to make the question mandatory.
• Factual — a toggle shown only on per-session tester questions that aren’t free text. Turn it on when the question has a correct answer; it stays visible to reviewers during blind review.

Reorder a question with the ↑ / ↓ buttons, or remove it with the trash icon, at the right of the card. Repeat per group to cover what each role should answer.

4. Activate the campaign

From the campaigns table, open the draft (the pencil / Edit icon). In the editor header, top-right, click Activate (rocket icon), then confirm in the dialog. From this point the configuration (agent, questions) is locked — activate only once you’re happy with it.

You must activate a campaign before inviting participants. Until then the Participants tab reads Save the campaign first, then invite participants. (new campaign) or Activate the campaign before inviting participants, so they can accept and find it in their list. (draft).

5. Invite testers and reviewers

In the editor’s Participants tab (available once the campaign is Active), the Invite participants box has:

1. Emails (comma or newline separated): enter one or more addresses.
2. Role: choose Tester or Reviewer. Everyone in one batch gets the same role, so invite testers and reviewers in separate batches.
3. Invite: click it to send the invitations. A People invited notification confirms it.

Each person gets a pending invitation, listed under Pending invitations below the participants table: People invited to this campaign who have not answered yet. Each card shows the email and when it was sent. If your platform sends emails, each participant receives the link by email. Otherwise, click the link icon (Copy invitation link) on each card and send the link to that participant. It opens the platform with their email already filled in. They must sign in, or sign up, with exactly that address. They sign in with your organization’s account and accept the invitation on the home page (see Sign in and accept your invitations). They can only accept while the campaign is Active.

Once a person accepts, they leave the pending list and appear in the participants table, with the columns Email, Role, Status and Actions.

To cancel an invitation that hasn’t been answered, click the trash icon (Delete invitation) on its card and click Confirm in the Delete invitation? dialog. To take access back from a participant, click Revoke on their row and confirm in the Revoke email? dialog; the participant is removed from the campaign and this cannot be undone.

6. Manage campaigns from the table

Each campaign shows its Name, Status, Members count, and Created / Updated dates. In the row’s Actions (right-hand column) you can:

• Edit (pencil icon) — reopen the editor.
• Delete (trash icon) — only available while the campaign is still a Draft.

7. Close the campaign

The Close button isn’t on the table — it lives inside the editor. To find it:

1. In the campaigns table, click the Edit (pencil) icon on the active campaign to open the editor.
2. In the editor header, top-right, click Close — this button only appears while the campaign is Active (a draft shows Activate there instead; a closed campaign shows neither).
3. Confirm in the Close “…”? dialog by clicking Close campaign.

A closed campaign is read-only: you can no longer edit it or invite anyone.

8. Read the report

Once closed, the editor gains a Summary tab, and the campaign’s aggregated results are available in the Campaign report — the mean tester rating, number of sessions, end-of-phase surveys and a per-question breakdown. Use Download CSV to export the raw data.

Tips

• Keep questions short and focused — testers answer them after every session.
• Mark a question as Factual when there’s a correct answer; it stays visible to reviewers during blind review.
• Invite a few testers and reviewers rather than many at once, so you can adjust the next campaign based on what you learn.
• Participants sign in with your organization’s account using the email you entered. Double-check each address.

Troubleshooting

• I can’t invite anyone. The campaign is still a Draft. Activate it first.
• An invitation stays pending. The person hasn’t accepted yet. Remind them to sign in with their organization’s account, using the email you invited, and to click Accept on the home page. If they see No access yet, they used a different email: delete the invitation and invite the right address.
• I can’t change the questions or agent — the configuration locks once the campaign is Active. Create a new campaign if you need a different setup.
• I can’t delete a campaign — only Draft campaigns can be deleted. Close an active one instead.
• I can’t find the Close button — it isn’t in the table. Open the campaign with the Edit (pencil) icon; Close is in the editor header, top-right, and only shows while the campaign is Active.
• I can’t change a question’s type — the Type dropdown is disabled once the campaign is Active (the configuration is locked). Types can only be changed while it’s a Draft.
• The results are empty — aggregated results become available once the campaign is Closed.