Last updated 2026-10-31
What the exercise is
You will be shown something an AI produced for a real-looking task: a pull request, a document, an architecture proposal, a set of claims about some code, three answers to one question, or a deck of short cards. Some of it is wrong on purpose. Your job is to review it the way you would at work: say what is wrong, where, and why, and give a verdict.
In the pull request and document exercises you also get an assistant you can ask questions. It is deliberately imperfect: it misses some problems and raises some that are not real. Treat it like a confident colleague. Check what it says before you act on it. Accepting a claim that turns out to be false counts against you; dismissing it with a reason counts for you.
How long it takes
- The start screen shows the time limit, if there is one, before you press start. The clock does not run until you do, so open the link whenever you like and start when you are ready.
- A pull request, document or design review usually takes twenty to forty minutes. Ranking three answers takes about ten. The rapid card deck gives you a few seconds per card and tells you how many cards there are.
- Submitting is final. Until then you can edit or delete anything you wrote.
What is recorded
- Your comments, answers and verdict, with the time each was made.
- Your questions to the assistant and its replies, and which of its claims you accepted or dismissed.
- Which files you opened, so an interviewer can see what you looked at.
- Nothing else. No camera, no microphone, no screen recording, no keystroke analysis, no browser tracking beyond what the exercise needs to work.
How AI is used in judging your work
After you submit, a model compares your work with the planted answer key and drafts a score: for each planted flaw, whether you caught it alone, caught it with the assistant, dismissed it, or missed it, and how clear your reasoning was. An interviewer at the employer reads that draft with your own words next to each line, changes anything they disagree with, and decides. No decision is made by the model alone. The model is never told your name, your email or anything about you beyond your work in the session. Read more on the fairness page.
If you need more time or an accommodation
Ask the person who sent you the link, before you press start. Sessions can be created without a time limit or with a longer one, and the link can be renewed if it has expired. Rankr is built to work with a keyboard and a screen reader; the accessibility page says what is supported and what is not yet. The code exercises need a screen at least 1,100 pixels wide.
Your data
What you write in the exercise is held by the employer that invited you, in their Rankr organization. They can export everything held about you as a file, and erase it, on your request to them. Rankr does not use your work to train models, and model providers are used through their APIs, which do not train on it either. The privacy policy has the details, including how to reach us if the employer does not respond.
Before you start
- Read the ticket or brief first. Most planted flaws are places where the AI did not do what was asked.
- Say why. A comment that names the problem and its consequence scores higher than one that only points at a line.
- Check the assistant's claims against the code or the document. You can ask it up to 8 questions in the public sample; the employer sets the limit in a real session.
- Give a verdict you would stand behind in a code review or a meeting, and summarize it in a sentence or two.
Want to see one before the real thing? Take the public sample; it is anonymous and shows you the answer key afterwards.