Someone takes a reaction test and gets 230 ms. The next day they get 255 ms. Did they get slower, or was it noise? The honest answer is that a single round cannot tell you. Reaction-time scores vary for reasons that are real but temporary, reasons that are measurement artefacts, and reasons that are genuine changes in skill. Separating them is what reliability is about.
The practical answer
A reaction-time score is more reliable when the measurement conditions are controlled, the number of trials is large enough, and the summary is a median across several rounds. It is less reliable when the device, time of day, or caffeine intake changes, or when a single round is treated as a verdict. Building a useful baseline takes repeated sessions, not one attempt.
Two different kinds of variation
Measurement variation comes from the instrument: display refresh, input polling, browser scheduling, the device's power state, and the exact moment the test starts its clock. State variation comes from you: sleep, fatigue, attention, arousal, caffeine, and time of day. Both change the number, and they can change it in the same direction or opposite directions.
There is also skill variation — genuine learning or loss. It tends to move slowly, while state and measurement variation can move quickly. A single bad round is more likely to be state or measurement variation than a real loss of skill; a consistent shift across many sessions is more likely to be meaningful.
What "reliable" means
Reliability is about consistency, not truth. A test can be reliable and still be biased: if every score includes a fixed device delay, the numbers will be consistent but systematically higher than a lab measurement. Reliability tells you how much a score would vary if you repeated the same measurement under the same conditions; validity tells you whether the measurement means what you think it means.
In practice, a reliable personal baseline means that two sessions under similar conditions produce similar medians. If your medians swing widely with no change in conditions, the score is too noisy for the comparison you are trying to make; more trials, fewer changes, or a longer test may help.
Practice effects
The first rounds on a new test are learning rounds. You are figuring out the rhythm, the visual layout, and the required movement. Scores often improve quickly over the first few rounds and then plateau. That improvement is real, but it is partly task familiarity rather than a permanent change in your nervous system.
This is why a baseline should be taken after a warm-up, not on the first attempt. It is also why a sudden improvement after a new test or a new setup should be interpreted cautiously: the change may reflect learning or a changed measurement chain, not a change in you.
Averaging and summary statistics
The five-attempt average reduces the influence of one outlier, but it does not eliminate it. A single 400 ms lapse can still pull the mean up; the median shown alongside it is less sensitive to that one value. Across rounds, the median of several averages is a better summary than the best single round.
More trials generally improve reliability, but there are diminishing returns for a casual personal test. Five attempts per round and three rounds per session is a reasonable compromise; a laboratory study would use more, and the accuracy guide explains why the measurement chain matters as much as the trial count.
What makes a change meaningful
A change is more convincing when it is larger than your normal session-to-session variation, consistent across repeated sessions, and not accompanied by a device or protocol change. A 5 ms difference in one round is almost always noise; a 20 ms shift that persists across a week, on the same setup, is worth investigating.
If you want to test a change — a new mouse, a new sleep schedule, a new warm-up — use a controlled comparison: baseline, one change, repeated measurements, same device, and a note of the conditions. The latency calibration guide covers the setup side, and the training guide covers practice effects.
How to build your own baseline
- Pick a fixed setup. Same device, browser, display, and input method; use the same reaction time test version.
- Warm up. Take one slow round before the scored rounds.
- Run three rounds per session. Record the median of the three five-attempt averages.
- Repeat on at least three separate days, at a similar time of day, under similar conditions.
- Write down the conditions — sleep, caffeine, device, and anything unusual.
- Compare medians, not bests. A personal best is a motivating anecdote, not a reliable baseline.
This is deliberately boring. The point is to make the number stable enough that a real change can be seen. A baseline built this way is far more useful than a screenshot of one unusually good round.
A worked example: what a 25 ms difference means
Suppose your baseline median is 240 ms and today's session median is 265 ms. Is that a meaningful slowdown? It might be. It might also be a bad night's sleep, a different time of day, a browser update, or simple round-to-round variation. The difference is large enough to notice, but one session is not enough to attribute it.
A better approach is to repeat the measurement on another day under similar conditions. If the 265 ms median persists, it is more likely to be a real state or setup change. If it returns to 240 ms, the first session was probably noise. The same logic applies to an apparent improvement: a single fast session is a hypothesis, not a conclusion.
This does not mean you need a laboratory to use the test. It means you should match the confidence of your conclusion to the amount of evidence you have. One round supports a guess; three sessions support a personal trend.
Reliability within a device, not across devices
Most of the practical reliability you can achieve is within one device. If the phone, browser, and conditions stay the same, repeated medians can be reasonably stable. Change the device and you change the measurement chain, so a cross-device difference is a comparison of instruments as much as a comparison of you.
This is why a new phone or laptop should be treated as a new baseline rather than as an improvement or decline. Record the first few sessions on the new device, then compare future sessions with those, not with the old device's numbers. The mouse vs touchscreen guide explains the device difference in more detail.
How many sessions are enough?
There is no universal number, but three sessions on separate days is a reasonable starting point for a personal baseline. If your medians are still swinging widely after three sessions, add more sessions or more rounds per session before drawing conclusions. If they are stable, you can use the median of the three as your reference and update it when your setup changes.
For a one-off comparison, a single session with three rounds is better than one round, but it is still weak evidence. For anything you care about, repeat it. The cost of another session is small compared with the cost of acting on noise.
Which test should you use for a baseline?
Use the measurement that matches the question. For simple visual response, use the reaction time test and keep its five-attempt format constant. If you care about target acquisition, use the aim trainer; if you care about repetition speed, use the CPS test. Do not mix the three into one baseline, because they measure different tasks.
The same principle applies to test versions. If the test's layout, trial count, or scoring changes, the baseline changes with it. Record the version or date you are using, and re-baseline after a material change. A stable measurement is more useful than a large number of incomparable sessions.
What this does not mean
A reliable baseline is not a medical measurement, and a change in your score is not a diagnosis. If you are concerned about a persistent change in your reaction time, attention, or coordination, that is a question for a qualified professional, not for a browser test. The site measures a narrow task under consumer conditions; it is useful for personal tracking and context, not for clinical conclusions. A baseline is a tool for self-comparison, not a score to defend; when the setup changes, record it and start a new baseline for that setup. Reliability is a property of a measurement protocol, not a single number, so treat the routine as part of the result.
A baseline worksheet
Record the date, device, browser, input method, warm-up, and three round medians. Keep the sheet for at least three sessions before comparing conditions. If you change devices, start a new section rather than carrying the old baseline across. The worksheet is more useful than a screenshot because it shows the conditions behind the number.
For the rest of the measurement series, see the reaction time guides.
Sources
- Woods, D. L., Wyma, J. M., Yund, E. W., Herron, T. J., & Reed, B. (2015). Factors influencing the latency of simple reaction time. Frontiers in Human Neuroscience, 9, 131. DOI.
- Der, G., & Deary, I. J. (2006). Age and sex differences in reaction time in adulthood. Psychology and Aging, 21(1), 62–73. PubMed record.
- Reaction Time Test Accuracy — what a single score includes.
- Reaction Time Training: What Transfers — practice effects and transfer limits.
- Methodology — how this site's tests are built and scored.