Practical Data How we test the decisions
The extension asks a decision model short questions about each table. The answers tell the extension which columns to shade and which direction is better.
We copied the tables of 19 public pages and gave hand labels to their columns. Each check compares one decision
with its hand label. The checks use recorded answers, thus you can run them again without a model key: npm run eval -- --offline.
344 of 347 checks pass. The 3 failures show 3 known limitations of the tool.
Checks
| Check | Question | Pass |
|---|---|---|
| Shade or ignore | Is it useful to compare the values in this column? | 198 / 199 |
| Which direction is better | Is a higher value better, a lower value better, or neither? | 115 / 115 |
| Log or linear scale | Are ratios more important than differences? | 11 / 11 |
| Order of labels | Which label is the lower grade? | 5 / 6 |
| Totals and averages | Is this row a total? | 3 / 3 |
| Shared scales | Do columns in two tables use one scale? | 9 / 10 |
| Rows or columns | Do you compare values down a column or across a row? | 3 / 3 |
More
- Known limitations: what the tool does wrong, with examples.
- Glossary: each question, and what each answer means.
- Sites to test against: the public pages behind the checks. Open them with the extension.
- Unit tests and parsing: the code that runs before and after the model.
CI also runs 314 checks on 21 synthetic pages, for layouts such as dark pages and tables that change after load. This site does not show them.