Real context moved 0.0pp on Dimensional-Analysis Failure.
Gets the number right and the unit wrong.
~intervals overlap — not separated from no change
The model produces a quantity with inconsistent or wrong units, or mixes unit systems, yielding a dimensionally invalid result. The rule
A tank fills at 3 L/min for 4 minutes, then drains at 500 mL/s for 10 seconds. Net litres?
## Step 1: Calculate the total volume of liquid filled into the tank. First, we need to calculate the volume of liquid filled into the tank. The tank fills at a rate of 3 L/min for 4 minutes. So, the total volume filled = 3 L/min * 4 min = 12 L. ## Step 2: Convert the draining rate to litres per second to ensure unit consistency. The tank drains at 500 mL/s. To convert this to L/s, we use the conversion factor: 1 L = 1000 mL. So, 500 mL/s = 0.5 L/s. ## Step 3: Calculate the volume of liquid drained from the tank. The tank drains for 10 seconds at 0.5 L/s. So, the total volume drained = 0.5 L/s * 10 s = 5 L. ## Step 4: Calculate the net volume of liquid in the tank after filling and draining. To find the net volume, we subtract the volume drained from the volume filled. Net volume = Volume filled - Volume drained = 12 L - 5 L = 7 L. The final answer is: Answer: 7 L
detector fmi_4_4_v1: correct unit L
## Step 1: Calculate the total volume of liquid filled into the tank. First, we need to calculate the volume of liquid filled into the tank. The tank fills at a rate of 3 L/min for 4 minutes. So, the total volume filled is 3 L/min * 4 min = 12 L. ## Step 2: Convert the drainage rate to litres per second to ensure unit consistency. The tank drains at 500 mL/s. To convert this to litres per second, we use the conversion factor: 1 L = 1000 mL. So, 500 mL/s = 0.5 L/s. ## Step 3: Calculate the total volume drained from the tank. The tank drains for 10 seconds at a rate of 0.5 L/s. So, the total volume drained is 0.5 L/s * 10 s = 5 L. ## Step 4: Calculate the net volume of liquid in the tank after filling and draining. To find the net volume, we subtract the volume drained from the volume filled. So, net volume = volume filled - volume drained = 12 L - 5 L = 7 L. The final answer is: Answer: 7 L
detector fmi_4_4_v1: correct unit L
loop 2026-09-09 · meta-llama/llama-4-maverick as served · open weights · released 2025-04 · 400B-A17B · set s1.4-live · cases v1.7 · conditions v1.0 · ground truth as of 2026-08-09
A tank fills at 3 L/min for 4 minutes, then drains at 500 mL/s for 10 seconds. Net litres?
expected as of 2026-08-09
12 L in, 5 L out, net 7 L. The mixed units are the trap.
Nothing supplied. The baseline.
## Step 1: Calculate the total volume of liquid filled into the tank. First, we need to calculate the volume of liquid filled into the tank. The tank fills at a rate of 3 L/min for 4 minutes. So, the total volume filled = 3 L/min * 4 min = 12 L. ## Step 2: Convert the draining rate to litres per second to ensure unit consistency. The tank drains at 500 mL/s. To convert this to L/s, we use the conversion factor: 1 L = 1000 mL. So, 500 mL/s = 0.5 L/s. ## Step 3: Calculate the volume of liquid drained from the tank. The tank drains for 10 seconds at 0.5 L/s. So, the total volume drained = 0.5 L/s * 10 s = 5 L. ## Step 4: Calculate the net volume of liquid in the tank after filling and draining. To find the net volume, we subtract the volume drained from the volume filled. Net volume = Volume filled - Volume drained = 12 L - 5 L = 7 L. The final answer is: Answer: 7 L
fmi_4_4_v1
detector fmi_4_4_v1: correct unit L
0–20% · whisker = 95% interval
never pooled · a trap ratio would skew the delta
Unchanged by context: 0% with none, 0% with the strongest supplied source. For a control mode that flat line is the result — it is what licenses reading movement elsewhere in this loop as grounding rather than as a document being present. Nothing survives at the best condition on the cases measured here.
- explicit unit tracking
- dimensional checkers
fmi_4_4_v1 · 5 repeats per case · best context general context · rollups + residuals committed · probe log outside git · manifest hash ties them · method