Lameness in dairy cattle is usually identified by judging the locomotion of dairy cows. The objective of this study was to determine differences in intra- and interrater reliability among 3 different locomotion scoring systems for dairy cows. The 3-, 4-, and 5-point locomotion scoring systems were used by 42 observers to score the gait of cows in a set of 45 video clips. Each observer scored the set of clips 4 times, once using each of the 3 scoring systems, then again repeating one of the 3 systems. The order of presentation of the video clips was randomized between each viewing, and participants were randomly assigned to repeat one of the 3 scoring systems. Intra- and interrater reliability were calculated as the weighted kappa coefficient (κw). The mean intrarater reliability for the 3-point system was κw = 0.57 (95% CI: 0.43–0.71), for the 4-point system κw = 0.60 (95% CI: 0.53–0.68), and κw = 0.64 (95% CI: 0.55–0.72) for the 5-point system. The CI for intrarater reliability suggests that no difference exists in κw among the 3 systems. The mean interrater reliability for the 3-point system was κw = 0.42 (95% CI: 0.41–0.43), for the 4-point system κw = 0.43 (95% CI: 0.42–0.44), and κw = 0.47 (95% CI:0.46–0.48) for the 5-point system. These intervals indicate that interrater reliability is greater for the 5-point system when compared with the 3- and 4-point systems. Using a level of acceptable reliability of κw >0.60, none of the scoring systems achieved good reliability for interrater comparisons, while both the 4- and the 5-point scoring systems achieved good intrarater reliability. This result suggest that within observer reliability is adequate for the 4- and 5-point scoring systems, but not for the 3-point system. In contrast, between observer reliability is lacking for the 3 locomotion scoring systems examined in this study. Further analysis is needed to determine if cows are consistently identified as lame and severely lame using the 3 scoring systems.