Worse than that - voting was very public. Even in those cases where Blacks were not threatened just for casting a vote, the boxes themselves were in full view, and your slip of paper went into one box or the other. Your boss saw where your vote went.
(Of course, there was also terrorism afoot to keep Blacks at home.)
Secret ballot voting didn’t occur until the Progressive Era (+/- 1900s).