who was excluded from being able to take the oath of loyalty to gain the right to vote