The N64 used optical sensors in its joysticks. If you take apart the N64 joystick you’ll see the joystick is attached to some disks with slits in them. The N64 had an optical sensor that would count how many slits passed by.

Here is a GIF demonstrating the mechanism.