You should have an app installed called Gemini Keyboard. (On the desktop screen, swipe up from the bottom to see all your apps.) From there, select Preferences, and you'll see an option for ‘Vibrate on keypress’.
I have a UK Gemini, which has @ printed below the ‘K’ key. (The symbols on the front, sloping faces of the keys show what you get if you press them while holding down Fn — but if you were a Psion user, you'll know that already :-)
If that doesn't match your US keyboard, you should be able to change that in the Gemini Keyboard app. (There are also related settings in the main Settings app -> Languages & input -> Physical keyboard — I don't know which one you're supposed to use.)