Had good results converting videos to play under SL-5500 using the following options:
scale to 320x240 or 320x180 for widescreen
16.6fps (25/3*2) for PAL, 15fps for NTSC (Z screen is slow enough updating that this looks smooth)
Audio in ogg format, stereo, 56kbps.
video divx 5.1.1, 400kbps, psychovisual enhancements on etc.
Have you tried mplayer -hardframedrop -quiet, after exiting Opie?