The SD card reader on the C-series is a serious bottleneck in my experience. The internal flash is faster, giving Angstrom et. al. the appearance of being faster. Also, with Debian, programs are bigger (as mentioned above, they are more featureful), so loading/unloading data from storage will naturally take longer anyway. However, in my experience, EABI programs in Debian run about as fast as their Angstrom counterparts once loaded into RAM.
If you are using a GUI, by far the biggest bottleneck with responsiveness is anti-aliasing. If you can turn it off (if your GUI even supports it), you will notice a HUGE speed boost. In fact, with XFCE on my Debian zaurus, the difference in speed before and after is about an order of magnitude.