What command line did you use (one of the test programs?)?
Afaik, flite can create words at faster than real-time on a PC (and possibly on a Z...), so you have to tell it to slow down to real-time speed and output that to the speaker/etc.
Edit: From the README in the source
On a 500Mhz PIII, a timing test of the first two chapters of
"Alice in Wonderland" (doc/alice) was done. This produces about
1300 seconds of speech. With flite it takes 19.128 seconds (about
70.6 times faster than real time) with Festival it takes 97 seconds
(13.4 times faster than real time). On the ipaq (with the 16KHz diphones)
flite synthesizes 9.79 time faster than real time.
So if it's fast on an iPAQ, it'll be fast on a Z too.
./bin/flite --setf duration_stretch=1.5
will make it talk more slowly (alter the 1.5 as you wish).