Help - Search - Members - Calendar
Full Version: Opie-reader Learns Speaking
OESF Forums > Distros, Development, and Model Specific Forums > Everything Development > User Request for Applications
malik
this would be a nice application!


malik
adf
It definitly would.
malik
hello adf,

there are some binaries for the zaurus concerning tts and speech synthesis like
flite (all in one, but not very useful), mbrola (only synthesis) and freetts (java).
one can easily integrate dictionaries (f.i. ztrans) into the reader. using the clipboard
content and the not yet implemented ability to start external application (hello
tim:-)) opie can easily do this job or not?!

malik
TimW
opie-reader used to have an interface to flite built-in. I did it as part of a project for a blind friend but because of the large buffer built into the front end of flite it was impossible to get decent control over stopping and starting the reading and making sure the bookmarks aligned with where flite had got to in its speech.

This was a long time ago with an older version of flite so maybe it will work better with the new version. The code is still in there but disabled with a #define so it should be reasonably easy to get something working again but I suspect a more direct interface with flite may be needed (I experimented with lots of different ways of interfacing with flite in case the buffering was caused by the interface and I can't remember with what I left in).

All I need is some spare time a copy of the current version of flite - preferably x86 as well as ARM. I'll try and take a look soonish.
malik
hello tim,

the last release is from february 2003, version 1.2. there is a gap of more
than one year to version 1.1 ( december 2001). one can find the corresponding
files here:

http://www.speech.cs.cmu.edu/flite/packed/flite-1.2/

it was only an idea. i know that the commercial stuff concerning speech synthesis
and recognition is getting better and better, maybe also the free projects. it would
be a kind of killer application and very useful. i really dont know much about
programming and compiling, but if i can help, i will try it.

malik
adf
slightly useless observations:
flite (the inclusive binary) seems to work fine..though I haven't really asked much of it
the IBM multimodal stuff seems to recognize speech pretty well.... maybe there is some hope along those lines to be found by mining at ibm (since you mentioned recognition)?
caunt
TimW,

Wouldn't the solution be to feed the freetts engine small known chunks...like paragraphs? Those ought to be easy enough to parse.
('course if you open a text with no paragraphs at all, like some writers create. then you are back to having to wait till the engine finishes what it's chewing on.) And then you know where to put the bookmark also.
padishah_emperor
Tim, if you could add support for flite that would be excellent, the Yopy has flite running fine and it would work well if you spat small chunks of text as suggested above to it at a time.

Support flite!!
TimW
QUOTE(caunt @ Feb 28 2005, 09:58 PM)
TimW,

Wouldn't the solution be to feed the freetts engine small known chunks...like paragraphs?  Those ought to be easy enough to parse.
('course if you open a text with no paragraphs at all, like some writers create. then you are back to having to wait till the engine finishes what it's chewing on.) And then you know where to put the bookmark also.
*


I haven't tried freetts yet but the problem was that flite would buffer up several minutes of speech with no feedback of where it had got to. Whilst it is trivial to send a paragraph at a time it was impossible to know when to send the next paragraph because there was no way of knowing when the previous paragraph had finished. The best I came up with was to have the user tap a key when (s)he was ready for the next paragraph.

When I get some spare time I'll have a look at the current versions of flite and freetts and see whether the same problem exists.
TimW
QUOTE(padishah_emperor @ Mar 1 2005, 12:04 AM)
Tim, if you could add support for flite that would be excellent, the Yopy has flite running fine and it would work well if you spat small chunks of text as suggested above to it at a time.

Support flite!!
*


I'll make sure to support both should I manage to get it working (unless freetts doesn't have the problem but flite still has - in which case would you settle for tapping the key for more data solution I described above?).
caunt
Hi Tim,
I'm the one who emailed you about this and have been looking into it. FreeTTS would be great, but unfortunately REQUIRES java1.4, which is apparently unavailable on the Z. (at least on my 5000d) That leaves us with flite.
The SPOKEN author managed to get flite working quite well, and I've contacted him. He hasn't released his source to the public yet, (and I don't really want to push him into it,) but he did tell me that he ended up feeding the engine one sentence at a time, and actually restarting after each sentence. The only problem with SPOKEN is that it is no longer in development and only accepts plain text. Marrying the remarkable cababilities of QTReader to obtain plain text from a number of formats, to the flite engine as in SPOKEN would be ideal.
TimW
I only mentioned freetts because someone else did. At least I know why I couldn't find it, now 8^).

Anyway, I think I've found a way to use flite correctly but I need to get flite compiled as a shared library for the desktop and for the zaurus but I can't find any pre-compiled versions and my laptop doesn't have enough memory to do it itself. I'll be giving it a go once I can get the kids off the "big computer" 8^).

The other alternative is just to start flite for each paragraph as seems to be done in SPOKEN. That would work but seems a bit inefficient. I may try this out as a quick and dirty way of getting something working.
caunt
Hi Tim,

I don't know about compiling as a shared obect except the flite web site said it could be done:
http://www.speech.cs.cmu.edu/flite/doc/flite_4.html#SEC4
I do know that there is an arm binary that runs from the command line that works just fine on my 500d. You can find a copy here:
http://cmuflite.org/packed/flite-1.2/flite_arm_bin.tar.gz

I wonder how much of a difference it would make to call the api yourself vs run the executable.

You said there was trouble starting/stopping and aligning bookmarks.
Can you set bookmarks to a particular line of visual text? I thought the bookmark was just for the visual screen. At any rate, I see no code available for "stopping", so it appears our only option is careful feeding of input.
TimW
QUOTE(caunt @ Mar 3 2005, 06:56 AM)
I wonder how much of a difference it would make to call the api yourself vs run the executable. 

You said there was trouble starting/stopping and aligning bookmarks.
Can you set bookmarks to a particular line of visual text?  I thought the bookmark was just for the visual screen.  At any rate, I see no code available for "stopping", so it appears our only option is careful feeding of input.
*


There is a certain amount of initialisation time required each time you start a program which would only be incurred once if you used flite as a shared library instead. I'm not an expert in these things but loading the app into memory, (re)locating function entry points (of main, at least) are the obvious things that spring to mind.

A bookmark is (in essence) applied to the first character that is on the screen. In theory you could apply it to any letter on the screen you want but the gui/user interation is considerably simplified if you don't have to specify exactly where on the screen you want to apply your bookmark. If you change formatting options, text size or whatever and go to the bookmark then you'll see that the only thing which is really in common is the first displayed character (give or take a few formatting marks or other stuff which is invisible in one view or the other).

As for careful feeding of text, there was just no way it was possible given that flite would swallow huge amounts of text in an instant but then take several minutes to say the text out loud with absolutely no feedback to the calling app. Anyway, calling flite for each separate paragraph seems to work quite well. I'll add a separate post below so that people don't have to read all this to notice I have something working.
TimW
I've got something working using the "invoke flite for each paragraph" approach. It works surprisingly well but I was only just able to run it on my SL5000D. OTOH, I do use very highly compressed documents which require several megabytes to decompress and I managed to bork my root partition by installing more apps to it than there was space for them so it may be that you won't get memory overload if you install it right. It should work okay on any of the production Zaurii though (I think - I obviously can't guarantee it since I don't have any of them).

I also did some tweaks to reduce the memory usage compared to my first attempt but I'd already borked my zaurus by overfilling the root partition at that point (I usually use OZ so I haven't tried too hard to fix it yet). I checked it out on the desktop so I believe it works okay. The code to do it is actually pretty trivial once you decide that you don't mind the overhead of invoking flite separately for each paragraph.

If anybody has a sharp based rom and wants to try it e-mail me at timwentford at hotmail dot com.

I'll be fixing my sharp based zaurus and retrying it and then I'll have a go at opie but as the code is so simple it should be in all new opie-reader releases (though as I don't have a proper compiler for the opie rom it may be a while before it gets into the opie CVS/feeds).
padishah_emperor
QUOTE(TimW @ Mar 3 2005, 12:55 PM)
I've got something working using the "invoke flite for each paragraph" approach. It works surprisingly well but I was only just able to run it on my SL5000D. OTOH, I do use very highly compressed documents which require several


Tim, PM me or send me an email if you want me to test it on Qtopia 1.7 on a Yopy with flite.
-Francis-
TimW
I've fixed everything and it all works, now (actually it was already working. All the problems I saw were down to my SD card corruption). The only other minor glitch is that flite doesn't like the "fi", "fl" etc ligatures so you have to turn "kerning" off (I got the name wrong when I wrote the dialog). If you don't you only get the paragraph read out as far as the first occurrence of such a ligature (this may be a problem with the way QT converts QStrings to char* rather than with flite but the net effect is the same - turn kerning off to make it work properly).

I've even gone one step further and I can now use the flite shared libraries directly but you need a version of flite compiled as shared libraries. I have an ipk for it but insufficient space/bandwidth to host it, plus I can't build for opie (ironically) ATM because I can't get a version of the build tools which supports softfloat (and don't tell me about OE - I have a modem connection to the internet with a cut off time far too short to get all the necessary atomic downloads done).

I've added a small architectural change to opie-reader so that it can load small plug-ins for output which means it supports either using the shared objects, or calling the pre-built binary via runtime configuration so even without the special build of flite you can still use tts in opie-reader (which means that I can do something for opie users providing they steer clear of the one floating point operation in opie-reader - using the "jump" dialog to jump to a %age location in the file rather than an absolute one).

As always, anyone who wants to try it, send me an e-mail. I may put out a proper release soon but the table support is a long way from being finalised so it'll be an unstable release (I don't think I've ever done anything other than unstable releases 8^) - its all relative).
lardman
QUOTE
I've even gone one step further and I can now use the flite shared libraries directly but you need a version of flite compiled as shared libraries. I have an ipk for it but insufficient space/bandwidth to host it, plus I can't build for opie (ironically) ATM because I can't get a version of the build tools which supports softfloat (and don't tell me about OE - I have a modem connection to the internet with a cut off time far too short to get all the necessary atomic downloads done).


Hi Tim,

Send me an email with the details plus any .bb/.oe files you have and I'll see what I can do this week to update the version in OE and send you an OZ binary too if that's what you're after.

I can also send you a (few?) CD(s) with the latest version of bb/oe/source code (I'll need a bit of time to download all of the source though), so that you have a working setup.

Cheers,


Si
caunt
I need some help with trying out the speaking version:

Tim has tried emailing me the archive twice now, and both times it has arrived corrupted...
but some of you have gotten it with no problems.
- Is there anyone who might be able to try sending me the package again
- or have in on a site where I can download it
- or even send me the pieces so I can package it into an ipk?
- any other ideas?

I really would like to try this out and Tim has already gone the extra mile.

Thanks
TimW
Sorry, I got a bit busy and it would take a while to sort out the individual files to put in a tar so haven't had a chance to do that yet.

However, I have managed to ftp a version of opie-reader with flite support to my website so you should be able to get it from this url.
ads
Tim...great work , you have just made opie reader twice as useful.

Just a hint for any having problems....flite needs to be in /home/QtPalmtop/bin/ and it needs to have the file name "flite" not "flite16k".

Also a bug ...you have to stop flite before exiting opiereader...otherwise flite will carry on reading and on restart only flite will be working (opiereader will be hidden in the background?)

Anyway thanks again Tim
caunt
OUTSTANDING!

Thanks Tim, this is great!

Thanks to the hint by ads about renaming flite. smile.gif

I'm still learning how to control all this...first I thought I had a suggestion or two for Tim, but then I found he had already implemented what was needed...I just hadn't found it yet!

Sorry I didn't post here sooner Tim (you had to email me sad.gif ). Actually I thought I had posted to tell you the link you posted had just what I needed, but didn't check back to see if it was successful.

EDIT:
Now I DO have a suggestion. Paragraph-at-a-time seems to lend itself to freezing - or at least seeming to. Have you tried sentence-at-a-time? At least the interface would be better able to respond to the user's wishes to start and stop it would seem.
xjqian
I don't know how to use the i version with flite. seems i'm missing some of the basics here, could anybody teach me how?
caunt
xjqian

- make sure you've followed the advice of ads above:
flite needs to be in /home/QtPalmtop/bin/ and it needs to have the file name "flite" not "flite16k".

- in the reader, in the settings, on the MISC tab of the configuration dialog:
-go to the "scroll" section
- under "scroll step" , choose "send to codec"
-the box underneath that should be "flitecmd"

to read, you "scroll"


That said, I keep running into a few glitches myself.
caunt
Tim,

I really would like to see the operation go to sentence-by-sentence now that I've been playing with it some.
I guess I read text with long paragraphs, but I've been maxing out the available memory (not that I have that much I guess).
The reader seems to work fine on short, choppy texts, but throw in a nice long paragraph and my Z chokes. sad.gif

Now THAT being said, I'm afraid I'm not much of a debugger/tester in this environment. I've been playing with C++ and QT for the first time this week, and I've gotten myself stuck already. [sigh]. I'll sleep on it.
xjqian
thanks caunt for the how-to. only wish it could pick up where it stops in the paragraph, apparent not possible with the current mechanism as discussed above. i guess i should take some time to read the whole thread again, a third time. I'm slow wink.gif
TimW
QUOTE(caunt @ Mar 27 2005, 04:50 AM)
I really would like to see the operation go to sentence-by-sentence now that I've been playing with it some.
I guess I read text with long paragraphs, but I've  been maxing out the available memory (not that I have that much I guess).
The reader seems to work fine on short, choppy texts, but throw in a nice long paragraph and my Z chokes.  sad.gif
*


I don't think that the length of the paragraph should have too much effect on the memory used so you must be pretty much at the limit already. I have a SL5000D with a Cacko ROM so my memory is pretty limited but I haven't had too much trouble (but I have made sure that *no* programs are fast loaded etc, and I have no programs installed to RAM).

It should be better using the shared library solution so I'll try and find a way of providing the ipk I created for that.

As for sending the text a sentence at a time rather than a paragraph at a time - the only reason I send a paragraph at a time is because I already had a function to read the next paragraph so, once I get the time, it shouldn't be a problem to change it to doing it a sentence at a time.
pframpton
Is there anychance you could post the source code, so that the opie cvs can be updated with this new feature.

Has anyone written a .bb for flite as a shared library?
TimW
QUOTE(pframpton @ Mar 30 2005, 10:29 AM)
Is there anychance you could post the source code, so that the opie cvs can be updated with this new feature.

Has anyone written a .bb for flite as a shared library?
*


Thanks to lardman I now have a working oe install so I'm currently updating my source so that it'll build properly with oe (previously someone had to move the sources around and rewrite the *.pro files every time I updated the code). I've got as far as updating what is already in CVS up to my current unstable and I'm now starting on adding the new stuff (REB support, bug fixes to html/chm/mobipocket, and the flite support). It shouldn't take long so I expect that it'll be in an "upgrade" feed pretty soon. I'm still behind a firewall so I will have to "post the code" but at least someone else getting it into CVS for me should be a "no-brainer" this time around (relatively, at least).

There is a bb file for flite as a shared library but it didn't build for me when I tried it. I haven't tried too hard yet, though.
lardman
flite built fine for me when I tried it - see the download date on the source I sent you for the exact date.


Si
TimW
I think it may be to do with the fact I'm building for Collie, or to do with it picking up some of my local configuration because it seems to be confused about whether or not it is building for OSS or ALSA. It defines something which is clearly supposed to be for an ALSA build but fails in building the OSS interface. I had the same thing when I compiled it by hand and had to fiddle with some #defines in the header files which ./configure hadn't set right.
TimW
I've tried passing a sentence at a time to flite and it doesn't work very well. If you listen to the way that flite says a paragraph, the first few words have a different emphasis to give an audible cue that there is a new paragraph. Having this cue applied to every sentence sounds terrible.

BTW, I managed to compile flite for OZ by manually editing the flite-1.2-release/config/config file to replace ALSA by LINUX (which enables OSS rather than ALSA). There is a setting in the bb file which tells config to do this but it doesn't work properly for me. I can now prepare an ipk for opie which will use either call the library direct, or exec flite to do the dirty work (ie the same as people have been testing on the SharpROM). There is a feature freeze in the opie CVS ATM but I can send an ipk to anyone who wants to try it. Unfortunately the flite ipks are too big for me to do this so you need to be able to build them yourselves. Also, there is some doubt (in my mind) about how to invoke flite on the ipaq vs the zaurus under OZ so initially I'll probably only send out one of each, so let me know which device you are using. Once it is confirmed as working I'll send out some more.
phunkhy
I would like to port Opie-Reader 0.7i to pdaXrom to benefit from the flite support. Who can give me any hint how to do that. I would like to have a tarball which contents can compile. I
simply don't find the sources in the opie CVS.

A nice flite ipk source I found:
http://www.unpleasant.net/chris/zaurus/pda....2_armv5tel.ipk
(if you have troubles with flite, like 'can't open /dev/dsp' then just do 'killall esd')

cheers
TimW
QUOTE(phunkhy @ Jun 10 2005, 05:11 PM)
I would like to port Opie-Reader 0.7i to pdaXrom to benefit from the flite support. Who can give me any hint how to do that. I would like to have a tarball which contents can compile. I
simply don't find the sources in the opie CVS.

A nice flite ipk source I found:
http://www.unpleasant.net/chris/zaurus/pda....2_armv5tel.ipk
(if you have troubles with flite, like 'can't open /dev/dsp' then just do 'killall esd')

cheers
*

The opie CVS source is here. To compile for X11 make sure that USEQPE is undefined and put all the shared stuff (like icons etc) into a directory referenced by the environment variable READERDIR (look at the source - search for READERDIR - to see the sub-directories used).
ken
TimW:

The flite homepage is here:

http://www.speech.cs.cmu.edu/flite/

It includes the source, etc

If it's festival you're interested in, it's here:

http://fife.speech.cs.cmu.edu/festival/
malik
for those who didnt lost their interest in tts systems on the zaurus.
i was suprised that the below setup worked

1) an older arm port of qemu + linux binary of the german tts system txt2pho
2) an arm port of mbrola
3) mplayer to play the generated wav files

its hard to compare the results for german and english speech, but i would
say that the mbrola system is better than flite but not as good as
the ibm plugin for opera.

for a text file consisting of one paragraph with around 4 lines (a 80)
step 1) took around 40 sec and step 2) around 12 sec. the generated wav file
was around 16 sec.

conclusion: the above setup doesnt seem to be really usable on the zaurus.

malik
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2014 Invision Power Services, Inc.