OESF Portables Forum

Everything Else => General Support and Discussion => Zaurus General Forums => Archived Forums => Software => Topic started by: jfr on February 11, 2007, 10:29:46 pm

Title: Morse Code Input
Post by: jfr on February 11, 2007, 10:29:46 pm
I'm looking into the possibility of writing an app for the Z that would take Morse Code input, and speak the resulting text.

The beneficiary would be a 92-year-old friend who has been a life-long radio ham, and who through a series of strokes has lost the power of speech.

Ideally, it'd be possible to connect a real Morse key to the Z by some means, so that he could use his own morse key.

I've found links to this sort of stuff via google, but unfortunately the links are dead. There are references to a sourceforge project called zmorsed, and I've seen a reference (without a link) to "How to connect a real Morse key via the audio connector". That reference said it was possible, but didn't go beyond that. The sourceforge links all lead to an "Access Denied" response. (Yes, I was logged in at the time.)

My plan would be to use flite for the speech synthesis, driven by the program that I'd write. I've proved out the speech synthesis part, but it's the Morse Code input that's lacking.

I'd be really grateful if anyone could help out here. For starters, does anyone have the zmorsed source?
Or any other code for interpreting morse code? Ideas for connecting the morse key?

Cheers

John
Title: Morse Code Input
Post by: InSearchOf on February 11, 2007, 10:57:26 pm
http://morseall.org/ (http://morseall.org/)
I found this...

Late
Title: Morse Code Input
Post by: Da_Blitz on February 11, 2007, 11:22:29 pm
what model Z were you thinking of using? if its one with usb hostthen get a usb printer adaptor and butcher it (sorry thats "creative restructuring") into somthing smaller that clips onto the Z gives you 8 bits of IO

attaching it to the audio port would be good. not that hard to do ethier. the remote control for the Z (inline headphone jack) uses this principle with a voltage divider. for you it would depend on how the keyer works

your distro of choice will make a huge diffrence. if you can get one with the uinput or user space input device driver, that way you can insert keycommands as a keyboard instead of having to do some hackery. means it will work on X and the console as well

here are some options
http://sourceforge.net/projects/cwtext/ (http://sourceforge.net/projects/cwtext/) takes asscii and outputs morse code as asscii or sound (8bit wav) not exacttly what you are looking for however if you were building an interface around morse code then this part could help, in conjunction with a shell you wrote
http://users.skynet.be/ppc/xcw/ (http://users.skynet.be/ppc/xcw/) X11 morse code input daemon
http://www.qsl.net/pg4i/linux/cwdaemon.html (http://www.qsl.net/pg4i/linux/cwdaemon.html) listens on a udp port and spits out morse code when a packet arrives
http://morseall.org/ (http://morseall.org/) THIS IS WHAT YOU WANT. it has support for using a mouse (ie usb ) and using 2 or 3 buttons, one for tic the other for tac so you can achive faster rates

so those are some links that might be handy, for futre refrence i used freshmeat.net to look everything up however i did that knowing the morseall package exsisted

if you dont have a cxx00 for use in this project then i can show you how to hook up the keyer to the Z over the serial port, its harder and you will have to do some processing yourself to decode the results but nothing too hard, the hardest bit is how to wire it so it dosent blow the Z

just a side note is this so they can use the Z as a terminal as well (ie like the blind do) or is this a speach only thing. you imply loss of voice but if they are using a keyer then i am guessing they have limited control over thier fine moter skills. just thinking you could add some stuff to allow them to switch between using your app to talk and controlling the PC

anyway back on topic, if you do use a usb mouse then you have to tell X or whatever the display/input thing is to not use your mouse as an input, then commect to it directlly (/dev/input/mouse) this varies from distro to distro and IMHO your best option is most likly a stripped build of OZ with a 2.6 kernel. YMMV
Title: Morse Code Input
Post by: jfr on February 12, 2007, 04:38:37 pm
Hey, thanks folks! That's a lot of input - I'll check it out.

To answer a couple of Da Blitz's questions:
I was planning to get the thing working on my C3000, and then if it does work, look for a 2nd hand Z on ebay. So I don't yet know what the final target machine would be.
And the reason for the morse key isn't mobility, it's that as a radio ham he's an excellent morser (if that's a word...). That's also why hooking up a real morse key would be an advantage.

Cheers, and thanks

John
Title: Morse Code Input
Post by: Da_Blitz on February 13, 2007, 04:30:35 am
sounds good then, in that case i would recomend the audio port, more versitile and can be plugged into many machines then
Title: Morse Code Input
Post by: ztep on February 13, 2007, 05:02:11 am
I think that the the audio port is the best option for you. The remote hardware is well know, so if you tell as how the Morse key you want to use works (both electrical and mechanical) we can help you to design the apropiate interface between the audio port and the morse key.

One problem that I found is that if you want to use the audio port for interface the morse key you need an external speaker because the internal is silenced.
Title: Morse Code Input
Post by: Da_Blitz on February 14, 2007, 01:41:27 am
are you using a 2.4 or 2.6 kernel. i belive i had the speaker working as well as the headphones after using alsamixer on the cmdline (speaker=yes)

and i thoght that trick would never come in handy , if you do do it under OZ then you have to disable the audio routing script that runs everytime a headphone device is plugged in or removed

but im up to helping you with this project. i have always wantede to get a braile terminal and hook it up to my box. you never know when that may come in handy
Title: Morse Code Input
Post by: jfr on February 14, 2007, 10:39:56 pm
Quote
are you using a 2.4 or 2.6 kernel. i belive i had the speaker working as well as the headphones after using alsamixer on the cmdline (speaker=yes)

and i thoght that trick would never come in handy , if you do do it under OZ then you have to disable the audio routing script that runs everytime a headphone device is plugged in or removed

but im up to helping you with this project. i have always wantede to get a braile terminal and hook it up to my box. you never know when that may come in handy
[div align=\"right\"][a href=\"index.php?act=findpost&pid=154022\"][{POST_SNAPBACK}][/a][/div]

WARNING: Long reply!

Long, because of the offers of support. What a great community this is. I hope with this post to put you fully in the picture, as it currently stands.

Gosh - thanks folks for those messages of support. I'll surely need it if this project is to be completed.

To Da Blitz: For now, I'm experimenting on my C3000, which has the original ROM and kernel. It works pretty well, and I never quite summoned up the courage to go for the upgrades...

I should perhaps say at this point that I don't know whether this project will actually be a goer, because there are ifs and buts still to be resolved.

The main 'if' is that I don't know whether the intended recipient still has the power to do morse - if speech is affected, then maybe morse (also language-related) could be. Also, I need to check his motor skills.

But before that (because I don't want to raise hopes if I then can't deliver), I now want to put together a simple prototype which I can test on another morse-proficient friend. If that works, then I'll feel confident about approaching B (the intended recipient).

So for now, it's the prototype that I'm working on.

BACKGROUND

I looked at the morseall project, and I was highly impressed. I don't think I can use that code, though. It's designed for sip-and-puff morse, where dots and dashes are separate actions (as opposed to the same action for different durations), and it assumes pretty fixed timings. Morse from a regular key is I believe more variable than that.

I've found (through the links you all supplied, and others that I googled) a lot of material on Morse for the disabled. There's a lot on supplying Morse solutions for all sorts of access needs, but a lot of that assumes that the person using the system will have to learn Morse to start with. The difference in my case is that I'm trying to design a system for someone to whom Morse is already a first language, and at the moment I'm assuming he still has manual motor skills.

I found quite a few university research papers via Google (none with code, alas), but they describe the problems of reading conventional hand-produced morse. The algorithms they propose are adaptive - they continually adjust for changes in sending speeds - and they also attempt error correction on the fly. The error correction is of the nature: This isn't a legal Morse character, so let's see which character it's closest to, and assume that.

There are papers that propose neural nets as the solution. I don't think I'm quite ready for that...

PROGRESS

The speech synthesis part is the really easy bit. I downloaded flite, and it worked out of the box. So whatever program I come up with, I will be able to run flite from the program, with the text on the command line.

So at the moment, the plan is to write an app for my C3000 that will accept the Morse just by tapping on the screen, and then speak it. If I can do that, then I'll feel confident enough to approach B and ask whether I should go further.

Which means I'm currently busy writing code based on the algorithm descriptions in the research papers, for basic Morse recognition.

The project plan therefore looks something like this, and the whole project will fail if any step fails:

1. Demonstrate speech synthesis from the command line - DONE.
2. Check out recognition algorithms - DONE.
3. Produce prototype for beta-testing. Prototype will accept Morse input by tapping the screen with the stylus, and will speak the result. User interface will be abysmal. Morse recognition algorithm will be good, and that's the difficult bit.
4. Get friend (a retired ship's wireless operator who doesn't yet know that I've appointed him for this task) to perform beta test using screen-tapping interface.
5. Approach B to see whether he thinks he could use a more polished version of the prototype. Expectations are raised at this point, which is why the plan doesn't involve contacting B earlier.
6. Choose which Zaurus to deliver the solution on, and buy it on Ebay. Yeah, thanks Sharp for pulling the plug just as it was all going so nicely. But at the moment, I'm thinking that maybe a 5500 will be able to cope, and there are several on offer at affordable prices. B is a friend though, so if the project gets this far, price won't be a problem.
6. Develop the more polished version, including a hardware interface for his preferred Morse key. (This part will need to be fleshed out if the project gets that far.)
7. Deliver system.

For now, step 3 is current. It involves no new hardware and it will work if step 4 does.

This project plan has evolved thanks to the collective input from forum members. I think the ball's in my court at the moment, and I'll let you know how it goes. Meanwhile, ideas are still very, very welcome!

Cheers

John
Title: Morse Code Input
Post by: Da_Blitz on February 15, 2007, 04:41:12 am
nothing i own has its originol firmeware , actually i briced my router an hour ago so i need to fix it the old fasioned way with tftp , still works as a switch however. thats why i always have 2

decoding morse code dosent seem that hard, an adaptive algorithim wouldnt be too hard to write ethier, with or without funky maths

another option you have is to get a hp 1910 or 1930, there is a very mature port for that device and its a very small machine, i saw one on ebay for $4 the other day because its 3 to 4 generations old and was low speced when it came out. think about it. the other option is any other hp pda as the linux port is very good and nearlly identical to what we use

my version of the tit or tap algo is to look at the width of the current pulse and compare it to the last, if its longer its a dash if its shorter or within a set variation for timing then its a dot. basically you only check to see if its a dot, if it isnt AND teh pulse is longer than is acceptable for the dot then its a dash otherwise ERROR

of course the parameters for how close and acurete the pulse widths have to be is a diffrent matter but i would say that you stick to the aproch that thi timing gets smaller the more dots and dashes per second (normailsed of course as they take diffrent ammounts of time) the more you tighten up the margin for errer, so that when they type slow there is less chance of a mistake and when they speed up it compinsates for that

hardware wise you are looking at a headphone jack, a 500mW speaker andat least one microswitch. along with the usual heatshrink and wire
Title: Morse Code Input
Post by: jfr on February 17, 2007, 06:48:23 pm
Da Blitz - thanks again for the ideas. They're all being squirrelled away...

ztep - thanks to you for the offer of help connecting a Morse key. I've no idea yet what key it might be, but if the project gets that far, I'd like to come back to you! I certainly don't think I could manage it on my own, because my hardware knowledge of the Z is, well, not exactly impressive...

I do now have some progress. On the desktop machine I now have a little system - a mixture of Tcl and C++ - in which I can input Morse via the left mouse button, and get it spoken. Given that I'm a Morse veteran of just a couple of days, it seems encouraging that it can understand my input :-)

Next comes the port to the Z, with the primitive GUI design. This may take a while, but once it's done I can try the beta test.

Cheers

John
Title: Morse Code Input
Post by: speculatrix on February 17, 2007, 06:56:29 pm
could you use a usb numeric keypad and use that as a simple control pad for your friend? he could use up/down/left/right to select letters and words which the zaurus could speak.

if you need something more custome, get a usb to parallel port converter and provide a very simple control panel that way?
Title: Morse Code Input
Post by: jfr on February 18, 2007, 11:26:48 am
Quote
could you use a usb numeric keypad and use that as a simple control pad for your friend? he could use up/down/left/right to select letters and words which the zaurus could speak.

if you need something more custome, get a usb to parallel port converter and provide a very simple control panel that way?
[div align=\"right\"][a href=\"index.php?act=findpost&pid=154553\"][{POST_SNAPBACK}][/a][/div]

Now, that's an idea. I still want this to be a Morse device because of his life-long familiarity with Morse, but maybe it'd be possible to open up a numeric keypad and connect a Morse key to one of the buttons. It'd solve the interface problem.

The only drawback might be the timing information. A numeric keypad probably wouldn't scan its keys all that often, in which case the sending rate would be restricted.

The parallel port sounds good too - if there's a suitable driver for it. I'm afraid that's not one of my good areas. Anything that can be made to generate a Qt event ought to do the trick.

Added to the ideas list. Thanks!
Title: Morse Code Input
Post by: speculatrix on February 18, 2007, 04:50:36 pm
this is a bit of a long shot, but for quite a while I've been musing about writing a simple zaurus game (qtopia based for sharp/cacko) and made a start before I started reading this thread.

I suddenly realised this morning that I could take what I started, and cut it down a bit and give you a really simple application for the Z which is simply four buttons on the screen, you could then hack in the code to do the morse or whatever... i.e. it's a get-you-started skeleton application. Hmmm. Let me attach the bare-bones program I wrote and you can see what you think.

No prizes for guessing what game I trying to write!

save this  [ You are not allowed to view attachments ]  and rename, just drop the .zip extension, it's a linux/arm binary, run it on the Z.
Title: Morse Code Input
Post by: jfr on February 18, 2007, 07:37:58 pm
Quote
this is a bit of a long shot, but for quite a while I've been musing about writing a simple zaurus game (qtopia based for sharp/cacko) and made a start before I started reading this thread.

I suddenly realised this morning that I could take what I started, and cut it down a bit and give you a really simple application for the Z which is simply four buttons on the screen, you could then hack in the code to do the morse or whatever... i.e. it's a get-you-started skeleton application. Hmmm. Let me attach the bare-bones program I wrote and you can see what you think.

No prizes for guessing what game I trying to write!

save this  [ You are not allowed to view attachments ]  and rename, just drop the .zip extension, it's a linux/arm binary, run it on the Z.
[div align=\"right\"][a href=\"index.php?act=findpost&pid=154610\"][{POST_SNAPBACK}][/a][/div]

Hey, thanks for that kindness! It's much appreciated, and the program does run, even though it complains about missing files:
/home/QtPalmtop/i18n/en/libsl.qmid
/home/QtPalmtop/lib/libqsfepj.so, libkke.so.1

I'm not so sure about using it yet, though that's mainly because I'm already using one of my other apps as a base. But I'd like to keep it in reserve, because my GUI design in Qt is no more sophisticated than just painting stuff directly to a pixmap and then updating the screen from it.


BUT: In the meantime, I seem to have hit a problem that may be severe. I was merrily coding away, assuming without thinking about it that a Qt event would have a timestamp field, telling me when the event actually occurred to some reasonable level of precision. I'm now dismayed to find that they don't. So if my app gets interrupted during Morse input, and events are forced to stay in the queue for a while, I'll have no means of getting back the original timings. And in Morse, those timings are quite small and obviously critical. Even if the app doesn't get interrupted, updating the screen to print what's being received looks as though it could cause trouble.

I guess I'm just going to have to carry on and see how it goes in practice, but any words of wisdom or comfort would be appreciated. Like, is there something obvious that I've failed to see, here?
Title: Morse Code Input
Post by: Capn_Fish on February 18, 2007, 08:31:33 pm
I was browsing Hackaday, and found this:

http://home.att.net/~jacksonharbor/palm.htm (http://home.att.net/~jacksonharbor/palm.htm)

It seems to be somewhat related, so I thought I'd post it.
Title: Morse Code Input
Post by: speculatrix on February 19, 2007, 05:36:59 am
Quote
I was browsing Hackaday, and found this:

http://home.att.net/~jacksonharbor/palm.htm (http://home.att.net/~jacksonharbor/palm.htm)

It seems to be somewhat related, so I thought I'd post it.
[div align=\"right\"][a href=\"index.php?act=findpost&pid=154624\"][{POST_SNAPBACK}][/a][/div]

actually, a Palm might be a good/cheap morse man/machine interface, if you can find the right program for it. there are text-to-speech apps for it but I don't recall any being free.
Title: Morse Code Input
Post by: speculatrix on February 19, 2007, 06:01:36 am
Quote
BUT: In the meantime, I seem to have hit a problem that may be severe. I was merrily coding away, assuming without thinking about it that a Qt event would have a timestamp field, telling me when the event actually occurred to some reasonable level of precision. I'm now dismayed to find that they don't. So if my app gets interrupted during Morse input, and events are forced to stay in the queue for a while, I'll have no means of getting back the original timings. And in Morse, those timings are quite small and obviously critical. Even if the app doesn't get interrupted, updating the screen to print what's being received looks as though it could cause trouble.

I guess I'm just going to have to carry on and see how it goes in practice, but any words of wisdom or comfort would be appreciated. Like, is there something obvious that I've failed to see, here?
[div align=\"right\"][{POST_SNAPBACK}][/a][/div] (http://index.php?act=findpost&pid=154619\")

I think if you want real-time execution, you're going to want to use SDL as a audio/video toolkit; I recall posting a link to a devshed article on getting started with SDL a month or so back. Ah yes, here it is:
[a href=\"http://www.devshed.com/c/a/Multimedia/Game-Programming-using-SDL-Getting-Started/]http://www.devshed.com/c/a/Multimedia/Game...etting-Started/[/url]
Title: Morse Code Input
Post by: jfr on February 19, 2007, 01:57:41 pm
Thanks, Speculatrix - this project is proving to be quite an education!

Here's an update to the plan. A colleague has found a device called an MFJ-461, which is a small box that takes Morse input as sound, and generates serial output. I think that's my new favourite, because it handles the timings and, together with a serial-USB converter, it should solve the difficult part of the problem. I'm now investigating to see whether there's a way of connecting a key directly to it.

<a href="http://www.mfjenterprises.com/products.php?prodid=MFJ-461">
http://www.mfjenterprises.com/products.php...J-461</a> (http://www.mfjenterprises.com/products.php?prodid=MFJ-461</a>)
Title: Morse Code Input
Post by: speculatrix on March 05, 2007, 07:34:03 am
did you buy one of these boxes?

don't forget that the C3000,3100 and 3200 don't have a serial port, you'd need to use a USB dongle.
Title: Morse Code Input
Post by: jfr on March 05, 2007, 03:04:39 pm
Quote
did you buy one of these boxes?

don't forget that the C3000,3100 and 3200 don't have a serial port, you'd need to use a USB dongle.
[div align=\"right\"][a href=\"index.php?act=findpost&pid=155398\"][{POST_SNAPBACK}][/a][/div]
Yes, I did buy one, and it's sitting here in front of me now. It expects tone input, but I've connected a Morse key to the output of the tone detector, and it displays characters as I clumsily key them in.

The Z serial port is the big issue now. I really thought people had decided it did work, and experiment seems partly to bear that out. I bought a Sharp CE-170TS serial cable from Trisoft, and wired it to the morse decoder, but can't get the Z to admit that it's seeing anything. However, if I look at pin 2 of the Z's serial cable (TXD) with a voltmeter, I see zero volts normally, but two to three volts when I do
cat <some-file> > /dev/ttyS0
(This is on a 3100 by the way).
So that suggests to me that there's something there. And stty seems to work: I set the speed to 19200 as the morse box requires, and it kept the setting. I've also been through all the settings that stty reports, and there's nothing I can see that should prevent the thing from working. The stty settings include clocal (ignore modem control signals), but just in case, I've wired CTS to RTS and DSR to DTR on the Z end. Each of those lines shows a positive signal level. The morse box has no control connections: just TXD and RXD (which it ignores).
Maybe I'll have to admit defeat on this, but it's tantalizing that I can get output yet not input.

I did find through Google reports that the serial port input on the 5500 was tied to a pppd listener, but that's not the case on the C3xxx range, according to /etc/inittab.

Any ideas welcome - otherwise I guess a USB serial cable is next on the shopping list.

Cheers, and thanks again for the interest.
Title: Morse Code Input
Post by: speculatrix on March 05, 2007, 04:25:23 pm
try installing "minicom" from a feed, it'll allow you to drive the serial port with no handshaking.

--- edit ---

p.s. I remember now, there's evidence that the serial port on the xy00 IS functional and connected, just that the drivers are quite weak, and only a genuine Sharp serial cable will work... there's discussions about it in the 1000/3000/3100/3200 specific forum
Title: Morse Code Input
Post by: jfr on March 05, 2007, 05:58:18 pm
Quote
try installing "minicom" from a feed, it'll allow you to drive the serial port with no handshaking.

--- edit ---

p.s. I remember now, there's evidence that the serial port on the xy00 IS functional and connected, just that the drivers are quite weak, and only a genuine Sharp serial cable will work... there's discussions about it in the 1000/3000/3100/3200 specific forum
[div align=\"right\"][{POST_SNAPBACK}][/a][/div] (http://index.php?act=findpost&pid=155451\")
Hmm. Just tried that, but the minicom ipk from the ZSI depends on other packages including libc6. I got libc6_6.1_arm.ipk from [a href=\"http://www.katastrophos.net/zaurus/packages/]http://www.katastrophos.net/zaurus/packages/[/url] but:
> ipkg install libc6_6.1_arm.ipk
Unpacking libc6..Done
Configuring libc6..dpkg: invalid option -- -
BusyBox v0.00.4 (2003.04.15-01:23+0000) multi-call binary

I can unpick the package to see whether I can fix it, but I thought I'd ask first whether a statically linked version of minicom was available anywhere.

AHH! WAIT!!!

Dont't know why I thought to do this, but I just tried
> sh < /dev/ttyS0
and now the transmission's getting through!

And a little C program I wrote is suddenly responding too.

I hadn't changed any settings, and I hadn't run minicom. And the minicom package has no preinst etc files, so merely installing it couldn't have made a difference.. Could I just have jogged a poor connection somewhere?

Whatever it was, I'm sure it wouldn't have been fixed if I hadn't been pursuing your suggestion.  So THANKS! even if we don't know why...

Now back to the coding...
Title: Morse Code Input
Post by: speculatrix on March 05, 2007, 06:13:10 pm
hmm,  I would have installed minicom from a cacko feed, but never mind, you got it working, well done!
Title: Morse Code Input
Post by: jfr on April 27, 2007, 09:11:42 pm
Please pardon me for bumping my own thread, but I hope this update might be helpful to someone who might find it in the archive one day - and it also gives me the chance to say a big THANK YOU to all those who have contributed their ideas.

The project is now at the stage where I have a prototype that works as well as I can make it work, and I'm waiting for a chance to let B have a go at it.

The project has evolved a bit since the earlier postings.

The big issue was always the question of how to interface the Morse key to the Z. At a sending speed of 20 words per minute, a dot lasts for 60 milliseconds. That's quite a short time, and the time measurement is obviously critical for correct decoding. Any form of direct hardware interface would have to record event times to much better than that accuracy. When I realized that Qt didn't inlcude the original event timing in the event data structure, I decided I needed something more precise. If the event doesn't record the time it happened, the best you can do is to record the time when it came *out* of the event queue. It seemed a bit too much to hope for the needed accuracy that way.

As previously mentioned, I tried the MFJ-461 Morse decoder. It's a neat little box that's designed to be held up to a radio receiver loudspeaker, or connected directly to the audio output jack. Kudos to the manufacturer here: it expects an audio signal, not a direct connection of a Morse key. After looking at the circuit diagram, I decided where to connect the key, and emailed them asking for any advice. Their reply was prompt and helpful and as a result they are in ny good books - but in the end, I had to conclude that the decoder was better suited to the interpretation of automated Morse than to hand-generated.

So I introduced a new element - a PIC development board that was surplus to requirements at $DAYJOB, and which they were happy to donate.

The PIC board now has the Morse key connected to it, and I wrote a program for it which debounces the key - a very necessary step, as it turned out - and sends to its serial port a series of ASCII characters that encode just the timings between key-up and key-down events. That's all the PIC does - it does the timing measurement of 'up' and 'down' events, and it knows nothing about Morse code.

The serial output from the PIC then goes via a 3-way cable to the Z, and also to my desktop machine. The Z is running a program which listens to the serial input, and decodes the timings to generate the ASCII characters corresponding to the Morse. Every time it sees a full stop, it speaks (via flite) what it's collected. My friend who is the retired ship's radio operator (that is, my guinea-pig) is able to use it and to speak long  and complicated sentences.

The desktop machine receives the same input, and also speaks it in the same way. However, it adds value in that it displays an oscilloscope-like trace of the Morse input - the idea of which is that I can tune the decoder to the hand of the operator, if necessary. The desktop machine is just a debugging aid - not part of the final system.

I've learned lots from this project, and it isn't over yet. Perhaps the biggest lesson, though, is this:
1. It's not too hard to decode Morse, if you don't mind a few errors. The human brain is very good at interpreting text where some characters are in error. I've now seen quite a bit of software that does decode Morse, but "a few errors" are to be expected.
2.. But if you want to feed the result to a speech synthesizer, those few errors become intolerable and can wreck the output. The speech synthesizer isn't intelligent, and it needs very clean data. That's the hard bit.

Cheers

John
Title: Morse Code Input
Post by: Da_Blitz on April 28, 2007, 06:42:01 am
just out of intreset which PIC did you end up using

i have been intrested in alternative input/output systems for years now and this have given me some thoughts (been trying to track down a cheap braille screen)

while i am at it (and to increse its historical significance) why chose the PIC?, i still belive that doing it all on the Z through the audio port would have been the best option, that way you would get the timing info you need as well as easy software/hardware upgrades (say you wanted a 2 paddle system)

thats intresting with the speach synthasiser, i remeber a long time ago playing with the speach stuff on the macs and found that incorect spellings and stuff that was phonetically correct gave the best output

perhaps a "tweaker" script between the program and the speach software is the way to go, i guess simple substitution would be best but is it worth the effort?
Title: Morse Code Input
Post by: speculatrix on April 28, 2007, 05:00:49 pm
fascinating project, am really impressed. I have a PIC programmer for some of the older PICs if you need some programming, I'd have to check out which ones it will do.

could you add a new character to the morse sequence to represent "rubout/delete"? that way if B sees there's an error, B could correct it.

some sort of automated spell checker/fixer might help with the text-to-speech.

I was wondering whether you could use the IR port on the zaurus to do away with the cable? Adding an IR transmitter to the morse key part wouldn't be too hard. It would also allow it to be used with other Z's which don't have a hardware serial port (the 1xxx or 3xxx for example).
Title: Morse Code Input
Post by: jfr on April 28, 2007, 06:42:32 pm
Quote
just out of intreset which PIC did you end up using

The PIC16F84A. The reason for choosing that particular model was purely that $DAYJOB had a pile of old development/programming boards for them, which I was able to scrounge - along with an old Windows laptop, needed to run the programming software.

Quote
i have been intrested in alternative input/output systems for years now and this have given me some thoughts (been trying to track down a cheap braille screen)

while i am at it (and to increse its historical significance) why chose the PIC?, i still belive that doing it all on the Z through the audio port would have been the best option, that way you would get the timing info you need as well as easy software/hardware upgrades (say you wanted a 2 paddle system)

I can't argue against that, except to say that I was nervous about things like operating system latency affecting the measured timings. Presumably I'd have had to build a tone generator to attach to the key, so there would have been external hardware of some sort anyway - though of course the tone generator would have been a lot simpler. I didn't know how long the Z might suspend my program while the kernel did stuff, and I wanted to eliminate that potential problem. But I never experimented to find out whether the problem was really there. To do the timing on the Z, you'd need to be confident that your userland program would be able to measure the timings of events with 10 to 20mS accuracy - preferably 10mS or better. I lacked that confidence, but I'd be very interested if anyone could prove me wrong.

Quote
thats intresting with the speach synthasiser, i remeber a long time ago playing with the speach stuff on the macs and found that incorect spellings and stuff that was phonetically correct gave the best output

perhaps a "tweaker" script between the program and the speach software is the way to go, i guess simple substitution would be best but is it worth the effort?
[div align=\"right\"][a href=\"index.php?act=findpost&pid=160012\"][{POST_SNAPBACK}][/a][/div]

I agree about phonetic spellings - they tend to be shorter, too. Not sure what you mean by the tweaker program, though?
Title: Morse Code Input
Post by: jfr on April 28, 2007, 07:36:04 pm
Quote
fascinating project, am really impressed. I have a PIC programmer for some of the older PICs if you need some programming, I'd have to check out which ones it will do.

could you add a new character to the morse sequence to represent "rubout/delete"? that way if B sees there's an error, B could correct it.

some sort of automated spell checker/fixer might help with the text-to-speech.

I was wondering whether you could use the IR port on the zaurus to do away with the cable? Adding an IR transmitter to the morse key part wouldn't be too hard. It would also allow it to be used with other Z's which don't have a hardware serial port (the 1xxx or 3xxx for example).
[div align=\"right\"][a href=\"index.php?act=findpost&pid=160032\"][{POST_SNAPBACK}][/a][/div]

Thanks!

Especially for the kind programming offer, though the development boards I have are enough for my needs in that department so you can relax on that one :-).

Rubout/Delete: it's already defined in the Morse code - 6 or more dots as a single character, meaning "delete last word". The program recognizes it. The human interface problem (in my very limited tests) is that a trained Morse operator gets distracted by having to read the text while keying the input, presumably because of the time lag - to key the Morse, you're thinking ahead, whereas reading and checking the text implies a parallel brain process running in a different time zone, even if they're only a fraction of a second apart. So although I've implemented it, I've found that in practice it's less useful than I'd expected.

A spell-checker sounds very attractive. I have to admit, here, to a personal bias against them - if they're allowed to make corrections, then in my experience they get it wrong more often than they get it right. And for this sort of project, there would be little point in a spell-checker that didn't do its stuff on the fly (correcting without asking first). [begin rant] Mind you, my experience with spell-checkers is mainly limited to $DAYJOB's insistence on using Word to create exam papers. Word's ability to corrupt a perfectly well-formed exam question - without alerting anybody to the fact that it's done so - is the eighth wonder of the world. [end rant]

There's a conflict, too, between the use of a spell-checker and the use of phonetic spelling. I think that on balance, I'd rather allow phonetic spelling, which a spell-checker would almost certainly mis-correct. It'd be an interesting research question.

IR port - hadn't occurred to me, and it's a great idea. The current prototype has an embarrassing rat's nest of cables. The only downside is that you'll be troubled with a post that says "Ok, so how do I use the IR stuff?". Let's not worry about that yet - we're still waiting for B to try out the prototype, so it's a bit early to refine it for this project - but it's now in the forum archive as a suggestion.
Title: Morse Code Input
Post by: Da_Blitz on April 28, 2007, 08:48:36 pm
the teweaker programe just changes spelling in some workds before its fed to flite to make the output easier to understand, i dont think its worth the effort of a dev, but if you could push its setup onto the user it might be worth it

ahh, the good old 16f84. a very nice chip. bty if you ever think of moving this to a cxx00 then you may want to consider a 18 series with usb built in. i know usb programming may not be your thing but there are some datasheets on how to make a keyboard out of the thing that could be adapted quite easily

dont forget Ir likes a 40Khz carrier, i had to do this once with a 16F877 and the best way was to use PWM hardware and another IO pin hooked together with a tranistor to get the on off keying (OOK), however with the 16f84 it looks lixe you may have to bit bang it (25 instructions per half cycle @ 4Mhz)

ps i may have stuffed up the above, cant remeber if its 40Khz (home remotes) or 455Khz

as for auido timing, its my understanding that you ask the alsa libs for X samples at a time, say you ask for 100 then you get 100 16bit sapmles at a time each representing 1/48000 fo a second

i dont think you wolud have to generate tones because the remote uses DC values on the line. so you should be able to get a way with a R2R bridge if you wanted more than one button otherwise a simple on/off should do
Title: Morse Code Input
Post by: Armagon on April 30, 2007, 03:48:37 pm
Quote
To do the timing on the Z, you'd need to be confident that your userland program would be able to measure the timings of events with 10 to 20mS accuracy - preferably 10mS or better. I lacked that confidence, but I'd be very interested if anyone could prove me wrong.
[div align=\"right\"][a href=\"index.php?act=findpost&pid=160035\"][{POST_SNAPBACK}][/a][/div]

10 mS is a long, long time to the computer.  I make video games for a living, and we try to get 60 frames per second.  Every 16.7 mS, the games I work on need to do physics calculations, transform and draw thousands of polygons, process artificial intelligence, and so on.  Granted, the hardware we develop for is designed for this sort of thing, but you can do a whole heck of a lot in twenty milliseconds of computational time.

I would be really surprized if multitasking on the Z would skew your timings too much.  I expect you could use a standard message pump and take for the time stamp the time that you get the callback to process the event, and that the amount of time that this would be off -- probably on the order of microseconds -- would be far less than the amount of variation time your program has to account for when looking at human-provided input.  

[Hmm... It appears that a single task may get up to ten milliseconds in one go, on the high end.  That computing quantum is higher than I thought, and may actually cause you a problem.  I don't know what the actual time is for the Z.  It may be, though, that multiple timings are skewed by the same amount, and, especially since your task will be the only one running, I don't think it'll be an issue.]

Armagon
Title: Morse Code Input
Post by: jfr on May 01, 2007, 12:26:22 am
Quote
Quote
To do the timing on the Z, you'd need to be confident that your userland program would be able to measure the timings of events with 10 to 20mS accuracy - preferably 10mS or better. I lacked that confidence, but I'd be very interested if anyone could prove me wrong.
[div align=\"right\"][a href=\"index.php?act=findpost&pid=160035\"][{POST_SNAPBACK}][/a][/div]

10 mS is a long, long time to the computer.  I make video games for a living, and we try to get 60 frames per second.  Every 16.7 mS, the games I work on need to do physics calculations, transform and draw thousands of polygons, process artificial intelligence, and so on.  Granted, the hardware we develop for is designed for this sort of thing, but you can do a whole heck of a lot in twenty milliseconds of computational time.

I would be really surprized if multitasking on the Z would skew your timings too much.  I expect you could use a standard message pump and take for the time stamp the time that you get the callback to process the event, and that the amount of time that this would be off -- probably on the order of microseconds -- would be far less than the amount of variation time your program has to account for when looking at human-provided input.  

[Hmm... It appears that a single task may get up to ten milliseconds in one go, on the high end.  That computing quantum is higher than I thought, and may actually cause you a problem.  I don't know what the actual time is for the Z.  It may be, though, that multiple timings are skewed by the same amount, and, especially since your task will be the only one running, I don't think it'll be an issue.]

Armagon
[div align=\"right\"][a href=\"index.php?act=findpost&pid=160140\"][{POST_SNAPBACK}][/a][/div]


I think you're very likely to be right - and for the archive, I agree with your point.

I note your "Hmm" though. I was trying to avoid getting involved with issues I couldn't measure, which is why I didn't want to go down that road. I don't have a problem with the amount the machine can do in a stated period of time, but I did worry about how long my program could potentially be backgrounded,

In my particular case:

Given that I'm developing this for one single individual, and given his particular situation, I'm not in the position where I can carry out experiments. Either it works first time when he's presented with it, or it's dead in the water. I feel more confident that my (let's say, in general terms unnessarily elaborate) solution will work out of the box, because I don't know how long the Z could take control away from my program, and I think I've evolved a solution where it isn't necessary to know that.

There's still the other issue: debouncing. A typical Morse key has brass contacts, and when contact is made, there's a whole lot of noise there. In my experiments, I found that the noise would persist for about two thirds of the length of a 20wpm dot. In my solution, the PIC does the debouncing, which makes the Z program a lot simpler. If the debouncing had to be done on the Z, I think the timing requirements would be an order of magnitude more severe. Well, either that, or I'd have to be a cleverer programmer than I am. Which is not likely to happen any time soon :-(

We've already seen in this thread the solution that I entirely missed: the use of the audio port, which by definition capures samples at regular and known intervals, and buffers them until you're ready for them. If I were starting again on this project, I think I might well begin there. That does seem to be a way of getting the Z to do the timings with accuracy.
Title: Morse Code Input
Post by: Da_Blitz on May 01, 2007, 03:03:34 am
one thing that can help with debouncing is ethier some circutry which i am no doubt aware of (just a cap and resistor) or mechanical debouncing

jet a thin bit fo rubber and put thatin a postion so that the reber gets struck before the copper, this should slow it down enough that you dont bounce the contact.

i guess it all depends on your clicker  but i hope that trick helps

I love the audio port, its so versitile yet so over looked. sure it cant do high speed sampaling but for home stuff its fine. My electrical egineering course had one C class in it where they maintain a 486 with a bourland C compiler, the idea is that we can use them for playing with the low level hardware. its alot easier to play around when your  project and IDE are the same thing (they were running DOS)

had a nice little ossciliscope going that could also inject signals if needed