Author Topic: Morse Code Input  (Read 9934 times)

speculatrix

  • Administrator
  • Hero Member
  • *****
  • Posts: 3706
    • View Profile
Morse Code Input
« Reply #15 on: February 19, 2007, 05:36:59 am »
Quote
I was browsing Hackaday, and found this:

http://home.att.net/~jacksonharbor/palm.htm

It seems to be somewhat related, so I thought I'd post it.
[div align=\"right\"][a href=\"index.php?act=findpost&pid=154624\"][{POST_SNAPBACK}][/a][/div]

actually, a Palm might be a good/cheap morse man/machine interface, if you can find the right program for it. there are text-to-speech apps for it but I don't recall any being free.
Gemini 4G/Wi-Fi owner, formerly zaurus C3100 and 860 owner; also owner of an HTC Doubleshot, a Zaurus-like phone.

speculatrix

  • Administrator
  • Hero Member
  • *****
  • Posts: 3706
    • View Profile
Morse Code Input
« Reply #16 on: February 19, 2007, 06:01:36 am »
Quote
BUT: In the meantime, I seem to have hit a problem that may be severe. I was merrily coding away, assuming without thinking about it that a Qt event would have a timestamp field, telling me when the event actually occurred to some reasonable level of precision. I'm now dismayed to find that they don't. So if my app gets interrupted during Morse input, and events are forced to stay in the queue for a while, I'll have no means of getting back the original timings. And in Morse, those timings are quite small and obviously critical. Even if the app doesn't get interrupted, updating the screen to print what's being received looks as though it could cause trouble.

I guess I'm just going to have to carry on and see how it goes in practice, but any words of wisdom or comfort would be appreciated. Like, is there something obvious that I've failed to see, here?
[div align=\"right\"][{POST_SNAPBACK}][/a][/div]

I think if you want real-time execution, you're going to want to use SDL as a audio/video toolkit; I recall posting a link to a devshed article on getting started with SDL a month or so back. Ah yes, here it is:
[a href=\"http://www.devshed.com/c/a/Multimedia/Game-Programming-using-SDL-Getting-Started/]http://www.devshed.com/c/a/Multimedia/Game...etting-Started/[/url]
Gemini 4G/Wi-Fi owner, formerly zaurus C3100 and 860 owner; also owner of an HTC Doubleshot, a Zaurus-like phone.

jfr

  • Newbie
  • *
  • Posts: 36
    • View Profile
Morse Code Input
« Reply #17 on: February 19, 2007, 01:57:41 pm »
Thanks, Speculatrix - this project is proving to be quite an education!

Here's an update to the plan. A colleague has found a device called an MFJ-461, which is a small box that takes Morse input as sound, and generates serial output. I think that's my new favourite, because it handles the timings and, together with a serial-USB converter, it should solve the difficult part of the problem. I'm now investigating to see whether there's a way of connecting a key directly to it.

<a href="http://www.mfjenterprises.com/products.php?prodid=MFJ-461">
http://www.mfjenterprises.com/products.php...J-461</a>

speculatrix

  • Administrator
  • Hero Member
  • *****
  • Posts: 3706
    • View Profile
Morse Code Input
« Reply #18 on: March 05, 2007, 07:34:03 am »
did you buy one of these boxes?

don't forget that the C3000,3100 and 3200 don't have a serial port, you'd need to use a USB dongle.
Gemini 4G/Wi-Fi owner, formerly zaurus C3100 and 860 owner; also owner of an HTC Doubleshot, a Zaurus-like phone.

jfr

  • Newbie
  • *
  • Posts: 36
    • View Profile
Morse Code Input
« Reply #19 on: March 05, 2007, 03:04:39 pm »
Quote
did you buy one of these boxes?

don't forget that the C3000,3100 and 3200 don't have a serial port, you'd need to use a USB dongle.
[div align=\"right\"][a href=\"index.php?act=findpost&pid=155398\"][{POST_SNAPBACK}][/a][/div]
Yes, I did buy one, and it's sitting here in front of me now. It expects tone input, but I've connected a Morse key to the output of the tone detector, and it displays characters as I clumsily key them in.

The Z serial port is the big issue now. I really thought people had decided it did work, and experiment seems partly to bear that out. I bought a Sharp CE-170TS serial cable from Trisoft, and wired it to the morse decoder, but can't get the Z to admit that it's seeing anything. However, if I look at pin 2 of the Z's serial cable (TXD) with a voltmeter, I see zero volts normally, but two to three volts when I do
cat <some-file> > /dev/ttyS0
(This is on a 3100 by the way).
So that suggests to me that there's something there. And stty seems to work: I set the speed to 19200 as the morse box requires, and it kept the setting. I've also been through all the settings that stty reports, and there's nothing I can see that should prevent the thing from working. The stty settings include clocal (ignore modem control signals), but just in case, I've wired CTS to RTS and DSR to DTR on the Z end. Each of those lines shows a positive signal level. The morse box has no control connections: just TXD and RXD (which it ignores).
Maybe I'll have to admit defeat on this, but it's tantalizing that I can get output yet not input.

I did find through Google reports that the serial port input on the 5500 was tied to a pppd listener, but that's not the case on the C3xxx range, according to /etc/inittab.

Any ideas welcome - otherwise I guess a USB serial cable is next on the shopping list.

Cheers, and thanks again for the interest.

speculatrix

  • Administrator
  • Hero Member
  • *****
  • Posts: 3706
    • View Profile
Morse Code Input
« Reply #20 on: March 05, 2007, 04:25:23 pm »
try installing "minicom" from a feed, it'll allow you to drive the serial port with no handshaking.

--- edit ---

p.s. I remember now, there's evidence that the serial port on the xy00 IS functional and connected, just that the drivers are quite weak, and only a genuine Sharp serial cable will work... there's discussions about it in the 1000/3000/3100/3200 specific forum
« Last Edit: March 05, 2007, 04:42:35 pm by speculatrix »
Gemini 4G/Wi-Fi owner, formerly zaurus C3100 and 860 owner; also owner of an HTC Doubleshot, a Zaurus-like phone.

jfr

  • Newbie
  • *
  • Posts: 36
    • View Profile
Morse Code Input
« Reply #21 on: March 05, 2007, 05:58:18 pm »
Quote
try installing "minicom" from a feed, it'll allow you to drive the serial port with no handshaking.

--- edit ---

p.s. I remember now, there's evidence that the serial port on the xy00 IS functional and connected, just that the drivers are quite weak, and only a genuine Sharp serial cable will work... there's discussions about it in the 1000/3000/3100/3200 specific forum
[div align=\"right\"][{POST_SNAPBACK}][/a][/div]
Hmm. Just tried that, but the minicom ipk from the ZSI depends on other packages including libc6. I got libc6_6.1_arm.ipk from [a href=\"http://www.katastrophos.net/zaurus/packages/]http://www.katastrophos.net/zaurus/packages/[/url] but:
> ipkg install libc6_6.1_arm.ipk
Unpacking libc6..Done
Configuring libc6..dpkg: invalid option -- -
BusyBox v0.00.4 (2003.04.15-01:23+0000) multi-call binary

I can unpick the package to see whether I can fix it, but I thought I'd ask first whether a statically linked version of minicom was available anywhere.

AHH! WAIT!!!

Dont't know why I thought to do this, but I just tried
> sh < /dev/ttyS0
and now the transmission's getting through!

And a little C program I wrote is suddenly responding too.

I hadn't changed any settings, and I hadn't run minicom. And the minicom package has no preinst etc files, so merely installing it couldn't have made a difference.. Could I just have jogged a poor connection somewhere?

Whatever it was, I'm sure it wouldn't have been fixed if I hadn't been pursuing your suggestion.  So THANKS! even if we don't know why...

Now back to the coding...

speculatrix

  • Administrator
  • Hero Member
  • *****
  • Posts: 3706
    • View Profile
Morse Code Input
« Reply #22 on: March 05, 2007, 06:13:10 pm »
hmm,  I would have installed minicom from a cacko feed, but never mind, you got it working, well done!
Gemini 4G/Wi-Fi owner, formerly zaurus C3100 and 860 owner; also owner of an HTC Doubleshot, a Zaurus-like phone.

jfr

  • Newbie
  • *
  • Posts: 36
    • View Profile
Morse Code Input
« Reply #23 on: April 27, 2007, 09:11:42 pm »
Please pardon me for bumping my own thread, but I hope this update might be helpful to someone who might find it in the archive one day - and it also gives me the chance to say a big THANK YOU to all those who have contributed their ideas.

The project is now at the stage where I have a prototype that works as well as I can make it work, and I'm waiting for a chance to let B have a go at it.

The project has evolved a bit since the earlier postings.

The big issue was always the question of how to interface the Morse key to the Z. At a sending speed of 20 words per minute, a dot lasts for 60 milliseconds. That's quite a short time, and the time measurement is obviously critical for correct decoding. Any form of direct hardware interface would have to record event times to much better than that accuracy. When I realized that Qt didn't inlcude the original event timing in the event data structure, I decided I needed something more precise. If the event doesn't record the time it happened, the best you can do is to record the time when it came *out* of the event queue. It seemed a bit too much to hope for the needed accuracy that way.

As previously mentioned, I tried the MFJ-461 Morse decoder. It's a neat little box that's designed to be held up to a radio receiver loudspeaker, or connected directly to the audio output jack. Kudos to the manufacturer here: it expects an audio signal, not a direct connection of a Morse key. After looking at the circuit diagram, I decided where to connect the key, and emailed them asking for any advice. Their reply was prompt and helpful and as a result they are in ny good books - but in the end, I had to conclude that the decoder was better suited to the interpretation of automated Morse than to hand-generated.

So I introduced a new element - a PIC development board that was surplus to requirements at $DAYJOB, and which they were happy to donate.

The PIC board now has the Morse key connected to it, and I wrote a program for it which debounces the key - a very necessary step, as it turned out - and sends to its serial port a series of ASCII characters that encode just the timings between key-up and key-down events. That's all the PIC does - it does the timing measurement of 'up' and 'down' events, and it knows nothing about Morse code.

The serial output from the PIC then goes via a 3-way cable to the Z, and also to my desktop machine. The Z is running a program which listens to the serial input, and decodes the timings to generate the ASCII characters corresponding to the Morse. Every time it sees a full stop, it speaks (via flite) what it's collected. My friend who is the retired ship's radio operator (that is, my guinea-pig) is able to use it and to speak long  and complicated sentences.

The desktop machine receives the same input, and also speaks it in the same way. However, it adds value in that it displays an oscilloscope-like trace of the Morse input - the idea of which is that I can tune the decoder to the hand of the operator, if necessary. The desktop machine is just a debugging aid - not part of the final system.

I've learned lots from this project, and it isn't over yet. Perhaps the biggest lesson, though, is this:
1. It's not too hard to decode Morse, if you don't mind a few errors. The human brain is very good at interpreting text where some characters are in error. I've now seen quite a bit of software that does decode Morse, but "a few errors" are to be expected.
2.. But if you want to feed the result to a speech synthesizer, those few errors become intolerable and can wreck the output. The speech synthesizer isn't intelligent, and it needs very clean data. That's the hard bit.

Cheers

John

Da_Blitz

  • Hero Member
  • *****
  • Posts: 1579
    • View Profile
    • http://www.pocketnix.org
Morse Code Input
« Reply #24 on: April 28, 2007, 06:42:01 am »
just out of intreset which PIC did you end up using

i have been intrested in alternative input/output systems for years now and this have given me some thoughts (been trying to track down a cheap braille screen)

while i am at it (and to increse its historical significance) why chose the PIC?, i still belive that doing it all on the Z through the audio port would have been the best option, that way you would get the timing info you need as well as easy software/hardware upgrades (say you wanted a 2 paddle system)

thats intresting with the speach synthasiser, i remeber a long time ago playing with the speach stuff on the macs and found that incorect spellings and stuff that was phonetically correct gave the best output

perhaps a "tweaker" script between the program and the speach software is the way to go, i guess simple substitution would be best but is it worth the effort?
Personal Blog
Code
Twitter

Gemini Order: #95 (roughly)
Current Device: Samsung Chromebook Gen 3
Current Arm Devices Count: ~30
Looking to acquire: Cavium Thunder X2 Hardware

speculatrix

  • Administrator
  • Hero Member
  • *****
  • Posts: 3706
    • View Profile
Morse Code Input
« Reply #25 on: April 28, 2007, 05:00:49 pm »
fascinating project, am really impressed. I have a PIC programmer for some of the older PICs if you need some programming, I'd have to check out which ones it will do.

could you add a new character to the morse sequence to represent "rubout/delete"? that way if B sees there's an error, B could correct it.

some sort of automated spell checker/fixer might help with the text-to-speech.

I was wondering whether you could use the IR port on the zaurus to do away with the cable? Adding an IR transmitter to the morse key part wouldn't be too hard. It would also allow it to be used with other Z's which don't have a hardware serial port (the 1xxx or 3xxx for example).
Gemini 4G/Wi-Fi owner, formerly zaurus C3100 and 860 owner; also owner of an HTC Doubleshot, a Zaurus-like phone.

jfr

  • Newbie
  • *
  • Posts: 36
    • View Profile
Morse Code Input
« Reply #26 on: April 28, 2007, 06:42:32 pm »
Quote
just out of intreset which PIC did you end up using

The PIC16F84A. The reason for choosing that particular model was purely that $DAYJOB had a pile of old development/programming boards for them, which I was able to scrounge - along with an old Windows laptop, needed to run the programming software.

Quote
i have been intrested in alternative input/output systems for years now and this have given me some thoughts (been trying to track down a cheap braille screen)

while i am at it (and to increse its historical significance) why chose the PIC?, i still belive that doing it all on the Z through the audio port would have been the best option, that way you would get the timing info you need as well as easy software/hardware upgrades (say you wanted a 2 paddle system)

I can't argue against that, except to say that I was nervous about things like operating system latency affecting the measured timings. Presumably I'd have had to build a tone generator to attach to the key, so there would have been external hardware of some sort anyway - though of course the tone generator would have been a lot simpler. I didn't know how long the Z might suspend my program while the kernel did stuff, and I wanted to eliminate that potential problem. But I never experimented to find out whether the problem was really there. To do the timing on the Z, you'd need to be confident that your userland program would be able to measure the timings of events with 10 to 20mS accuracy - preferably 10mS or better. I lacked that confidence, but I'd be very interested if anyone could prove me wrong.

Quote
thats intresting with the speach synthasiser, i remeber a long time ago playing with the speach stuff on the macs and found that incorect spellings and stuff that was phonetically correct gave the best output

perhaps a "tweaker" script between the program and the speach software is the way to go, i guess simple substitution would be best but is it worth the effort?
[div align=\"right\"][a href=\"index.php?act=findpost&pid=160012\"][{POST_SNAPBACK}][/a][/div]

I agree about phonetic spellings - they tend to be shorter, too. Not sure what you mean by the tweaker program, though?

jfr

  • Newbie
  • *
  • Posts: 36
    • View Profile
Morse Code Input
« Reply #27 on: April 28, 2007, 07:36:04 pm »
Quote
fascinating project, am really impressed. I have a PIC programmer for some of the older PICs if you need some programming, I'd have to check out which ones it will do.

could you add a new character to the morse sequence to represent "rubout/delete"? that way if B sees there's an error, B could correct it.

some sort of automated spell checker/fixer might help with the text-to-speech.

I was wondering whether you could use the IR port on the zaurus to do away with the cable? Adding an IR transmitter to the morse key part wouldn't be too hard. It would also allow it to be used with other Z's which don't have a hardware serial port (the 1xxx or 3xxx for example).
[div align=\"right\"][a href=\"index.php?act=findpost&pid=160032\"][{POST_SNAPBACK}][/a][/div]

Thanks!

Especially for the kind programming offer, though the development boards I have are enough for my needs in that department so you can relax on that one :-).

Rubout/Delete: it's already defined in the Morse code - 6 or more dots as a single character, meaning "delete last word". The program recognizes it. The human interface problem (in my very limited tests) is that a trained Morse operator gets distracted by having to read the text while keying the input, presumably because of the time lag - to key the Morse, you're thinking ahead, whereas reading and checking the text implies a parallel brain process running in a different time zone, even if they're only a fraction of a second apart. So although I've implemented it, I've found that in practice it's less useful than I'd expected.

A spell-checker sounds very attractive. I have to admit, here, to a personal bias against them - if they're allowed to make corrections, then in my experience they get it wrong more often than they get it right. And for this sort of project, there would be little point in a spell-checker that didn't do its stuff on the fly (correcting without asking first). [begin rant] Mind you, my experience with spell-checkers is mainly limited to $DAYJOB's insistence on using Word to create exam papers. Word's ability to corrupt a perfectly well-formed exam question - without alerting anybody to the fact that it's done so - is the eighth wonder of the world. [end rant]

There's a conflict, too, between the use of a spell-checker and the use of phonetic spelling. I think that on balance, I'd rather allow phonetic spelling, which a spell-checker would almost certainly mis-correct. It'd be an interesting research question.

IR port - hadn't occurred to me, and it's a great idea. The current prototype has an embarrassing rat's nest of cables. The only downside is that you'll be troubled with a post that says "Ok, so how do I use the IR stuff?". Let's not worry about that yet - we're still waiting for B to try out the prototype, so it's a bit early to refine it for this project - but it's now in the forum archive as a suggestion.

Da_Blitz

  • Hero Member
  • *****
  • Posts: 1579
    • View Profile
    • http://www.pocketnix.org
Morse Code Input
« Reply #28 on: April 28, 2007, 08:48:36 pm »
the teweaker programe just changes spelling in some workds before its fed to flite to make the output easier to understand, i dont think its worth the effort of a dev, but if you could push its setup onto the user it might be worth it

ahh, the good old 16f84. a very nice chip. bty if you ever think of moving this to a cxx00 then you may want to consider a 18 series with usb built in. i know usb programming may not be your thing but there are some datasheets on how to make a keyboard out of the thing that could be adapted quite easily

dont forget Ir likes a 40Khz carrier, i had to do this once with a 16F877 and the best way was to use PWM hardware and another IO pin hooked together with a tranistor to get the on off keying (OOK), however with the 16f84 it looks lixe you may have to bit bang it (25 instructions per half cycle @ 4Mhz)

ps i may have stuffed up the above, cant remeber if its 40Khz (home remotes) or 455Khz

as for auido timing, its my understanding that you ask the alsa libs for X samples at a time, say you ask for 100 then you get 100 16bit sapmles at a time each representing 1/48000 fo a second

i dont think you wolud have to generate tones because the remote uses DC values on the line. so you should be able to get a way with a R2R bridge if you wanted more than one button otherwise a simple on/off should do
Personal Blog
Code
Twitter

Gemini Order: #95 (roughly)
Current Device: Samsung Chromebook Gen 3
Current Arm Devices Count: ~30
Looking to acquire: Cavium Thunder X2 Hardware

Armagon

  • Full Member
  • ***
  • Posts: 106
    • View Profile
    • http://
Morse Code Input
« Reply #29 on: April 30, 2007, 03:48:37 pm »
Quote
To do the timing on the Z, you'd need to be confident that your userland program would be able to measure the timings of events with 10 to 20mS accuracy - preferably 10mS or better. I lacked that confidence, but I'd be very interested if anyone could prove me wrong.
[div align=\"right\"][a href=\"index.php?act=findpost&pid=160035\"][{POST_SNAPBACK}][/a][/div]

10 mS is a long, long time to the computer.  I make video games for a living, and we try to get 60 frames per second.  Every 16.7 mS, the games I work on need to do physics calculations, transform and draw thousands of polygons, process artificial intelligence, and so on.  Granted, the hardware we develop for is designed for this sort of thing, but you can do a whole heck of a lot in twenty milliseconds of computational time.

I would be really surprized if multitasking on the Z would skew your timings too much.  I expect you could use a standard message pump and take for the time stamp the time that you get the callback to process the event, and that the amount of time that this would be off -- probably on the order of microseconds -- would be far less than the amount of variation time your program has to account for when looking at human-provided input.  

[Hmm... It appears that a single task may get up to ten milliseconds in one go, on the high end.  That computing quantum is higher than I thought, and may actually cause you a problem.  I don't know what the actual time is for the Z.  It may be, though, that multiple timings are skewed by the same amount, and, especially since your task will be the only one running, I don't think it'll be an issue.]

Armagon
Sharp Zaurus SL-C1000 -- 2 GB SD Card -- Zippy Slim Mini Keyboard -- Loving pdaXii13!