OESF Forums > Angstrom Hangs With Soft Lockup

Mainframe-junkie

Mar 9 2008, 12:32 PM

Hello friends,

after reading the thread abut the new image with PCMCIA support i decides to flash the image http://www.angstrom-distribution.org/unsta...ad.rootfs.jffs2,
after rebooting Simpad hangs with the following messages :

CODE

.udev/ already exists on the static /dev!
Starting the hotplug events dispatcher udevd
Synthesizing the initial hotplug events
hostap_cs: Registered netdevice wifi0
hostap_hw.c: INTERRUPT BEFORE DEVICE INIT!
hostap_cs: index 0x01: , irq 1, io 0xc4800000-0xc480003f
wifi0: NIC: id=0x800c v1.0.0
wifi0: PRI: id=0x15 v1.1.1
wifi0: STA: id=0x1f v1.7.4
wifi0: NIC: id=0x800c v1.0.0
wifi0: PRI: id=0x15 v1.1.1
wifi0: STA: id=0x1f v1.7.4
Waiting for /dev to be fully populated
Remounting root file system...
root: mount: special device /dev/mtdblock/2 does not exist
Setting up IP spoofing protection: rp_filter.
Configuring network interfaces... SIOCGIFFLAGS: No such device
eth0      No such device

udhcpc (v1.2.1) started
udhcpc[2641]: udhcpc (v1.2.1) started
SIOCGIFINDEX failed!: No such device
udhcpc[2641]: SIOCGIFINDEX failed!: No such device
done.
Starting portmap daemon: portmap.
BUG: soft lockup - CPU#0 stuck for 11s! [hwclock:2664]

Pid: 2664, comm:              hwclock
CPU: 0    Not tainted  (2.6.24 #1)
PC is at __do_softirq+0x4c/0xd0
LR is at __do_softirq+0x2c/0xd0
pc : [<c003d240>]    lr : [<c003d220>]    psr: 60000013
sp : c2c03d84  ip : c2c03d84  fp : c2c03d9c
r10: c032d424  r9 : c2c02000  r8 : 00000002
r7 : 00000003  r6 : 0000000a  r5 : c0358e5c  r4 : 00000022
r3 : 20000013  r2 : c0358e20  r1 : 00000104  r0 : 00000000
Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment us
Control: c2c1b17f  Table: c2c1b17f  DAC: 00000015

I attach the logs for further information.Click to view attachment

Thanks for any suggestions and help

Oliver

koen

Mar 9 2008, 11:59 PM

Did you try reading the angstrom frontpage? http://www.angstrom-distribution.org/

mister_data

Mar 10 2008, 08:11 AM

QUOTE(Mainframe-junkie @ Mar 9 2008, 09:32 PM)

Hello friends,

after reading the thread abut the new image with PCMCIA support i decides to flash the image http://www.angstrom-distribution.org/unsta...ad.rootfs.jffs2,
after rebooting Simpad hangs with the following messages :

CODE

.udev/ already exists on the static /dev!
Starting the hotplug events dispatcher udevd
Synthesizing the initial hotplug events
hostap_cs: Registered netdevice wifi0
hostap_hw.c: INTERRUPT BEFORE DEVICE INIT!
hostap_cs: index 0x01: , irq 1, io 0xc4800000-0xc480003f
wifi0: NIC: id=0x800c v1.0.0
wifi0: PRI: id=0x15 v1.1.1
wifi0: STA: id=0x1f v1.7.4
wifi0: NIC: id=0x800c v1.0.0
wifi0: PRI: id=0x15 v1.1.1
wifi0: STA: id=0x1f v1.7.4
Waiting for /dev to be fully populated
Remounting root file system...
root: mount: special device /dev/mtdblock/2 does not exist
Setting up IP spoofing protection: rp_filter.
Configuring network interfaces... SIOCGIFFLAGS: No such device
eth0      No such device

udhcpc (v1.2.1) started
udhcpc[2641]: udhcpc (v1.2.1) started
SIOCGIFINDEX failed!: No such device
udhcpc[2641]: SIOCGIFINDEX failed!: No such device
done.
Starting portmap daemon: portmap.
BUG: soft lockup - CPU#0 stuck for 11s! [hwclock:2664]

Pid: 2664, comm:              hwclock
CPU: 0    Not tainted  (2.6.24 #1)
PC is at __do_softirq+0x4c/0xd0
LR is at __do_softirq+0x2c/0xd0
pc : [<c003d240>]    lr : [<c003d220>]    psr: 60000013
sp : c2c03d84  ip : c2c03d84  fp : c2c03d9c
r10: c032d424  r9 : c2c02000  r8 : 00000002
r7 : 00000003  r6 : 0000000a  r5 : c0358e5c  r4 : 00000022
r3 : 20000013  r2 : c0358e20  r1 : 00000104  r0 : 00000000
Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment us
Control: c2c1b17f  Table: c2c1b17f  DAC: 00000015

I attach the logs for further information.Click to view attachment

Thanks for any suggestions and help

Oliver

Hi,

first: thank you for testing

second: koen's suggestion is to fill a bug report for such errors on http://bugs.openembedded.org

third: i called this error the sa1100-rtc-bug, and there is a dirty solution for simpad available , but is not included yet in oe -> SIMpad-rtc-sa1100.patch

This error comes while running /etc/init.d/hwclock.sh on startup or any other run busybox->hwclock.

Not everyone should have this bug and there is no root cause known yet.

So sorry and keep in mind, this is work in progress, not many developers are around who work's for SIMpad-kernel-2.6.

I will talk to mr_nice for future way to solve this error.

Regards,

mrdata

koen

Mar 10 2008, 08:36 AM

QUOTE(mister_data @ Mar 10 2008, 05:11 PM)

second: koen's suggestion is to fill a bug report for such errors on http://bugs.openembedded.org

Actually my suggestion was to read the news telling people to test r9, not r8. Filing bugs is still recommended, though

fredyd

Mar 10 2008, 09:38 AM

Well, I vote for the inclusion of your quick'n'dirty patch in OE while waiting for a better solution, since it block me too (didn't try r9 yet). I don't know how many tested r8 or r9, but at least two of them can't boot.

The strange thing is that I never came across this one with previous releases (the ones where PCMCIA was broken).

The good news is that if I insert my NE2K PCMCIA card before it locks up, the card is properly recognized, so I guess PCMCIA is working again!
Thanks a lot.

Fred

fredyd

Mar 11 2008, 01:26 AM

same thing happens with r9, but I also noticed some strange messages in the boot sequence (did I sleep for 30 years and wake up in 2038?):

CODE

sa1100-rtc sa1100-rtc: rtc core: registered sa1100-rtc as rtc0
SA1100/PXA2xx Watchdog Timer: timer margin 60 sec
TCP cubic registered
NET: Registered protocol family 1
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
sa1100-rtc sa1100-rtc: setting system clock to 2038-01-19 05:58:33 UTC (2147493513)

On a side note, I booted with my CF adaptor plugged in, and I also noticed that message:

CODE

Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 50MHz system bus speed for PIO modes; override with idebus=xx
hda: TOSHIBA THNCF256MMA, CFA DISK drive
ide0 at 0xc4800000-0xc4800007,0xc480000e on irq 1
ide-cs: hda: Vpp = 0.0
hda: max request size: 128KiB
hda: 500736 sectors (256 MB) w/2KiB Cache, CHS=978/16/32
 hda: hda1
fat: disagrees about version of symbol struct_module
modprobe: FATAL: Error inserting fat (/lib/modules/2.6.24/kernel/fs/fat/fat.ko): Invalid module format

mr_nice

Mar 12 2008, 02:32 PM

hi,
thanks a lot for testing. On my device I am not able to reproduce the error so I suggest the following as a dirty work around.

We create the Image with rtc patch enabled, which will make rtc mostly useless. But also provide an unpatched rtc module? Or the other way around? I do not really like it but an nearly usabe Image is still better than an not usable one?
What do you think?

regards
mr nice

btw: please, someone file a bugreport at bugs.openembedded.org and make it block 2595

Mainframe-junkie

Mar 16 2008, 05:57 AM

The same happened with 2007-12 R10.
Filed Bug 4084, and made him block 2595, as suggested.

Best regards

Oliver

fredyd

Mar 16 2008, 06:26 AM

mr_nice,
If you can't reproduce the bug, is there any way we could help fixing it?

Fred

mr_nice

Mar 18 2008, 07:50 AM

Hello all,
once again mrdata helped us and gave me a patch for it

.
I added it to the bugtracker as a kernel update. hopefully some oe dev with commit rights will add it to .dev and stable.

If you cannot wait

I builded a new _unoffical_ Image with the patches and uploaded it here [1].

happy testing

kown bugs:
http://bugs.openembedded.org/show_bug.cgi?id=4096

mr nice
1 http://opensimpad.org/images/c/c1/Angstrom...ad.rootfs.jffs2

Mainframe-junkie

Mar 20 2008, 05:19 AM

Hello mr nice,

just flashed the image and tried if simpad will boot without soft lockup, but unfortunally the error is still there .....

CODE

Starting portmap daemon: portmap.
Machine: Sim
SA110
BUG: soft lockup - CPU#0 stuck for 11s! [hwclock:2076]iteba

Pid: 2076, comm:              hwclock0]
jffs2_scan_eraseblock():
CPU: 0    Not tainted  (2.6.24 #1)x01459a04: 0xbee0 inry cache hash
PC is at __do_softirq+0x54/0xcc2768 bytes)
LR is at __do_softirq+0x3c/0xcc
pc : [<c003e0dc>]    lr : [<c003e0c4>]    psr: 60000013tead
jffs2_scan_eraseblock(): Magic bitmask 0x1985
sp : c3027d74  ip : c3027d74  fp : c3027d9c
r3 : 20000013  r2 : 00000104  r1 : c3026000  r0 : 00000000
stead=4, C
jffs2_scan_eraseblock():
Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
Control: c3fbb17f  Table: c3fbb17f  DAC: 00000015
stead
jf
[<c0026fd8>] (show_regs+0x0/0x4c) from [<c006471c>] (softlockup_tick+0xf4/0x13c)
TC

r4:00000028 configured
[<c0064628>] (softlockup_tick+0x0/0x13c) from [<c00429cc>] (run_local_timers+0x1
stead6:000
jffs2_scan_eraseblock(): Magic bitmask 0x19
8/0x1c)ound at
[<c00429b4>] (run_local_timers+0x0/0x1c) from [<c0042c9
r5:00000000 r4:c034d79c
Starti
[<c00649ec>] (handle_IRQ_event+0x0/0x108) from [<c00667e4>] (handle_level_irq+0x
Console: switching to
Synthesizing the initial hotplug ev
ac/0x154)ag at
r8:00000104 r7:00000105 r6:c034d79c r5:0000001a r4:c035r /dev to be

[<c0025000>] (__exception_text_start+0x0/0x60) from [<c0025aa0>] (__irq_svc+0x40A1100 flash: CFI device at 0x0
/0xb0)

Exception stack(0xc3027d2c to 0xc3027d74)0itebackements = 000000
7d20:                            00000000 c3026000 00000104 20000013 c037a71c
VFS: Can't find a valid FAT filesystem on dev hda.devices at 0x0
7d40: 00000022 0000000a c037a6e0 00000001 c037ba20 00000000 c3027d9c c3027d74
FAT: invalid media value (0x00)
7d60: c3027d74 c003e0c4 c003e0dc 60000013 ffffffff
VFS: Can't find a valid FAT filesystem o
r6:00000011 r5:fa050000 r4:ffffffff
eth0      No such device
[<c0025000>] (__exception_text_start+0x0/0x60) from [<c0025aa0>] (__irq_svc+0x40

Pid: 2077, comm:              hwclock found on MTD devic
[<c0064f68>] (setup_irq+0x0/0x264) from [<c006529c>] (request_irq+0xd0/0xf8)
r7:c01e7f80 r6:0000001e r5:c3e5fee0 r4:00000000
[<c00651cc>] (request_irq+0x0/0xf8) from [<c01e7b9c>] (sa1100_rtc_open+0x2c/0x10
8)
[<c01e7b70>] (sa1100_rtc_open+0x0/0x108) from [<c01e6b64>] (rtc_dev_open+0x64/0x
d8)
r5:c3dd6800 r4:c3dd6938
[<c01e6b00>] (rtc_dev_open+0x0/0xd8) from [<c009119c>] (chrdev_open+0x1dc/0x1fc)

r6:c31f805c r5:c3dd6938 r4:00000000
[<c0090fc0>] (chrdev_open+0x0/0x1fc) from [<c008bfa4>] (__dentry_open+0x140/0x22
8)
r8:c3e81980 r7:c0090fc0 r6:c31f805c r5:c30e0680 r4:00000000
[<c008be64>] (__dentry_open+0x0/0x228) from [<c008c12c>] (nameidata_to_filp+0x38
/0x4c)
[<c008c0f4>] (nameidata_to_filp+0x0/0x4c) from [<c008c17c>] (do_filp_open+0x3c/0
x44)
r4:00020000
[<c008c140>] (do_filp_open+0x0/0x44) from [<c008c1dc>] (do_sys_open+0x58/0xe4)
r5:00000000 r4:00020000
[<c008c184>] (do_sys_open+0x0/0xe4) from [<c008c2a0>] (sys_open+0x24/0x28)
r8:c00260e4 r7:00000005 r6:0007804c r5:00000000 r4:00000000
[<c008c27c>] (sys_open+0x0/0x28) from [<c0025f40>] (ret_fast_syscall+0x0/0x2c)

Attached is the boot-log of the image.

Anything i can do to help you (and mrdata) with the bug ??

Best regards

Oliver

Click to view attachment

mr_nice

Mar 20 2008, 08:15 AM

QUOTE(Mainframe-junkie)

Anything i can do to help you (and mrdata) with the bug ??

Hi,
thank you for testing the image!
please try this [1] image. It is build with an different rtc-sa1100 patch.

best regards
mr nice

1 http://opensimpad.org/images/b/b8/2-try-An...ad.rootfs.jffs2

Mainframe-junkie

Mar 20 2008, 12:56 PM

Hello mr_nice,

thanks for this image is flashed it and your easter egg has been brocken.
It fails with soft lookup bug (sorry only some of us have it ...).
Maybe we can pin the bug when we collect our HW-revisions or Serial# about the failing simpads ??

I wish all nice easter

Oliver

Click to view attachment

mr_nice

Mar 23 2008, 09:25 AM

Hi,
here [1] is another image build with a new patch. Please test it

.
It provides some debug information, but if you are lucky it solves the rtc problem as well.
It is a repacked r12 image with a new kernel (for my convenience) so it could cause some other problems but should not (tested on my pad).

best regards and happy easter
mr nice

1 http://opensimpad.org/images/1/1e/Simpad-2...image-r12.jffs2

btw: If it really boots and you will be able to log in please do the following as root:

cat /dev/rtc &
cat /proc/interrupts

and test suspend/resume for the rtc bug

My SIMpad is:
HW 4 / SW 2
KW 06/02

Mainframe-junkie

Mar 23 2008, 11:40 AM

Happy easter mr nice,

just flashed and booted ....
New highscore on date, year 2104 beats 2036, but the image seems to loops in hwclock (just a guess...).
We need to hunt on ...

CODE

udhcpc (v1.2.1) started
udhcpc[2677]: udhcpc (v1.2.1) started

SIOCGIFINDEX failed!: No such device
udhcpc[2677]: SIOCGIFINDEX failed!: No such device

done.
Starting portmap daemon: portmap.
IN SA1100_RTC_OPEN:
pointer to sa1100_rtc_interrupt befor IRQ_RTC1Hz: C01DBCAC
rtc pointer in sa1100_rtc_interrupt: C3D83800
INTERRUPT BEFORE DEVICE INIT! - IRQ NOT HANDLED
rtc pointer in sa1100_rtc_interrupt: C3D83800
INTERRUPT BEFORE DEVICE INIT! - IRQ NOT HANDLED
rtc pointer in sa1100_rtc_interrupt: C3D83800
....

My Simpad is KW 25/02 HW 2 / SW 3

best regards and nice easter

Oliver

Click to view attachment
P.S.:
read on ibot about my logs, this was captured with PuTTY (learned how to use serial ...)

mr_nice

Mar 24 2008, 12:46 AM

QUOTE

We need to hunt on ...

well ok,
a new day new image [1] to test

1 http://opensimpad.org/images/1/1f/Simpad-2...image-r12.jffs2

Mainframe-junkie

Mar 24 2008, 01:59 AM

Hello mr nice,

i think you pinned the bug :

CODE

.-------.
|       |                  .-.
|   |   |-----.-----.-----.| |   .----..-----.-----.
|       |     | __  |  ---'| '--.|  .-'|     |     |
|   |   |  |  |     |---  ||  --'|  |  |  '  | | | |
'---'---'--'--'--.  |-----''----''--'  '-----'-'-'-'
                -'  |
                '---'

The Angstrom Distribution simpad ttySA0

Angstrom 2007.12-r12 simpad ttySA0

simpad login:

I can't login to show the interrupts and the content of /dev/rtc ....

What userid and password is valid to login ??

Best regards

Oliver

Click to view attachment

mr_nice

Mar 24 2008, 03:14 AM

Fine

.
The bug fix is not only my work, most of it was done by mister_data.

For the password you have to follow the instructions on your simpad screen. You will have to calibrate the screen after this you will be able to give a password for root. With this password you will be able to log in via serial connection.
But be aware the image I created was mainly for debugging the rtc stuff. It is a rebuilt image so there could be some side effects the normal image does not have. I will hand the patch in and ask for a new official autobuilder image.
In the meanwhile feel free to have some fun with this image

Thank you for continuous testing!

mr nice

Mainframe-junkie

Mar 24 2008, 04:04 AM

Hello mr nice,

first of all thanks to all contributors (you, mr data, florian, woglinde .....)
I didn't look to the screen, only watched the console (Simpad layed on floor face down)....

Here is the output :

CODE

root@simpad:~$ cat /dev/rtc & cat /proc/interrupts
RTSR in rtc_open: 0
OIER in rtc_open: 1
OSSR in rtc_open: 0
           CPU0
  0:          0      GPIO-l  power button
17:        486          SC  sa11x0-uart
26:       8778          SC  SA11xx Timer Tick
27:          0          SC  rtc timer
30:          0          SC  rtc 1Hz
31:          0          SC  rtc Alrm
43:          0      GPIO-h  UCB1x00
45:          0      GPIO-h  CF_CD
Err:          0

after suspend/resume it was "dead" ...

CODE

root@simpad:~$ Syncing filesystems ... done.
Freezing user space processes ... (elapsed 0.02 seconds) done.
Freezing remaining freezable tasks ... (elapsed 0.00 seconds) done.
Suspending console(s)
U3WKUP

So we need to track down the resume process ....

Thanks and best regards

Oliver

Click to view attachment

mr_nice

Mar 24 2008, 06:46 AM

Ok, with this image [1] suspend/resume should work. Can you please test it?

thanks
mr nice

1 http://opensimpad.org/images/f/f5/Simpad-2...image-r12.jffs2

Mainframe-junkie

Mar 24 2008, 08:15 AM

Hello mr nice,

just tried the image and suspend/resume was working

Thanks and have a nice day

Oliver

Click to view attachment

P.S.:
Now i can continue on to get WPA running ....
Are the scripts in the image from OE, /etc/network/if-pre-up.d/wpasupplicant doesn't look like the samples on http://opensimpad.org/index.php/HowToUseWPA and raise an error from busybox/sed.

mr_nice

Mar 24 2008, 08:57 AM

Fine, so we can hand in the patches :).

I have to split the knowledge base better. I will do that now.
The WPA howto was written for Familiar 0.8.4. We will collect everything which is device-dependening on opensimpad.org and everything which is independent and related to angstrom will be add on distribution.org . This is not really angstrom-alike because most other devices are documented there as well.

I don't like this approach for the SIMpad because of the following reasons:

a) Distributions become obsolete and unmaintained . Look at SIMpads history. Angstrom is now the third main distribution and I believe that SIMpad may live longer than Angstrom. Then you have to sort out all the device dependent stuff which changed there and recollect it somewhere else. Nobody knows about the recent state and a lot of effort is needed which could be better used to do other things.

b) SIMpad users are a very small community. It would be nice to have a home. Some people maybe don't like Angstrom and prefer something else. These people can still contribute on a site which is distribution-independent and profit from such a documentation.

c) Documentations will split in many different parts which will all get some changes. This is hard to track.

For WPA on Angstrom follow this discription:
http://linuxtogo.org/gowiki/AngstromManual...b15a8f220ba262f

Mainframe-junkie

Mar 24 2008, 09:56 AM

Hello mr nice,

so the next autoimage will be ok ...

Thanks for the link, i will read on ...

Thanks and nice evening

Oliver

fredyd

Mar 26 2008, 12:59 AM

Simply amazing, it boots like a charm! Thanks a lot for the great work. Now I can at least experiment a bit with that Angstrom distribution and the 2.6 kernel.

I still have to test my various PCMCIA cards, but it looks like lots of strange boot messages have been fixed already.

So what's the next challenge? Getting the sound to work? Will we be able to use that old guy as a SIP phone some day?

fred

mr_nice

Mar 26 2008, 06:06 AM

Nice to hear that you like it

.

QUOTE(fredyd @ Mar 26 2008, 12:59 AM)

So what's the next challenge? Getting the sound to work? Will we be able to use that old guy as a SIP phone some day?

2.6.24 misses 4 things:
- sound
- usb
- smartcard
- DECT interface

There is a paper available for a similiar chip. I will try to use it as a base for an ALSA System on Chip (ASoC) driver. But I don't know if I will be able to do it.
There are two different patches available for usb net support one is the old driver ported to 2.6 (look at hh.org kernel tree) and the other is a driver written for balloon. Both are not working without changes on SIMpad. For me USB is not importent but maybe I will find some time...
The smartcard driver should not be that hart to develop but no one is really interested in having it except I.
I really have no clue about DECT.

A good challenge would be to clean up the patches and send them to lkml.

I am currently try to port u-boot to the pad, and I was able to boot Angstrom with it

.

What do you think about the current bootldr partions?
Should we switch with u-boot to one big ~31 MB partition and delete the 8MB home?
pro:
- More space for applications
con:
- You have to backup /home befor you flash a new image.
- If you download a to big file to /home you may not be able to boot your device.

I also try to get poky linux [1] build for SIMpad

.

mister_data what do you think we should do next?

1 http://www.pokylinux.org/

Master One

Mar 26 2008, 04:27 PM

Well done, the latest test-image (Simpad-2.6.24.rtc-debug-4-image-r12) seems to be working so far, but there is not enough space left on the root partition to install minimo, and without it, it all makes no sense to me, because all I want to do with it, is surf the web.

I tried deleting as many packages, as I thought are not needed for my purpose (like all the bluetooth and irda stuff, GPE package manager, firmware-stuff, kernel-modules for cards not used), but still not enough space left:

CODE

$ df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/mtdblock1           23.8M     23.5M    304.0k  99% /
tmpfs                    40.0k         0     40.0k   0% /mnt/.psplash
/dev/mtdblock1           23.8M     23.5M    304.0k  99% /dev/.static/dev
tmpfs                     2.0M     32.0k      2.0M   2% /dev
tmpfs                    30.1M         0     30.1M   0% /dev/shm
tmpfs                    30.1M    120.0k     29.9M   0% /var/volatile
tmpfs                    30.1M      4.0k     30.1M   0% /media/ram
/dev/mtdblock2            8.0M      5.8M      2.3M  72% /home

The minimo package itself has 4.7 MB (but it has two dependencies as well), the old familiar image (which allowed me to install minimo without problem) was 14.027 KB, this Angstrom image has 17.984 KB.

What can be done to make minimo fit in?
What else could I delete / remove?

mr_nice

Mar 26 2008, 04:52 PM

You have at least these options:
-use gpe-mini-browser
-delete your home partition

I wrote this [1] howto about removing your home partition. It requires to reflash the jffs2 image.

happy testing
mr nice

1 http://opensimpad.org/index.php/Get_rid_of..._home_partition

Master One

Mar 27 2008, 09:04 AM

Thanks, mr_nice, that did the job.