Hang on Shutdown

[jakfish]

Running a fully updated stock Debian, i find that I can no longer shut down Debian without a a hard turn-off. I can reboot, and I can shut down if I log off first.

I did not have this issue before a dist-upgrade earlier this month.

Any handle to jiggle?

Jake

[Kiriririn]

If you can't get the cause from the systemd journal (set to non-volatile), then you'll need to make a serial console cable

I had a similar issue and had to do the latter unfortunately. It turned out to be a custom systemd service I had set up, so not something that would help

[jakfish]

Thank you for your very fast response.

>>systemd journal (set to non-volatile)<< That's a bit above my paygrade in Linux. Could you give me quick pointers on the right way to access that journal.

I'm using the stock version of systemd.

Thanks again,
Jake

[Kiriririn]

Thank you for your very fast response.

>>systemd journal (set to non-volatile)<< That's a bit above my paygrade in Linux. Could you give me quick pointers on the right way to access that journal.

I'm using the stock version of systemd.

Thanks again,
Jake

I think all you need to do is create /var/log/journal (with appropriate permissions) and killall -USR1 systemd-journald. That'll give you a persistent journal so reboot and then hopefully your issue will be visible via journalctl

[depscribe]

Running a fully updated stock Debian, i find that I can no longer shut down Debian without a a hard turn-off. I can reboot, and I can shut down if I log off first.

I did not have this issue before a dist-upgrade earlier this month.

Any handle to jiggle?

Jake

what happens if you open a terminal and do

Code: [Select]

sudo shutdown -h now
?

[Murple2]

Thank you for your very fast response.

>>systemd journal (set to non-volatile)<< That's a bit above my paygrade in Linux. Could you give me quick pointers on the right way to access that journal.

I'm using the stock version of systemd.

Thanks again,
Jake

Edit /etc/systemd/journald.conf and set Storage=persistent. You'll need to reboot or

Code: [Select]

systemctl restart systemd-journald. You should then have logs saved to /var/log/journald. Although you still need to use journalctl to read them, which is a tad controversial but allows you to do neat things like 'journalctl -b -1' (show messages from last boot) or 'journalctl --since "5 min ago"'

[jakfish]

Folks, thank you all for your follow-ups.

I had already re-flashed the linux img so the shut-down issue is gone, but if it returns, the posts here will me start the troubleshooting.

Jake

[Kero]

Doesn't seem to be related to `apt update` on my device, just luck...
Created /var/log/journal and rebooted. After a short session, I did `sudo reboot` which hung.

Today I booted into gemian again, and indeed, `journalctl -b 1` gives me loads of info. I can see systemd trying the reboot.

One entry of which I am not sure it is relevant (probably not, but I'm messing with audio, so better report it)

Code: [Select]

Aug 07 23:18:12 gemini alsactl[3675]: /usr/sbin/alsactl: get_control:256: Cannot read control '2,0,0,Audio_DL2_DataTransfer,0': Operation not permitted
But then we get this

Code: [Select]

Aug 07 23:18:12 gemini kernel: [Power/PPM] (0x420)(14500)(1)[4L_LL]->[L_ONLY]: (15)(0)(0)(0) (8)(0)(1)(4) (15)(9)(0)(0)
Aug 07 23:18:12 gemini kernel: [EXTD-MGR]:[fb_notifier_callback] - blank_mode:0
Aug 07 23:18:12 gemini kernel: [EXTD-HDMI]:hdmi_power_enable
Aug 07 23:18:12 gemini kernel: [EXTD-HDMI]:return in 1080
Aug 07 23:18:12 gemini kernel: DISP/MTKFB [FB Driver] enter late_resume
Aug 07 23:18:12 gemini kernel: DISP/MTKFB [FB Driver] leave late_resume
Aug 07 23:18:12 gemini kernel: [VcoreFS]SCREEN ON
Aug 07 23:18:12 gemini kernel: Power/swap CNT(soidle,rgidle): [0] = (0,4678326), [1] = (0,157322), [2] = (0,27014), [3] = (0,6775), [4] = (0,299315), [5] = (0,110423), [6] = (0,16936), [7] = (0,8175), [8] = (0,22380), [9] = (0,1821),
Aug 07 23:18:12 gemini kernel: Power/swap soidle_block_cnt: [by_cpu] = 13133, [by_clk] = 28626, [by_tmr] = 0, [by_oth] = 0, [by_vtg] = 0, [by_frm] = 0, [by_pll] = 0, [by_pwm] = 0, [by_dvfsp] = 0, [by_conn] = 0,
Aug 07 23:18:12 gemini kernel: Power/swap soidle_block_mask: 0x00000000, 0x00000100, 0x00000200, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, 0x00000000,
Aug 07 23:18:12 gemini kernel: i2c i2c-5: addr: 5b, transfer ACK errorfollowed by i2c_dump_info which is so huge I'll post it only if someone really wants it (and somehow cannot reproduce). The Power entries seem to be regular kernel debug info (but not 100% sure).

There are a few ION warnings around this, like

Code: [Select]

Aug 07 23:18:12 gemini kernel: [ION]warning: release handle @ client destroy: handle=ffffffc0a3f35280, buf=ffffffc0a3ec6500, ref=2, size=9334784, kmap=0
Then sddm starts a new X server (wtf?)
Then things go haywire.

Code: [Select]

Aug 07 23:18:13 gemini sddm[2074]: Running: /usr/bin/X -nolisten tcp -dpi 192 -auth /var/run/sddm/{0a9bec47-e7f8-472c-9031-534b7a50fddc} -background none -noreset -displayfd 19 -seat seat0 vt7
Aug 07 23:18:13 gemini kernel: [Power/PPM] (0x420)(12473)(1)[L_ONLY]->[4L_LL]: (15)(0)(0)(4) (15)(0)(1)(4) (15)(9)(0)(2)
Aug 07 23:18:13 gemini kernel: [DISP][release_session_buffer #215]ERROR:release_session_buffer: no session 196610 found!
Aug 07 23:18:13 gemini kernel: binder: 3733:3733 binder_context_mgr_node is NULL
Aug 07 23:18:13 gemini kernel: binder: 3733:3733 transaction failed 29189, size 0-0
Aug 07 23:18:13 gemini kernel: binder: read put err 29189 to user 0000007e24f13808, thread error 29189:29185
Aug 07 23:18:13 gemini kernel: [Power/PPM] (0x420)(15037)(1)[4L_LL]->[L_ONLY]: (15)(0)(0)(0) (8)(0)(1)(4) (15)(9)(0)(0)
Aug 07 23:18:13 gemini kernel: [Power/PPM] (0x420)(15800)(0)[L_ONLY]->[LL_ONLY]: (3)(0)(1)(4) (8)(0)(0)(0) (15)(9)(0)(0)
Aug 07 23:18:13 gemini kernel: [Power/PPM] (0x420)(15939)(0)[LL_ONLY]: (15)(0)(1)(4) (8)(0)(0)(0) (15)(9)(0)(0)
Aug 07 23:18:13 gemini kernel: ------------[ cut here ]------------
Aug 07 23:18:13 gemini kernel: WARNING: CPU: 0 PID: 3545 at /home/kero/GeminiPDA/linux-kernel-3.18/kernel/irq/manage.c:454 enable_irq+0x88/0xd0()
Aug 07 23:18:13 gemini kernel: Unbalanced enable for IRQ 394
Aug 07 23:18:13 gemini kernel: CPU: 0 PID: 3545 Comm: kworker/0:0 Tainted: G        W      3.18.41-kero+ #9
Aug 07 23:18:13 gemini kernel: Hardware name: MT6797X (DT)
Aug 07 23:18:13 gemini kernel: Workqueue: events aw9523_key_eint_work
Aug 07 23:18:13 gemini kernel: Call trace:
Aug 07 23:18:13 gemini kernel: [] dump_backtrace+0x0/0x15c
Aug 07 23:18:13 gemini kernel: [] show_stack+0x14/0x1c
Aug 07 23:18:13 gemini kernel: [] dump_stack+0x80/0xa4
Aug 07 23:18:13 gemini kernel: [] warn_slowpath_fmt+0xb4/0xd8
Aug 07 23:18:13 gemini kernel: [] enable_irq+0x88/0xd0
Aug 07 23:18:13 gemini kernel: [] aw9523_key_eint_work+0x43c/0x5cc
Aug 07 23:18:13 gemini kernel: [] process_one_work+0x160/0x468
Aug 07 23:18:13 gemini kernel: [] worker_thread+0x140/0x4e4
Aug 07 23:18:13 gemini kernel: [] kthread+0xd8/0xec
Aug 07 23:18:13 gemini kernel: ---[ end trace 7e7208ab083510d3 ]---
Aug 07 23:18:14 gemini kernel: binder: 3733:3733 binder_context_mgr_node is NULL
Aug 07 23:18:14 gemini kernel: binder: 3733:3733 transaction failed 29189, size 0-0
Aug 07 23:18:14 gemini kernel: binder: read put err 29189 to user 0000007e24f13808, thread error 29189:29185
Aug 07 23:18:14 gemini kernel: [Thermal_TA][ta_nl_send_to_user] send failed -111
Aug 07 23:18:15 gemini kernel: binder: 3733:3733 binder_context_mgr_node is NULL
Aug 07 23:18:15 gemini kernel: binder: 3733:3733 transaction failed 29189, size 0-0
Aug 07 23:18:15 gemini kernel: binder: read put err 29189 to user 0000007e24f13808, thread error 29189:29185
Aug 07 23:18:15 gemini kernel: [CQ] task 11 ready not clear when DMA
Aug 07 23:18:15 gemini kernel: [Power/PPM] (0x420)(15246)(1)[LL_ONLY]->[L_ONLY]: (15)(0)(0)(0) (8)(0)(1)(4) (15)(9)(0)(0)
Aug 07 23:18:15 gemini kernel: [Power/PPM] (0x420)(14529)(1)[L_ONLY]->[4L_LL]: (15)(0)(0)(4) (15)(0)(1)(4) (15)(9)(0)(2)
Aug 07 23:18:15 gemini kernel: NOHZ: local_softirq_pending 75
Note the Thermal entry,
There is

Code: [Select]

Aug 07 23:18:16 gemini kernel: [WDK]: kick Ex WDT,RT[20971198997764] after that, but other than that, loads of binder errors and a few more Thermal entries. I can see where I pressed Esc and held it, and ten seconds later, the log stops.

[jakfish]

I don't know what's going on, but you're definitely right about luck rather than a dist-upgrade gone wrong.

I reflashed the linux image, have done nothing except set up wifi, and my shut-down problem returned.

I'm holding out for an expert to make a custom distro that we can at least dd to the linux partition.

Here's hoping you find the solution,
Jake

[Kero]

The aw9523_key_eint_work stacktrace can also occur when reboot functions normally.
Seems to be an attempt to switch on the keyboard when Esc is pressed, by checking whether the screen was 'on'.
Naturally, when rebooting, the screen might be off, as well.
Specifically, this would be unrelated to the ghosting fix.

[jakfish]

It's so interesting--if that's the word--that you and I have this issue and others may not. Your Debian seems very customized and mine is factory default.

That's why I think it might be something to do with the wifi, though I must admit, I did not play with the shut-down before setting up wifi.

[ArchiMark]

FWIW, I had the shutdown issue at one point a while back, ie, several reflashes ago....

Knock on wood, seems to be OK now....

Have had times where it takes a while for it to complete the shutdown process.

[Kero]

Had to reflash because of audio troubles (still have them, but more benign now), so current install is rather vanilla.
Running self-built kernel, but it's the vanilla kernel with a bunch of debug statements removed, audio debug added and improved.

It could very well be that systemd has a problem terminating sddm, and I'm not patient enough,
But frankly, if that were the case, I'd call that a bug, too.

Since sddm-greeter is using 5% CPU, it's time to customize my install and replaced sddm with something else. Who knows what I'll learn about the reboots.

Upon that, those "binder" entries do not appear on a smooth shutdown/reboot, so I'm guessing there is more going on.

[jakfish]

Well, good luck to you--it sounds as if you already know a lot about this and here's hoping that the problem can be solved.

Just for kicks, I disabled wifi and did two successful shutdowns, but the third shutdown hung as usual. So with a factory install, no wifi, the problem appears to be specific to the distro, not because of user modification.