tl;dr; power management shuts off my screen after 10 minutes and then my computer reboots and I don't know why. This post is long because I've done SCIENCE! I apologize.
I have a minimal install of CentOS 7; this is meant to be a headless server for throwing up a few prototype web services. This was setup about a month back and then put aside until I picked it up again this morning.
So it boots, I ssh into it, start preparing it for what I need it to do, after 10 minutes of uptime it reboots. Consistently. Exactly 10 minutes, maybe 10 minutes and a couple seconds. I ssh in, work for 10 minutes, reboots.
Here's a snip of my "last reboot" output
Code: Select all
reboot system boot 3.10.0-1062.el7. Thu Dec 19 06:43 - 11:50 (05:06) reboot system boot 3.10.0-1062.el7. Thu Dec 19 06:32 - 11:50 (05:18) reboot system boot 3.10.0-1062.el7. Thu Dec 19 06:08 - 11:50 (05:42) reboot system boot 3.10.0-1062.el7. Thu Dec 19 05:58 - 11:50 (05:52) reboot system boot 3.10.0-1062.el7. Thu Dec 19 05:48 - 11:50 (06:02) reboot system boot 3.10.0-1062.el7. Thu Dec 19 05:38 - 10:47 (05:08) reboot system boot 3.10.0-1062.el7. Thu Dec 19 05:28 - 10:47 (05:19) reboot system boot 3.10.0-1062.el7. Thu Dec 19 05:18 - 10:47 (05:29) reboot system boot 3.10.0-1062.el7. Thu Dec 19 05:08 - 10:47 (05:39)
Things to note here: the fourth line from the bottom, 05:38 - 10:47, I actually forced a clean reboot there, waited 9 minutes 50 seconds, just to see if that made a difference. It did not; the next loop was exactly 10 minutes again.
5:48, 5:58, 6:08, all ten minutes apart, right? Then during the 6:08 run I wanted to see exactly how long it runs, to the second, so I started running "watch -n uptime". And I waited 10 minutes... and it kept going, 10 seconds, 20 seconds, one minute, two minutes. So long as I'm doing "something" (even a simple watch) it doesn't reboot. I expected it to reboot at 06:18 but it didn't. At 6:20 I plugged in a monitor... odd, screen is black. Plug in the keyboard, hit a key, EH, there's my login prompt. I stop the watch process in my ssh terminal, log in on the real terminal, make sure everything is fine, and logout again. I wait, I set a timer on my phone this time because I have an idea. It is 6:22 now.
10 minutes later, 6:32 the screen goes black. 2 seconds later it reboots.
Now the 6:32 session, I started "watch -n 1 uptime" and waited, this time with monitor plugged in. 10 minutes later the screen went black. My watch continues going, 10 seconds, 20 seconds, 30 seconds. I stop it. 5 seconds later, reboot.
Since then I have discovered the following:
1) The screen shuts off after 10 minutes of inactivity at the main terminal. If I log in through ssh it still shuts the screen off. If I plug in a keyboard and log in then 10 minute timer starts from the last key stroke. Conclusion: power management, even in minimal, non-gui mode, is shutting off my screen. Logging in via ssh obviously does not change this.
2) This causes my computer to reboot for some reason. May be directly related, may be a side effect of something tangentially related, I don't know.
3) The reboot can be mitigated for 3 minutes (2 minutes 55 seconds, consistently) by running a process in an ssh session; instead of rebooting at the 10 minute mark when the screen turns off, it will reboot at 13 minutes with the screen being off for the last 3 minutes. If, after the first 10 minutes and the screen going black, I stop this process, within 10 seconds it will reboot.
4) I can combine these two facts: run a process in an ssh session, hit a key on the "real" keyboard within 13 minutes (screen going black after 10 minutes and wakes up when the key is pressed), the apparent 10 (13) minute countdown begins again.
5) I can log in on the main terminal, run "setterm -blank 0", and this solves everything: the screen no longer goes black and the PC no longer reboots. Doesn't matter if there's an ssh session running a watch process or not, 10 minutes or 13 minutes or 20 minutes, it just keep going.
6) I can "setterm -blank 1" and trigger the blank/reboot in 1 (or 4) minutes.
1) Why does it blank after 10 minutes? And how can I properly disable this? I'm inundated with advice on how to change it in a graphical environment, but as mentioned, this is meant to be headless, no GUI is installed and I don't intend to install one. One suggestion I found was to add "setterm -blank 0" to the rc.local file but that doesn't seem to work (even after fixing everything so it actually runs)
2) What magic is going on that a watch process in a separate session can keep it going for an extra 2 minutes 55 seconds?
3) Why does it suddenly decide to reboot within seconds of the watch process stopping?
I feel like I should just wipe the drive and start over again but this feels like too big of a puzzle to solve; if I wipe it I may never know!
I don't even know how to go about debugging this further. Here's all the things it does in these controlled environments... WHY?