journalctl Basics

I have mixed feelings about Systemd - I have to admit I love the service files, which are ecstatically easy compared to init scripts. But I have significant issues with how it handles logging (binary logs? really?) and the whole one-service-to-rule-them-all attitude, which goes entirely against the Unix "do one thing and do it well" philosophy. That said, it's pretty much a lost battle: the only hold-out distributions are minor ones like Devuan which I suspect will have a long but very thin tail (ie. it will be around a while, but not have very many users).

So let's get used to the tools we have to use. My desktop is crashing, and there's no such thing as a /var/log/syslog anymore. That's simple enough: journalctl. This will give you the jumbled log of everything that used to go into syslog, but journalctl adds a bunch of nice twists. Digging through the logs to the time of the crash can be simplified by saying journalctl -b -k -1 which essentially means "show me only kernel messages ('-k'), from a specific boot ('-b'), and make it the previous boot ('-1')." journalctl also nicely colourizes the output so you can see what's not important (gray), important (white), or really important/disastrous (red). There are numerous other interesting looking switches, such as ways to filter on a given service or -e which jumps immediately to the end of the output. man journalctl is your friend.

journalctl automatically pipes all its output to less: I love less, but they use a feature I despise. They unwrap the text, which forces you to scroll sideways to see long lines. I actively dislike that behaviour on its own, but where it goes completely off the rails as far is I'm concerned is that it breaks cut-and-paste. To fix this seriously evil behaviour, you can set an environment variable: export SYSTEMD_LESS="FRXMK". The default (if this value isn't exported) is "FRSXMK", where the 'S' is responsible for unwrapping lines. If you prefer this behaviour, add the export ... line to your ~/.bashrc or system profile.

Finally, if you'd like to actually keep a copy for your records or to email so others can see the fun involved in your crashes (my use case), journalctl -k -b -1 --no-pager > badboot.txt.