Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I think the next level above this might be https://sadservers.com/


Oh hi, author here, AMA :-)

I'm happy that a lot of people use and find SadServers beneficial while not costing me a lot of money. Still a lot of features to implement on the website and a shrinking backlog of scenario ideas to materialize (if somebody has ideas for Linux/Docker/Kubernetes etc troubleshooting scenarios, please let me know).


I've just tried several of your challenges and they're all painfully accurate for real-world scenarios. I will definitely point people at these next time I'll get asked how I learned to fix <random Linux configuration problem>!

As for suggestions, here are some random things I needed to do recently:

- resize the boot partition of an OS (don't know how doable this is with your vserver setup, maybe use one of those WASM Linux emulators?)

- set up a systemd service/timer/socket that starts at the right time and responds correctly to reloads/restarts

- set up IPv6 correctly

- troubleshoot why a device wasn't connecting to the WiFi (DHCP service problem!)

- set up a VPN (wireguard/openvpn/etc). Expert mode: make the remote endpoint have an A/AAAA record that the server isn't listening on

- troubleshoot why some of my devices couldn't ssh into a server despite the pubkeys being in the authorized_keys folder (old sshd version didn't understand the most recent key algorithm!). Bonus problem: ~/.ssh had the wrong permissions so the authorized keys weren't loading.

- renew an ACME/letsencrypt certificate in nginx in proxy mode (location / was proxied but location /.well-known/... shouldn't have been!)

- check your preferred smtp daemon to see if it's set as an open relay

- upgrade postgres from an old version to a new version without data loss (hard mode: the partition postgres uses by default doesn't have the free space to make a copy and migrate the data)

- figure out why the firewall isn't blocking port 1234 despite UFW being enabled and a block-all rule being present (it was because of Docker iptables rules overriding UFW rules)

- update a package that has some kind of dependency issue (i.e. an external repository that is no longer needed)

- make Ubuntu shut up about Ubuntu Pro and stop it from fetching ads on ssh login

- alter a systemd service file so that it no longer runs as root (hard mode: set up dynamic users and other hardening features)


thanks! (some of these ideas are not possible to implement in the current setup but I can explore some of the other ones)


Discussion: https://news.ycombinator.com/item?id=33344142 (597 points | 3 months ago | 128 comments)


hehe..

lsof | grep bad.log

ps -ef | grep (pid)

sudo apt remove python3

kill -9 (pid)


I got lucky:

ps ax showed a python3 process running /home/ubuntu/badlog.py, so I knew to kill that. If it wasn't named so obviously it would've been much harder to find it.


lsof is list open files. Good for finding things like what process is writing to a log file. You could also go dig around in /proc




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: