Omid's Blog

Rants on FOSS, tech, homeprod, music and other stuff.

On Single Points of Failure

Tue Apr 14 22:33:47 2026

No, that’s not a mistake. I really did mean points, plural. Let me explain!

One day I was out of town and I realized, oh no, my Matrix instance is down. Oh no, my Synapse is on the same node that runs my website (which my students (hopefully) use to study and review materials). Well not a problem, I will ssh in, and fix this in a jiffy. Oh wait, my VPN is down. Oh no. I had hardened everything down. I was not getting into my server remotely.

It was then that I realized “wow, I am getting quite reliant on my server”. That was a wake-up call for me to:

  1. upgrade my server machine into something that is preferably not old enough to vote
  2. invest in a UPS
  3. set up a redundant machine for VPN

’A’ was quite simple. I bought a second hand Emu A1-N100 Silent. It consumed about 10% of the power that my Dell R220 used. The fans are quiet enough that I can tolerate them, however I have already purchased some Noctuas that I am going to pop in the server when I get the chance. It’s not the noise that is unbearable, but the pitch of it does get on my nerves a bit. The second necessary purchase was the UPS. Since I like buying everything used, I had to search and research and wait a bit. In the end, I settled for a recently refurbished APC SMT1500RMI2U. What a catchy name!

So, everything was hooked up. The transition and restoration from my PBS backups was painless. Booting up containers took seconds, not minutes. Reboots took a few minutes, not many minutes. The NVMe was faster, the HDDs didn’t exist (so didn’t need replacing), the power consumption was lower, life was better. Although I started to miss the spinning rust noises, I realized that I was doing it for the greater good. A basic RoI checker was the little push I needed to pull the trigger. So, thank you, /u/value1338. As for the VPN situation, I set up a second subnet forwarding Tailscale node on a Raspberry Pi.

Cue today. I knew for some time that this week there would be power cut-offs. Not a problem! I am armed with a terribly overpowered UPS, and a machine that consumes about 7w. According to my (I mean the UPS’) calculation, I have an uptime of 5 hours! That’s practically forever. I was ready. This is how preppers feel every day, probably. I will face this storm as it comes and I will permit it to pass over me and through me. And when it has gone past I will turn the inne… zap. Blackout. The UPS starts beeping, I am smiling. I shut down my PC gracefully, and prepare to send a message to my little group of Matrix users: “I may need to shut down, I will let you know”. Pressing on send puts my message in a queued state. Hmm, why is Tailscale trying to activate on my phone, I am at home. Oh no… I had forgotten about the router.

A single point of failure. I can prepare everything, but if the router doesn’t get the juice it needs, there is no connection to the outside world. I had neither powered my switch, nor the router, and so understandably, we went dark. The power outage took 70 minutes (which reduced my uptime by some many nines) but when the power came back up, I thought the issue was solved till I realized that the reconnection of the power had flipped a safety fuse in our box. Even if I set everything up properly, there is a chance that things will go down, and if I am not home and if a fuse pops, there’s nothing I can do. I guess I have to be fine with that.

I did an express order for some cables for the router. I will get to connecting it to the UPS now. I have also set up an automatic shutdown feature via the NIC of the UPS to auto-shutdown my server gracefully in the event the battery goes below a certain expected available run time. Maybe I will set up a system so that it auto-starts on AC power resume. The good news is that the popped fuse was unrelated to my UPS, so hopefully I don’t have to actually deal with this problem, but today I learned that it is possible, and that I am not a big data center, and that downtime is fine. Maybe even the internet will go down, and I still haven’t set up a cellular backup. As long as I have higher uptimes than GitHub, I will be happy.

Theme: Dracula  /  Fonts: Mononoki Nerd Font, Inter /  Der CSS dieser Website wurde mit Unterstützung von KI erstellt.
Meine Haltung zu LLMs und KI / Impressum / RSS