Sysadmin

They hired a new hotshot engineering manager (the kind that makes physical things). He hates the engineering software we run. I don't blame him, it's crap software. He constantly complains about how slow it its. He's right again. Crap Software Vendor says it's my platform that makes their software slow and buggy. I'm willing to make any changes they recommend, but they've got nothing. They're like, "it runs fine in our test env." So hotshot goes rogue and signs contracts to move engineering to a cloud platform that he used at his old job. I wasn't brought in until after the ink dried.

New vendor sends me a link, login, and password via email. I go to the link. It's fucking remote desktop gateway. Open to the internet. The password isn't a temp, that's my permanent unchangeable password. This is how they handle user access control. No MFA. Nothing between the screaming void and our data but IIS and an AD password.

So I start pissing in the tent. I tell everyone this is unacceptable security for our IP. Vendor acknowledges that their security is insufficient and lays out their roadmap to fix it, hopefully by the end of year(I'm holding my breath). I ask if we can just run the software ourselves.

I have a convo with our CEO who usually listens to my advice. He asks if we can just host the new software on our platform (the one that already has MFA and a whole lot of other security measures). I say, "That's exactly what I was thinking." So, CEO email in hand I go back to the group and tell them to make preparations to move the implementation to our platform.

Hotshot starts bitching and moaning about how he doesn't want another slow app. A data analyst chimes in with her two cents out of fucking nowhere. I'm not even sure why she's on the email chain. I'm about two seconds away from going Joe Pesci on these goombas.

What the fuck guys? Who cares if the app is slower on our platform (not that it necessarily will be)? What good is a fast app that's insecure? How fast is it gonna be when it's ransomwared to hell? It'll be nice that the app is fast when BianLian is downloading all our designs so they can extort us.

"Well they're a big company and they haven't gotten hacked yet?" Thanks for that Captain Smith, but I know a fucking iceberg when I see one.

I plan to move an external DNS server to a hosted VPS in the near future. I would appreciate advice on VPS specs for this purpose, or an other helpful feedback from others who have done this before. I've used a lot of low end boxes to host web services, and would like to do that with this project as well but don't want to under spec it. It will be used regularly by around 300 users.

Recently I had the pleasure of troubleshooting why all of the computers in the office had broken secure tunnels. It turns out that Microsoft pushed out an update that breaks Samba AD.

The good news is the Samba team was very fast to put out a fix. The patch has already been pushed out to Debian and I'm sure its in many others.

Its a pity it took me several hours on a Friday to figure out what had gone wrong.

My company is about to shift a large workload to a vendor that uses an RD Gateway hosted at Amazon to serve access to the front-end application. It's open to the internet at 443. There's no MFA. How worried should I be?

IIRC correctly, the free version allows to manage 10 remote devices. It should be enough to see how it works and whether it's the solution for your specific environment.

Is it just me or is the learning curve a lot greater with Zabbix? The error messages seem extremely vague or completely useless. The web GUI fields don't have proper validation. They moved a lot of things around in the 6.4 version and now googling a solution gives me out of date info. The template network sensors are picking up about 20 ethernet interfaces on windows VMs in HyperV and I cant select just the one that I want to monitor (I guess I have to write my own sensor for that?).

I was demo'd Zabbix by a friend who has 39k+ sensors working on less hardware than my 1000 sensors use in PRTG, and the price difference is huge... So I really want this to work for me but I spent the whole day today feeling uneasy about it.

What are your guys thoughts?

I recently have been playing around withTacticalRMM and I am very impressed. I have tried many different products but all of them has either had a minimum of 100 endpoints or been complete garbage (sometimes both)

With tacticalRMM I can manage the 25 devices just fine. It have built in update policies, automated checking, automated tasks, asset tracking and a bunch of other features.

What I really like about it is the test scripts that can trigger actions. In my environment I have legacy software that is prone to breakage. With TacticalRMM I was able to automate the restarting and fixing of the issues as the arise which saved me some serious headaches.

I'm sorry about this post sounding like a sales pitch but I really like the software. Also if your installing go for the docker install as the standard install didn't work reliability

According to Microsoft, the compromised key was inactive and therefore any access token signed by this key must be considered suspicious.

Unfortunately, there is a lack of standardized practices when it comes to application-specific logging. Therefore, in most cases, application owners do not have detailed logs containing the raw access token or its signing key. As a result, identifying and investigating such events can prove exceedingly challenging for app owners.

In the Team’s principal meeting for the F1 Hungary F1 race, a journalist asked the question of more or less AI taking over positions. It was interesting to hear the responses from the F1 Team Principals. What does the Lemmy or Ex-Reddit Sysadmin community think of the responses?

Is it just me or are system requirements by vendor applications getting out of hand? In the past 5 years I've watched the minimum specs go from 2vCPU or 4vCPU with 8GB or 16GB RAM now up to a minimum of 24vCPU's and 84GB of RAM!

What the actual hell?

We run a VERY efficient shop where I work. Our VM infrastructure is constantly monitored for services or VM's that are using more resources than they need. We have 100+ VM's running across 4 nodes, each with 2TB of RAM and 32 cores. If we find an application that is abusing CPU usage, or RAM consumption, we will tune it so it's as efficient as can be. However, for vendor solutions where they provide a VM image to deploy, or they install a custom software suite on the VM, the requirements and the performance have been getting absolutely out of hand.

I just received a request to deploy a new VM that is going to be used for managing and provisioning switch ports on some new networking gear. The vendor has provided a document with their minimum requirements for this.

24 vCPU's 84GB of RAM 600GB HDD with a minimum I/O speed of 200MB/s

I've worked as a System Administrator for a long time. One thing I've learned is that a measure of a company's product is not only how well it functions and how well it does what it advertises, but also how well it's built. This includes system resource usage and requirements.

When I see system requirements like the ones I was just given, it really makes me call into question the quality of the development team and the quality of the product. For what it's supposed to do, and what the minimum specs are, it doesn't make sense. It's like they ran into a performance bottleneck somewhere along the line, and instead of diagnosing and fixing the code to be more efficient, they just pulled a Jeremy Clarckson and added "More power!". Because throwing more CPU's and RAM at a performance issue always fixes it. Lets just pass the issue along to our customers and make them use more of their infrastructure resources to fix our problem. Jeez!

Just to be clear, I'm not making a blanket statement about all developers, there are a lot of developers or development teams that do put quite a bit of effort into refining their product and making it quite efficient, however it just seems more common place now that these "basic" applications from very large vendors have absurd system requirements.

Is anyone else experiencing this? Any similar stories to share?

We recently had a meeting with our new (as in 4th in a year) rep. They let us know ROBO licensing is moving away from the VM Pack method it is now to per socket licensing. Minimum of 16 core per socket purchase, and you can't stretch a license across multiple cores.

We about blew a gasket when we were told this. It is going to make our ROBO license jump from about $2K up to $30K PER YEAR. We were told changes to Ent+ are coming too, but details were not known. We are in the process of looking at how moving to another option would look like. Either Hyper-V or Nutanix AHV.

I guess we can see how Broadcom is making their money back. By screwing over their customers.

Good news, everyone!! Microsoft outage this morning in the UK. The tickets are streaming in..

Got any useful blogs/guides to share? Videos, tips and tricks, other useful resources.

There are apparently known problems (GitHub Issues #5807) in both Edge and Firefox with this newly released browser extension update (2023.7.0), which should hopefully be fixed soon.

We're happy to announce the release of BusKill v0.7.0!

Most importantly, this release allows you to arm the BusKill GUI app such that it shuts-down your computer when the BusKill cable's connection to the computer is severed.

What is BusKill?

BusKill is a laptop kill-cord. It's a USB cable with a magnetic breakaway that you attach to your body and connect to your computer.

Watch the BusKill Explainer Video for more info youtube.com/v/qPwyoD_cQR4

If the connection between you to your computer is severed, then your device will lock, shutdown, or shred its encryption keys -- thus keeping your encrypted data safe from thieves that steal your device.

Upgrading

You can upgrade your BusKill app to the latest version either by

1. Clicking "Update" in the app or
2. Downloading it from GitHub

Changes

This update includes many bug fixes and new features, including:

1. Adds support for 'soft-shutdown' trigger to GUI
2. Adds a new buskill.ini config file
3. Adds a new "Settings" screen in GUI
4. Merges kivy & buskill config files into one standardized location
5. Fixes in-app updates on MacOS
6. Fixes lockscreen trigger on Linux Mint Cinnamon
7. Fixes background blue/red disarm/arm color to propagate to all screens
8. Fixes --run-trigger to be executed inside usb_handler child process and communicate to root_child through the parent process

You can find our changelog here:

• https://docs.buskill.in/buskill-app/en/stable/changelog.html

Documentation Improvements

We've also made many improvements to our documentation

1. Updated the Software User Guide to include how to arm the BusKill app with the soft-shutdown trigger in the GUI
2. Added a manpage
3. Better documentation on how to build your own USB-C BusKill Cable
4. Better documentation on how to test the buskill app
5. Fixes in Release Workflow
6. Added some additional related projects to our documentation

Soft-Shutdown Trigger

This release now allows you to choose between either [a] locking your screen or [b] shutting down your computer when you arm the BusKill app from the GUI. By default, the BusKill app will trigger the lockscreen. To choose the 'soft-shutdown' trigger, open the navigation drawer, go to the Settings Screen, click Trigger, and change the selected trigger from lock-screen to soft-shutdown. For more information, see our Software GUI User Guide.

• https://docs.buskill.in/buskill-app/en/v0.7.0/software_usr/cli.html

BusKill Now in Debian!

We're also happy to announce that, with the release of Debian 12, it's now possible to install BusKill in Debian with Apt!

sudo apt-get install buskill

Testers Needed!

We do our best to test the BusKill app on Linux, Windows, and MacOS. But unfortunately it's possible that our app doesn't fully function on all versions, distributions, and flavours of these three platforms.

We could really use your help testing the BusKill app, especially if you have access to a system that's not (yet) listed in our Supported Platforms.

And in this release, we specifically would like you to help us test the new soft shutdown feature. Please let us know if it does or does not work for you.

Please contact us if you'd like to help test the BusKill app :)

This is different from last week's warning: https://www.bleepingcomputer.com/news/security/300-000-plus-fortinet-firewalls-vulnerable-to-critical-fortios-rce-bug/

Fortinet is on fire :)

Does anyone want to see sysadmin blogs/guides posted here? I used to check sysadminblogs but the quality of content used to vary a lot. Would this be helpful or just clutter the feed?

I have seen the documentation saying to build an empty VM with slightly more space for each volume than was on the physical server, then use clonezilla to create an image of the server, then import it. That seems ok, but I'm hoping someone out there has more real-world experience in doing this and can share if they did it differently, or encountered any pitfalls.

As my environment matures, I am moving from "Hey I have 1 physical server with everything on it" to "Let's use a hypervisor and spin off services onto their own." When the base OS is P2V'd, I'll be able to have 2 hypervisors and start implementing HA. I've been using this system as a scratchpad and dev box for 10 years and would love to just migrate it over.

Hello c/sysadmin, and welcome to the Patch Megathread! I'm editing this post and leaving it up as a single catch-all sticky post for patch days for the time being, since we're not seeing enough activity to warrant new threads IMO. If someone wants to help moderate / curate content and actively create new patch day posts, please let me know and I'll add you to the mod team.

This is the place to talk about the latest patches, updates, and releases. We put this thread into place to help gather all the information about this month's updates: What is fixed, what broke, what got released and should have been caught in QA, etc. We do this both to keep clutter out of the community, and provide a singular resource to read.

While this thread is timed to coincide with Microsoft's Patch Tuesday, feel free to discuss any patches, updates, and releases, regardless of the company or product.

Remember the rules of safe patching:

• Deploy to a test/dev environment before prod.
• Deploy to a pilot/test group before the whole org.
• Have a plan to roll back if something doesn't work.
• Test, test, and test!

Will this community doing the monthly patch Tuesday thread that was on Reddit? Was sometimes pretty useful