After self hosting several services for a few users, with SSO, backups, hardware issues etc, I really appreciate how good the IT was in my old company. Everything was connected, smooth, slick and you could tell it was secure. I had very few issues and when I did, they were quickly solved. Doing this all at scale for thousands of employees spread across the world, it is a wonderful sight to see.
Now at my current company, it’s at the opposite end of the scale where I almost believe that I could do a better job by myself! They’ve trying to do everything you would expect but somehow doing it wrong. They are so heavy on security I have a Citrix environment that takes me 3 logins to get to, fails constantly and means I can’t work without internet (like on a long train journey for work purposes recently), and on the other hand they’ve only just turned off admin rights for users so we could’ve installed anything we wanted!!! All our attachments (incoming and outgoing) are saved to a secure website (like OneDrive) and replaced with a link. It doesn’t save the file names on the email so it’s really tricky to find old emails if it’s a document you’re looking for. I could go on but just venting at this point as it’s so frustrating!!!
Thank you to the good IT people out there. Your roles are so important but not appreciated enough!
They are so heavy on security I have a Citrix environment that takes me 3 logins
My daily routine:
- Take laptop out of locked shelf
- Start Laptop and enter boot password
- Enter Bitlocker password
- Enter username (not saved) and password
- Open Citrix website and login with different username and password
- Enter MFA token to access said website
- Start server connection
- Enter different username/password (not saved) to access server
- Enter different MFA token for the server login
- Start the business-specific application with 3rd set of not saved and different login data
They also have plans to make MFA mandatory for laptop login, too.
Passwords need to be at least 15 characters long for laptops and 30 for servers and 10 for the business-specific application. All need to have uppercase, lowercase, numbers, and special characters and need to be changed every 60 days (for the server login) and cannot be the last 30 passwords.
Fucking hell.
And then they wonder that people resort to easily predictable patterns such as !1Qaz@2Wsx#3Edc and simply shift it one position to the right with every forced change and repeat at the end of the keyboard.
Some users have a barcode scanner connected to the system for doing the business stuff. The barcode scanner registers as HID keyboad …
Yes, they did exactly what you think.
Smart. I’ve seen it on manufacturing lines for operators logging into SAP. They put the barcode on the back of their badge.
That make the badges NFC tags but without actual NFC …
At least they had the code not in direct sight on their desk.
This is advanced post-it under keyboard level
Wint3r!2024
I see we work with the same people.
Tell them to move to yubikey or similar hardware key which is far more secure than any password policy will ever be and vastly more user friendly. Only downside is the intense shame if you manage to lose it.
The key should stick with the user thus not be stored with the computer when not in use. The key isn’t harmless of course but it takes a very deliberate targeting and advance knowledge about what it goes to and how it can be used. It’s also easy to remote revoke. If you’re extra special paranoid you could of course store the key locked at a separate site if you want nuclear codes levels of security.
This insane torture is why there are post-it notes under the keyboards.
And they believe all employees actually remember so many wildly different and long passwords, and change them regularly to wildly different ones? All this leads to is a single password that barely makes it over the minimum requirements, and a suffix for the stage (like 1 for boot, 2 for bitlocker etc), and then another suffix for the month they changed it. All of that then on sticky notes on the screen.
I’ve seen plenty of solutions. Sticky notes, a simple text file. External tools like barcode scanners. Using all letters and just
1!
at the end (not that this is less secure on technical level than a completely random string, but it’s easier to bruteforce - theoretically), etc. Some people use KeePass (with a stupid 5 letter password).
Yubi keys… for all logins, would solve this mess, geez.
Hahah, omg don’t they realize people are writing that shit down?
I’m big on proper security for business, but holy cow that’s nuts.
Guess I’ve been fortunate to work at some huge, well-known orgs that also really understood how to do these things.
One, in the 90’s,had already developed an early form of SSO for the 20 backend systems that all had unique username and password requirements. Their call center agents really appreciated it.
The early US phone numbers were from a set of a few names, and the 5 digits, because they thought people couldn’t remember 7 digits.
And now people have to remember several sets of at least 8 random symbols, and change them every 60 days.
Modern day Kafka.
This sounds like my old place, but much worse.
We used to have laptops we had to lock in a cabinet (yeah, one of those cabinets with a really puny lock that’s easy to pick). And we had to log into n old mainframe system that had numerous environment instances which each required a unique password that had to be changed every 90 days.
We (the software devs) basically rebelled on the laptop situation and insisted they find a better solution. Thankfully they changed policy and of allowed the laptops to be locked into our docking stations, which in turn were locked to our desks.
As for the mainframe system credential management, I tried using a standard third party password manager, but a) it wasn’t a good fit for the credentials, and b) the sys admins or security team forcibly uninstalled it because it wasn’t sanctioned software (even though it was a well-respected and actively maintained one). And our security group refused to go out and find one.
So being a dev, I wrote my own desktop password manager for the mainframe credentials. It was decently secure, but nowhere near as secure as a retail password manager. But it fit the quirks of the mainframe credentials requirements. And after my colleagues and manager did a code review of it, it was considered internal software, and thus fit for use.
As I was leaving they were in the process of removing all our local admin rights (without a clear path on how to accommodate for us developers debugging code - fun times ahead!).
But all of those annoyances pale in comparison to the shit you are having to deal with! Holy hell, that sounds like pure misery! I’m sorry.
Temporary workaround applications/scripts become de-facto standards sounds familiar. They disabled loading script files in Powershell but you can still copy&paste the file’s content …
People have no idea how absurd IT in corporations is.
I was going.to auggest a hardware key, but I fear they would add it as another layer rather than as a replacement for this nonsense
And I guarantee every one of those passwords are written on a piece of paper at the desk under the keyboard.
This is very close to my workplace but we have about 17 domains to work across, with a separate account for each. It’s frustrating sometimes, but in the end I get paid the same either way.
Ladies and gentlemen, we have a winner!
My advice for this company: fire 2/3 of all IT staff (including managers). Then tell the remaining ones to cut off unneccessary things and do it better in the future.
Big international corporate, IT security hired by personal connections instead of skill, IT security never worked in daily business.
The fun thing is, that they refer to NIST guidelines. Which is even funnier because NIST says 12 digits are enough, user-generated 8 digits are fine, no complexity rules, and password changes only “when necessary” (i.e. security breaches).
Or they work in a regulated industry that requires pseudo-airgapped machines for remote users, e.g. the machine actually interacting with the systems needs to be within the controlled boundary but the company has a presence in multiple locations, so the solution is to have a Citrix server that the users remote into. But because the SSP also has access control requirements at every stage that take a long time to get updated to newest industry standards, the user still needs to have passwords rotated, MFA, and all that kaboodle.
My favorite is when IT deploys software that replaces all the links in your e-mails with
https://example.com/phishing/YiCdMdsY
so you can’t tell whether the e-mail is phishing or not, frequently sends you very obvious fake phishing e-mails that interrupt your work by going straight to your priority inbox, and punishes anyone caught clicking on phishing e-mails. Then HR sends out e-mails that have all the indicators of low effort phishing and you’re supposed to click on those.Omg, my previous company did the same. But you missed a part. If you accidentally left out a real email, thinking it’s a scam, then the client will file a complaint.
I always click to see if the phishing email is real or not!
New action items have been assigned to you:
- Remedial cybersecurity training (4hr): due by Mar 22
Had one of those. Very convincing. Showed my boss. My boss also thought I could be real. So I clicked it. The landing page was an internal “you’ve been caught” page. Then I got the phishing-email training assignment.
Gallagher were great at that, rubbish solution for “teaching” staff about phishing which would infuriate all staff caught in the net. Would come from internal email addresses too which, if one person’s email / credentials are compromised they’ve got bigger fish to fry.
Broken company mechanics in a nut shell
Most IT departments have the mindset of avoiding troubles instead of making things easy for users. They don’t want to get blamed for security incidents. They want things to be predictable and within control.
They sacrifice a lot of user convenience doing that. On the other hand, IT giants are enshittificating IT services.
how do you think IT giants are enshittifying their services?
Who said security ever had to be difficult for the end-users? The companies that charge $15k per month per service to keep your company audit-ready. Oh and Microsoft is one of the more “seamless” providers for auth and security services out there, amazing.
I am a former IT Desktop drone…er…support worker… I used to swap towers for my local municipality back when Windows XP was being replaced with 7. I saw passwords on post-its attached to the monitor, mouse pad, and even under the keyboard or keyboard drawer (I had to get under desks to do the swap). Our policy was to remove those whenever we saw them and trash them in a different can across the building or a different one. They have a standard 90 day password cycle and most people couldn’t handle that. I would answer the phone often to 'unlock" their account after 3 attempts. My all time favorite when I would help an end user with software was when I would encounter someone’s “God Mode” icon for some of the registry hacks that used to float around. Everyone had Admin privileges (ironically), so it wasn’t really needed anyway.
Their primary server admins and IT folks in the main office were Top notch though. Never any downtime and the main security guy was very strong in making sure everything was adhered to. We, as desktop support didn’t have the master password to decrypt a laptop which was GPG protected and had to bring it to him if we had a user which locked themselves out. With great consternation, only a few machines would be allowed to XP and those were VLAN’d and isolated from the outside world.
The rest of the server admins handled everything with ease seemingly. The fun part was when they had a third party come in and do a security audit. No problems on the server side, but it wasn’t a success. They did the 'ol drop a flash drive randomly in different locations test. Knowing human nature, they knew someone would pick it up, plug it in and be baited with an excel file which looked like it had financials. Unbeknownst to the user, it sent a ping to their reporting server and the drive ID. Which was later reported back. They also did physical security penetration tests - walk in behind you type of thing. I remember seeing a group of guys non company ID badges try to follow me into the main IT office. I stopped them and asked who they were and what they wanted (this was a Govt building), and the look of confusion mixed with satisfaction from them that I stopped them was priceless. I let the head IT guy know who was at the door and left it up to them to unlock it for them.
I now work in a help desk position for a software company and miss those days of desktop support. But, I know for a fact that I.T. Guys an Gals don’t get enough recognition. They are the understated backbone of a company’s well-being especially when holidays and weekends are prime time for systems to fail and they are practically on call no matter what.
When we do it right, you forget we’re here and we’re ok with that.
I’ve been in IT for a couple decades at this point. I stopped doing almost any swlf-hosted stuff years ago as I just don’t have the time or energy to deal with things. There’s a lot to keep up on with technologies, security, etc. not to mention all the constants of keeping things up-to-date, back-ups, troubleshooting issues, and more
True words. The sustained effort to keep something in decent shape over years is not to be underestimated. Now when life changes and one is not able or willing anymore to invest that amount of time, ill-timed issues can become quite the burden. At one point I decided to cut down on that by doing a better founded setup, that does backup with easy rollback automatically, and updates semi-automatically. I rely on my server(s), and all from having this idea to having it decently implemented took me a number of months. Just because time for such activities is limited, and getting a complex and intertwined system like this reliably and fault tolerant automated and monitored is simply something else than spinning up a one off service
Compliance and money. Probably has little to do with the team itself and more about the business needs.
My experience with my company is exact opposite. Apparently Bitwarden and Vivaldi are not allowed because they have a lot of vulnerabilities so people should continue using edge/chrome and a plain text for storing all their passwords that they often show it on screen share. Had an issue with 2FA cause those assholes decided it’s fun to force the Microsoft propriety authenticator for everyone so I can’t use aegis anymore. That issue took a whole fucking month to get resolved cause none of them could comprehend their almighty Microsoft app didn’t work on my grapheneos. On a unrelated note, anyone got any openings at your company?
My new workplace has a web portal to give jobs to the maintenance team, and the web browser says “Insecure” in the corner.
My self hosted stuff has that too. Difference is I plan to learn how to do something about it
I recently set up DNSSEC on my home domain, and I have been shocked to learn that none of my financial institutions use it. Going back through my logs, the only external host that even tries is api.weather.gov, and only for the CNAME pointing to a CDN host.
I mean it’s not the worst. Is it still https? Or are they serving plain ol http? My internal services (at home) are mostly https, but the certs are self signed so browsers will flag them as “insecure”.
I just clicked on it to see. “This site does not have a certificate”
My internal servers are the same, all http. But I started self hosting 2 weeks ago…
How could you tell it was secure?
Lots of little things really. Obviously I couldn’t say for certain but they seemed to on top of it without causing us too much difficulty in doing our jobs.
Sometimes things were blocked like if a new email, or questioned after to check it was expected and followed policy. Policies were clear, and there were helpful prompts or warnings.
We were involved in something where we had to copy a sh*t load of files from a shared folder to a hard disk. There were like three automatic blocks that kicked in at different times, which was a pain at first to figure out but because we had a good reason, someone in IT just kept at it to get it done and looking back, that should have raised flags given the size of it all.
They changed from passwords changing every 6 months to no changes but had to be longer and mandatory 2FA. We were told to use keepass for all passwords for things that weren’t SSO for various reasons.