this post was submitted on 09 Jan 2025
555 points (98.9% liked)
Programmer Humor
32710 readers
664 users here now
Post funny things about programming here! (Or just rant about your favourite programming language.)
Rules:
- Posts must be relevant to programming, programmers, or computer science.
- No NSFW content.
- Jokes must be in good taste. No hate speech, bigotry, etc.
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
As a data engineer, testing with production loads is critical to performance checking, as well as finding edge cases where your assumptions about what can be expected in the data are curb stomped and send you back to the drawing board to cry and think about what you've done.
Yeah we finally set up a workflow where we get production data available in a staging environment. This has saved a lot of trouble via "well it worked on my local where there were 100 records, but prod has 1037492 and it does not"
Same. Early on as a new dev, I failed to performance check my script (as did my qa tester) before it was released to production, and that was my first roll back ever. It was very unoptimized and incredibly slow under one of our highest density data streams. Felt like an idiot that I was good with it's 1-2 second execution time in the dev environment.
I deal with this constantly. Profilers are your friend. I keep begging my team to use the database dumps from production to test with, but nope. Don't feel bad about messing up though. The amount of fuck ups I deal with in prod is exasperating. At least most of the things I break is a quick 5 minute fix and not weeks of rework.
The hardest thing I have explaining to the team is the concept of time. Once you have done controls programming and get to witness how much happens in 50-100ms, it sinks in. Your thing takes 500ms? 1 second? They think this is acceptable on something that is dealing with less than 100 database records. 😭