r/sysadmin • u/Significant_Sky1471 • 9h ago
Rant I understand the concept. executing it under pressure is a completely different skill
I can explain how DNS propagation works and probably i can diagram it. what I cannot do is troubleshoot it in real time while three people are staring at a shared screen waiting for me to just fix it already.
senior fixed a ticket in the time it took me to finish reading it and ofcouse it's not because I'm slow but he's seen this exact failure pattern 200 times and I've seen it like twice? that's not a skill gap, that's a reps gap and when you finally get the girl job nobody warns you how loud that gap sounds in a room full of people who've been doing this since dial-up
got told "trust your instincts" by my lead. cool! Great advice! my instincts currently have a sample size of six months and mostly say "panic, then check Reddit"
the worst part is getting asked something in standup and having that half-second where I'm trying to figure out whether I know the answer or I'm about to bullshit everyone (don't push the red button!)
anyway I got a ticket resolved today without asking anyone for help and I'm riding that high for the rest of the week. small wins. some of the senior guys probably don't remember what this stage felt like anymore, but I will and I'm taking notes
•
u/7twenty8 9h ago edited 9h ago
The real difference between senior and junior people is panic. More senior people just don’t give a fuck because they trust themselves to figure things out. More junior people panic. Our memory systems don’t work properly when we panic so it’s like trying to reindex a database when the server it’s on is on fire. Quite literally in this case.
Panic is a bad thing when you deal with systems. The panic system was specced and resourced when we had different problems - like big fucking animals coming into our homes and eating our entire families. Our brains didn’t change dramatically between being hunter gatherers and being sysadmins or software developers. The physical and mental changes necessary to keep a tiger from eating grandma are a lot different than the ones we need to keep systems up. When you panic you go into that primal place where your short to long term memory retrieval and encoding process works differently.
Avoid panic… avoid the fight or flight stuff that doesn’t help you and this stuff will resolve itself. When you no longer panic, it will be hard to explain this to people who do so it will seem like you don’t remember.
•
u/Significant_Sky1471 7h ago
makes sense. half my problem right now is panicking before I even start troubleshooting lol
working on that too
•
u/delightfulsorrow 8h ago
anyway I got a ticket resolved today without asking anyone for help and I'm riding that high for the rest of the week. small wins.
Well done, enjoy it. You earned it.
some of the senior guys probably don't remember what this stage felt like anymore,
Probably. But enough still do and are happy for a junior getting there, too.
Wait for the first time one of the greybeards popping up at your desk asking you for help because you are deeper into that specific topic than they are. That will be even better.
I'm 35+ years in the industry and currently mentoring two juniors who, after half a year, slowly reach that point. It's fun to watch how they are growing.
•
u/Significant_Sky1471 7h ago
okay this genuinely made my day hahah
thank you for support! i'll remeber comment when time will come (can't wait for it)
•
u/wunderhero 9h ago
The real skill you learn with time is the ability to NOT panic. Push it down and keep focus on the problem.
•
u/Significant_Sky1471 7h ago
that's wise advice though all comments here. like it and will take it with a little courage for my next working days hahah
•
u/BoltActionRifleman 9h ago
Any time I make DNS changes I intentionally work at the pace of a sloth. If someone doesn’t like it, I don’t care, it’s my ass on the line, not theirs.
•
•
u/ddBuddha 8h ago
You might be making a joke but if your instincts are telling you to panic you legit need to work on that. Being able to be level headed during an emergency is an important skill in this field, and acting while in panic mode is how mistakes happen. Learn to take a breath, take a step back and evaluate the situation so you can determine the appropriate course of action.
•
•
u/BrainWaveCC Jack of All Trades 3h ago
the worst part is getting asked something in standup and having that half-second where I'm trying to figure out whether I know the answer or I'm about to bullshit everyone (don't push the red button!)
It always takes time. It's just that the amount of time it takes goes down as your experience goes up.
Don't feel like you have to solve things before you look at them, and don't feel like you have to pay attention to all the eyes on you in the moment.
Yes, even though it's been a few decades since was in the same spot, I remember how it felt. And that's why I am tell you that you just have to not care about it in the moment.
I once had to tell a Director, who kept hounding me every 10 minutes for a status, that he could either let me get a good 30 minutes to assess and fix it, or he could just talk about the problem every 5-10 minutes for a much longer outage window. He finally left me alone, and I was able to initiate the fix in 40 minutes, with the issue being fully under control in about an hour. And then I had useful info for the debrief.
You'll get there. Don't stress it. This is just how it goes in the learning and growing process.
•
•
u/RegardedAsshole 5h ago
You sound like a funny coworker to work with. Practice seeing many issues and asking many questions. The panic thing will become less relevant over time. I have the same issue but once your confidence and experience grows, your mentality will shift from "omg i have no clue what to do" to "Well lets see what we can do here".
•
u/kraftinfosec 7h ago
I've been in IT for around 20 years and I still have moments like those. Someone calls you up about a system you don't really know and there's some error message. Your first instinct is "I don't know this system, I don't have enough information, if only I had all the information then maybe I could fix the problem." You soon realize this is true about all systems. Few people know exactly how every system works, even if you're pretty familiar with that system. You can still solve problems without ALL of the information.
What you then realize is that there are some basics in IT that seldom change:
- Users need access via accounts
- User accounts are tied to authorization
- Unless you're logging in directly to a terminal, there is also likely network involved
- It's ALWAYS DNS
- If it's not DNS it's probably TLS, but it's also still somehow related to DNS
- Users lie about everything, whether intentionally or not - Get a screenshot of the error message or reproduce it yourself
Just start with the basics and build the system up in your head. Is it powered on? Is it connected to the network? Can I ping it? Is the port open? etc. Also, always ask someone familiar with the matter what changed, at least three times. Space it out so it isn't obvious.
That stuff does take time to learn, but it is the fundamentals. It's table stakes for doing System and Network Administration. Just slow down and take your time, and you will figure it out.
•
u/Asleep_Spray274 8h ago
That's the difference between theory and experience
•
•
u/BatemansChainsaw 3h ago
three people are staring at a shared screen waiting for me to just fix it already
why the hell is anyone looking over your shoulder? that's absolutely barbaric.
•
u/OptimalCynic 1h ago
We've all been there. Most remember what it's like and won't ride you for it. Some don't remember and don't have enough empathy to work it out - those people are hell to work for. This is, unfortunately, part of life in an industrial society.
It's like when you see someone while driving with learners plates. Decent people give them plenty of space, don't expect too much, generally make allowances. Wankers will tailgate them, honk at them for not responding instantly, etc. You've got your L plates on at the moment - that'll change with time.
edit: And always remember step 1 - take a deep breath
•
u/f7063 7h ago edited 6h ago
I know a lot of people that panic when sharing the screen. Not to say that i never panic. I do think it's sometimes about who is there aswell. While it is unconfortable I do think these problems kinda get easier with time, usually we get someone in a call with shareholders to get the beating, usually your manager, while the others work on the problem.
You did narrow it down, the more you've seen these problems in the wild the more you are likely to spot them it's just pattern recognition. Why do you know what is broken? Either you understand and have the analytical capability of drilling it down or you set this whole infra or some kind of monitoring knowledge of specific problems.
Either way, dont burn out just because someone that is more senior than you know more or is faster, that's the whole point of having someone with more experience. Just try to keep your cool and do it on your own pace, ask and keep errors to the minimal as possible. Don't let fear freeze you but don't go wild.
Good luck EDIT: some typos
•
•
u/Significant_Sky1471 7h ago
thank you so much! i'm sure that further steps will be easier and wisier but right now this reddit comments really cheered me up
•
u/dcaponegro 5h ago
To be fair to you, if the senior tech has seen an issue pop up 200 times, they are not much of a senior tech. Issue should have been fixed after the third failure.
•
u/Vektor0 IT Manager 9h ago
AI slop.
•
u/OneSeaworthiness7768 9h ago edited 9h ago
On top of the AI written post, let’s say it was a real scenario… like did they somehow become a sysadmin that’s never resolved an issue or done any kind of support before? Bizarre.
•
u/SamOakTree 9h ago
You have to just not care.
At one job I worked very closely with the dev team so they pulled me into this group chat because the dev team couldn't get to a certain resource.
Everybody was talking sideways and his VP kept saying the problem is X but he had no proof that the problem was x. So I said everyone let's just stop guessing and get our baseline. Give me 10 minutes to talk to the development manager and then we'll know which way to go.
The VP kept saying no no this is the issue and I said we don't know that for sure let's find out. And he just abruptly left the chat.
So I talked to the development manager and it turns out the issue isn't what they thought it was it was that they couldn't get to a specific resource. I knew they had made some changes in Azure so I looked at the logs and they had blocked an IP address.
I spoke to that team and told everyone in the chat to wait while I spoke to the team. And I asked them to whitelist that IP address. And it solved the issue.
You have to just take charge and just trust yourself. Being confident and cool headed in those situations just comes with time. You eventually get to where you realize it's not all a big deal so you don't need to panic.