How do you actually capture tribal knowledge from tickets? Our techs keep reinventing the wheel

[techguy32674]

Our team keeps losing tribal knowledge between tickets, how do you solve this ?

i’m part of a small MSP, and we’re struggling with how scattered our troubleshooting info has become.When an engineer gets a ticket, half the time they’re bouncing between IT Glue docs, old Slack threads, or random emails trying to find how someone solved a similar issue before. If they finally figure it out, they might do a quick “lunch-n-learn” or write a short blurb in IT Glue

but most of that tribal knowledge still sits in people’s heads.We’ve tried improving documentation hygiene, but since most of the best fixes come from past tickets or random “hacks” that aren’t in official docs, it’s messy to organize

IT Glue is ok for process docs, but terrible for quickly finding a specific fix, and it doesn’t look at unstructured data. is anyone solving this better? How are you making previous ticket resolutions searchable or accessible when a new incident comes in?

I’ve heard people suggest implementing a RAG + LLM solution, but i think it only works when all your information is in one application, which doesn’t really work for us because we have ticket resolutions in one place, SOPs in another, etc.

 

[Markverhyden]

Is there any indexing/labeling/tagging going on? Difficulties like you describe usually are due to failure to properly maintain such a system. Of course the storage mechanism also has to be able to properly utilize those references.

 

[Sky-Knight]

You're looking at this backwards.

If you're fixing something more than once, a SOP needs to be drafted, that goes into IT glue. If you have technical people not sticking to SOPs, it's time to find better technical people. Your engineers should be doing root cause analysis and generating initial SOPs as part of the ticket resolution process too. In my experience this is a normal growing pain for any technical org. You spend all your time just trying to keep the wheels from falling off you don't take time to organize the process. It's impossible to get this information back out of tickets... And yes, the additional documentation time means you're spending just as much time documenting as you were fixing. This too is normal, and many engineers HATE IT.

AI MIGHT solve that specific issue, if your tickets are in a place an AI can review them all. But even then it's just a glorified search engine for your tickets, not a proper stack of SOPs.

 

[britechguy]

And yes, the additional documentation time means you're spending just as much time documenting as you were fixing.

Sometimes more, if you want really usable and user-friendly Standard Operating Procedures (SOPs). Opaque ones are worse than useless.

Your point about the human(s) who made the fix actually giving good instructions on how to do it again is well taken, and very often its the very person/people who wrote the SOP who have to call upon it again and again. And once you've found yourself in that position, you should instantly come to realize why they're so important and why writing them "the right way" is equally important.

 

[timeshifter]

Maybe AI could be used to create the whatever writeup is needed?

As a one man shop I use ChatGPT a lot. If it's been involved in the problem I'll ask it for a paragraph that I can paste into the invoice. You could do the same or similar to build a tech note or whatever.

A tip that I find useful: I talk to ChatGPT. I click it's mic icon and talk away. I can say a lot more and get much better results. Know that this is different than regular dictation you've seen, it's not typing out what you say, it listens to everything then puts it together. I'm very productive with it.

 

[hiddenpcmaster]

A Self hosted data hoarding solution. https://karakeep.app

 

[Sky-Knight]

Maybe AI could be used to create the whatever writeup is needed?

As a one man shop I use ChatGPT a lot. If it's been involved in the problem I'll ask it for a paragraph that I can paste into the invoice. You could do the same or similar to build a tech note or whatever.

A tip that I find useful: I talk to ChatGPT. I click it's mic icon and talk away. I can say a lot more and get much better results. Know that this is different than regular dictation you've seen, it's not typing out what you say, it listens to everything then puts it together. I'm very productive with it.

LLMs and I mean ALL of them make wonderful note takers and summarizers, I abuse the heck out of Copilot for this purpose. It's addictively convenient to have this functionality right in my office applications. That doesn't mean other models aren't "better" in one way or another, but they aren't as easy to access, and since the value comes from saving my time, the integration really saves my bacon.

 

[timeshifter]

Ironically just a little while ago I was working with my small business owner client who paid for the whole year of CoPilot that’s tied into his 365 Business Standard account. He said it’s useless and he’s using ChatGPT more. We were doing other things but I wanted to dive deeper and make sure he was using it right and had the right expectations.

 

[Sky-Knight]

Ironically just a little while ago I was working with my small business owner client who paid for the whole year of CoPilot that’s tied into his 365 Business Standard account. He said it’s useless and he’s using ChatGPT more. We were doing other things but I wanted to dive deeper and make sure he was using it right and had the right expectations.

Typically people that prefer ChatGPT over Copilot do so because they don't understand how Copilot works. ChatGPT will learn how to use you while you use it, Copilot will never do this. You need to give Copilot better prompts as a result. The nice thing is the prompts can be shared, the bad thing is if your use case is simple, it's takes more time and effort.

In this case, the personal tool is probably the better play, because the customer isn't correctly investing in M365 to begin with, and likely isn't going to.

 

[britechguy]

You need to give Copilot better prompts as a result.

I don't care what AI Chatbot you're using, when it comes to prompt quality GIGO absolutely applies.

People have got to learn AI prompt refinement just like they did (or, many, didn't) for refining a web search to widen, narrow, or otherwise modify "the net."

If they can't, and won't, understand that what you get out is directly dependent on what you give as a prompt, and that you often have to modify your prompt based on getting "too much" or "too little" then they are a lost cause.

 

[Sky-Knight]

I don't care what AI Chatbot you're using, when it comes to prompt quality GIGO absolutely applies.

People have got to learn AI prompt refinement just like they did (or, many, didn't) for refining a web search to widen, narrow, or otherwise modify "the net."

If they can't, and won't, understand that what you get out is directly dependent on what you give as a prompt, and that you often have to modify your prompt based on getting "too much" or "too little" then they are a lost cause.

Well that's just it, again if you use the paid version of ChatGPT, over time it will learn from you all the things it needs for you to prompt it less, and still get what you want.

Copilot will NEVER do this, paid or otherwise. It's a strange world when you're using software that adapts to you, while you adapt to it. I do not like this specific feature because it leads to reproducibility issues. But your typical SMB owner likely doesn't care, nor will invest the time to do things better until he has to.

 

[YeOldeStonecat]

To me this is a discipline issue....to be discussed across the team. "Document...document...document!"

We use HUDU. Where we document each clients "stuff"....(software, network, user credentials, DNS, etc etc etc etc etc).
And we have a global KB.

For each client, we document their software "stuff" within their space in HUDU. Installation steps, troubleshooting, licensing, etc.
And in the global KB in HUDU....we put "stuff"....and have it broken down into sections. 365. Servers. Workstations. Ubiquiti. Various security software we use, etc.

Easily findable and/or searchable. And the editing part of HUDU, WYSIWYG, pasting in snipping screenshots easily, it's wonderfully fast and easy to whip up a new article.

 

[techguy32674]

Is there any indexing/labeling/tagging going on? Difficulties like you describe usually are due to failure to properly maintain such a system. Of course the storage mechanism also has to be able to properly utilize those references.

Great point, and yes, we have tagging. However, I think my MSP has become too flexible on tags, i.e., there are too many types of tags which makes the system less searchable

 

[techguy32674]

You're looking at this backwards.

If you're fixing something more than once, a SOP needs to be drafted, that goes into IT glue. If you have technical people not sticking to SOPs, it's time to find better technical people. Your engineers should be doing root cause analysis and generating initial SOPs as part of the ticket resolution process too. In my experience this is a normal growing pain for any technical org. You spend all your time just trying to keep the wheels from falling off you don't take time to organize the process. It's impossible to get this information back out of tickets... And yes, the additional documentation time means you're spending just as much time documenting as you were fixing. This too is normal, and many engineers HATE IT.

AI MIGHT solve that specific issue, if your tickets are in a place an AI can review them all. But even then it's just a glorified search engine for your tickets, not a proper stack of SOPs.

Thanks for empathizing with the growing pain I think organizing the process is paramount, as you mentioned. Will reiterate the importance of proper documentation to the team, and the long-term benefit of it

 

[techguy32674]

To me this is a discipline issue....to be discussed across the team. "Document...document...document!"

We use HUDU. Where we document each clients "stuff"....(software, network, user credentials, DNS, etc etc etc etc etc).
And we have a global KB.

For each client, we document their software "stuff" within their space in HUDU. Installation steps, troubleshooting, licensing, etc.
And in the global KB in HUDU....we put "stuff"....and have it broken down into sections. 365. Servers. Workstations. Ubiquiti. Various security software we use, etc.

Easily findable and/or searchable. And the editing part of HUDU, WYSIWYG, pasting in snipping screenshots easily, it's wonderfully fast and easy to whip up a new article.

Thank you on the issue classification. I think you're absolutely right - documentation practices trickle through the company culture, and that'd need to improve on my side. Principle of WYSIWYG is spot on and how you laid out the foundations to new KB article creation is helpful as well. thank you

 

[techguy32674]

Maybe AI could be used to create the whatever writeup is needed?

As a one man shop I use ChatGPT a lot. If it's been involved in the problem I'll ask it for a paragraph that I can paste into the invoice. You could do the same or similar to build a tech note or whatever.

A tip that I find useful: I talk to ChatGPT. I click it's mic icon and talk away. I can say a lot more and get much better results. Know that this is different than regular dictation you've seen, it's not typing out what you say, it listens to everything then puts it together. I'm very productive with it.

really opened up a new perspective with the chat voice feature - thanks. I guess it's better than 2011's Siri, lol

 

[timeshifter]

I guess it's better than 2011's Siri, lol

Night and day! Even 2025's Siri. I usually find that using Apple's (or anyone's for that matter) dictation type out what I want to say an exercise in frustration. It spells things wrong (wrong word) and stops at the wrong spots.

ChatGPT on the other hand, I just say want I want to say, stumble through it, and out comes clear coherent text.

 

[britechguy]

I usually find that using Apple's (or anyone's for that matter) dictation type out what I want to say an exercise in frustration.

About this we are in complete agreement (and I know many others who feel this way).

That being said, voice-to-text (AKA voice dictation) is to AI summarization/generation functionality as apples are to papayas. They just can't be compared because they're completely different things.

 

[Sky-Knight]

Especially sense the AI can correct the busted input if the input is long enough.

 

[callthatgirl]

I worked with the KB team back in corporate and I loved contributing fixes for Outlook, but that was internal.

Now with the internet and the zillions of fixes, I don't even know if you can have a true KB anymore for anything outside of your own control. We are not in the old days. If this was me starting my own helpdesk company, I'd start with the top 20 calls you take and decide if it's controlled or uncontrolled. Go from there.

Outlook and 365 are mysteries now. I come across new things everyday and I have to figure them out myself, not in google yet.