Presumably a document query system, one of the actually useful reasons to use an LLM.
Having it trawl through thousands of pages of NASA and government files to answer regulatory questions is probably legitimately helpful, even if all it does is point you towards the right pages for a human to review.
That makes sense. Like you, I’ve generally found that LLMs are incredibly useful for certain, highly specific things but people (CEOs especially) need to understand their limitations.
When it first came out, I purposely used ChatGPT on a trip to evaluate it. I was in a historic city on a business trip where I stayed an extra few days so I was traveling alone. It was good at being a tour guide. Obviously, I could have researched everything and read guidebooks but I was focused on my work stuff. Being able to ask follow-up questions and have a conversation was a real improvement over traditional search.
That’s obviously a limited use case where I was asking questions that could have been answered in traditional ways but I found that to be a good consumer use case. It knew details that wouldn’t necessarily be in a Wikipedia article or Guidebook that would take me 15 Google searches to answer. Just my own little curiosity questions about an old building or whatever. I cross-checked things later and it didn’t hallucinate. Obviously, a very limited use case but it was good at it.
This is the one I’m waiting the most from the LLM hype. It would be a massive benefit for companies around the world (mine included) if they could just dump their documentation in all shapes and flavours into a model and have it parse standardized documents for you.
But the generic OpenAI/Copilot models aren’t reliable enough just yet, hallucinations and made up data just doesn’t go with that. I’m not even sure if those models are ever capable on such task alone, maybe it needs additional component which checks the facts from originals or something to make it actually useful.
I created a relatively complex board/card game and just shoved the rulebook PDF into an LLM, so it’d be easy to answer people’s questions with exact references to cards and rulebook pages. So far it’s successfully corrected ME twice on specific rulings. It’s actually quite useful in this regard.
Generic models are not ready yet and won’t ever be but my company is using something from AWS called “Q” I think (heh, great name this day in age) but because it performs training rounds (*I think!) on a set of documents you define (like process documentation) its actually pretty good at finding exactly the information and suggesting additional reading. We’re using it on a process guide of a couple hundred pages; not sure how it scales to other sizes or what the cost is.
Presumably a document query system, one of the actually useful reasons to use an LLM.
Having it trawl through thousands of pages of NASA and government files to answer regulatory questions is probably legitimately helpful, even if all it does is point you towards the right pages for a human to review.
That makes sense. Like you, I’ve generally found that LLMs are incredibly useful for certain, highly specific things but people (CEOs especially) need to understand their limitations.
When it first came out, I purposely used ChatGPT on a trip to evaluate it. I was in a historic city on a business trip where I stayed an extra few days so I was traveling alone. It was good at being a tour guide. Obviously, I could have researched everything and read guidebooks but I was focused on my work stuff. Being able to ask follow-up questions and have a conversation was a real improvement over traditional search.
That’s obviously a limited use case where I was asking questions that could have been answered in traditional ways but I found that to be a good consumer use case. It knew details that wouldn’t necessarily be in a Wikipedia article or Guidebook that would take me 15 Google searches to answer. Just my own little curiosity questions about an old building or whatever. I cross-checked things later and it didn’t hallucinate. Obviously, a very limited use case but it was good at it.
This is the one I’m waiting the most from the LLM hype. It would be a massive benefit for companies around the world (mine included) if they could just dump their documentation in all shapes and flavours into a model and have it parse standardized documents for you.
But the generic OpenAI/Copilot models aren’t reliable enough just yet, hallucinations and made up data just doesn’t go with that. I’m not even sure if those models are ever capable on such task alone, maybe it needs additional component which checks the facts from originals or something to make it actually useful.
I created a relatively complex board/card game and just shoved the rulebook PDF into an LLM, so it’d be easy to answer people’s questions with exact references to cards and rulebook pages. So far it’s successfully corrected ME twice on specific rulings. It’s actually quite useful in this regard.
Generic models are not ready yet and won’t ever be but my company is using something from AWS called “Q” I think (heh, great name this day in age) but because it performs training rounds (*I think!) on a set of documents you define (like process documentation) its actually pretty good at finding exactly the information and suggesting additional reading. We’re using it on a process guide of a couple hundred pages; not sure how it scales to other sizes or what the cost is.