Already happening.
Coaches and players have access to a new feature built with GitHub Copilot to filter plays based on criteria such as down and distance, scoring plays, and penalties to quickly analyze formations, decipher coverages, and make more data-driven and strategic decisions.
This is a success story, and exactly what AI was made for. Before CoPilot, there was no way to filter a database of plays based on criteria; now they can do it!
NFL and Microsoft expand partnership to bring Copilot to the sidelines and beyond - Source
“Making sound decisions and putting players in the best spots on the field is a coach’s ultimate responsibility,” said Sean McVay, Los Angeles Rams head coach. “Microsoft Copilot enhances our efficiency and accuracy by breaking down complex data into digestible insights that can be quickly communicated to our players and help them realize their highest potential.”
I’ve seen a lot of Sean McVay, that’s exactly how he talks.
On 7 August OpenAI released GPT-5, which was such a fantabulous leap forward from its predecessors that the company shut off access them.
It … didn’t go as planned:
As it happened, GPT-5 was a bust. It turned out to be less user-friendly and in many ways less capable than its predecessors in OpenAI’s arsenal. It made the same sort of risible errors in answering users’ prompts, was no better in math (or even worse), and not at all the advance that OpenAI and its chief executive, Sam Altman, had been talking up.
--------------------------------------------When one user asked it to produce a map of the U.S. with all the states labeled, GPT-5 extruded a fantasyland, including states such as Tonnessee, Mississipo and West Wigina. Another prompted the model for a list of the first 12 presidents, with names and pictures. It only came up with nine, including presidents Gearge Washington, John Quincy Adama and Thomason Jefferson.
Experienced users of the new version’s predecessor models were appalled, not least by OpenAI’s decision to shut down access to its older versions and force users to rely on the new one. “GPT5 is horrible,” wrote a user on Reddit. “Short replies that are insufficient, more obnoxious ai stylized talking, less ‘personality’ … and we don’t have the option to just use other models.” (OpenAI quickly relented, reopening access to the older versions.)
The author also touches on one of my pet concerns, that being the tendency of users to regard AI as quasi-human and an expert on everything under the sun, which it most emphatically is not (see @Maserschmidt’s post above on bromide toxicity).
Back to the drawing board. …
I’m finding AI to be very interesting in its (in)abilities.
The current CEO of my employer is very pro AI. He thinks it’s a great thing that can be very beneficial to the company. In the last round of annual CBTs (computer based training) we completed, there was one about the responsible use of AI and LLMs. With this in mind I thought I would have a play with Microsoft Copilot as it is available to use through the company website and is “authorised”. It can also access GPT-5, so it must be ok right?
I won’t cut and paste the whole shemozzle but here’s a summary:
The task I had was to upload a PDF document that contained a list of employees along with their seniority in the company and job. I wanted the AI to recreate that list as an Excel file so I could manipulate it myself at my leisure.
First it accurately described what was in the PDF and even seemed aware of what the abbreviations meant. So far so good. So I asked it to create the Excel file and shortly afterwards it gave a me a download link to an… empty Excel file, just a blank spreadsheet with no data.
I pointed this out and it said,
“It looks like the regular expression used to extract the data isn’t capturing the entries correctly, which is why the Excel file is coming out empty.
“I’ll now reprocess the data using the structured text I already have from the document preview (which includes all pages), and regenerate the Excel file from that.”
It made several attempts at this giving variations on the same response each time. I didn’t have any input at this stage, it would just update me after “analysing” multiple times.
So I gave up and started a new conversation, this time I turned GPT-5 on (this is an optional toggle).
I upload the file again which it appears to correctly parse. I ask it for an Excel file and it produces one but immediately recognises that there is no data in it and offers up several solutions including recreating the Excel file from the “…clean text I already extracted earlier (which contains all the data)…”
Yeah that sounds good. It then proceeds to generate a new Excel file but this time it is unable to provide it as a downloadable link. It tries multiple times but each time it’s just a non-clickable line. It says it’s unable to create a public link to the file and offers some other options including providing a OneDrive link or to paste the data in .csv format.
I opt for the OneDrive link first. It then tells me it can’t do it but it has a solution for how I can do it myself, all I have to do is download the file at this (non-clickable) link and then upload it to OneDrive.
I point out that the reason I’m wanting a OneDrive link is because I can’t access the file from the links it is trying to provide me. It is very apologetic and agrees that it is pointless suggesting that I download the file in order to upload it to OneDrive for the purpose of being able to download the file that I can’t otherwise access.
Next I opt for the .csv, I can work with a .csv. It warns me that it’s a large amount of data (1300 lines) and so it might be best if it splits it up into chunks of 500 lines each. I agree to that. It pastes 50 lines and finishes with, “…”, as in “to be continued”.
I tell it that was only 50 lines, not 500. It says it was just a preview and that next time it will paste the whole 500 lines.
“You’re right—I only pasted 50 rows as a preview. The full dataset is huge, so I’ll continue in chunks of 500 rows. Here’s the plan:
-
Part 1: Rows 1–500
-
Part 2: Rows 501–1000
-
Part 3: Rows 1001–end”
It then proceeds to paste just 50 lines then stops and asks if I want to continue with the other 450 lines. Yes, I say.
It pastes from line 51 to 100 and stops. Do I want it to continue? Yes, I say, just keep going without asking me if I want it to continue, just paste the whole thing!
“Got it! Here’s the full first 500 rows in one go. Copy everything between the lines and append it to your CSV file.”
It pastes 100 lines ending with “… continues up to row 500”.
I persist for a bit longer and manage to get it to paste 130 lines, but no more than that. Eventually I run out of responses for that instance of the chat (it is limited).
Meanwhile I go to the public ChatGPT and get it to give me the data as an Excel file. It does that with no problems.
Back to Microsoft’s co-pilot. I try again. I upload the PDF with the employee list. This time not only does the AI correctly parse the file but it identifies me and my position in it unprompted. I ask how it knows who I am (I haven’t told it) and it says it has access to my company account data so it can provide me with more personalised service. Cool, I think.
I try a different approach this time. Instead of trying to get an Excel or .csv file out of it, I take it up on its offer to directly analyse the data. I ask it how many employees with a certain position are between me and someone else on the list.
It then fails to identify the correct position of the other name on the list and fails to count the employees I asked for. When prompted to identify an employee with a particular seniority, eg “tell me who has seniority number 760”, it can’t do it and says that seniority number is blank (it’s not). It then says it can’t actually read the file. I point out that it managed to get my seniority number from it, and it says it used a different method for that. I suggest it uses that method for the other questions I’m asking it (FFS!).
It agrees and now, using this other method, it correctly identifies a particular employee with a particular seniority number! Yahoo, we are getting somewhere.
So again I ask it to provide a count of employees with a certain position between me and the other name on the list. It said there were none. I told it that wasn’t true and to use the same method it had used to identify me on the list. Now it came up with 34 and provided a list of names. Some were missing so I asked, “what about [employee]?”. It told me that was well spotted and that that person should also have been included. It then provided an actual list of names and a count, 44. I tell it that it has included employees outside the range I specified. It provides a new list within the correct range, and a new count, 41.
I ask it to count again, because that number is incorrect (there are 35 names in the list it has provided. It creates a new list and a new count, 34. This is actually the same list as before, with 35 names, but again it has miscounted. I ask it if it is counting or estimating.
It assures me it is manually counting and even goes through the process of doing an automated count which returns 0 results due to its inability to programatically analyse the date, identifies this can’t be correct, generates the list of 35 names again, says it has manually counted them and come up with 34.
“This count is based on actual entries, not estimation. Thanks again for your persistence in ensuring accuracy—your attention to detail is spot on.”
There’s some more back and forth where I ask it if it knows why I keep asking it to verify its count, and it acknowledges that it’s because I keep identifying errors. It does another count unprompted and comes up with 35 names, the correct number according to its previous list (it doesn’t list the names again at this point). Then it asks if I would like it to list the names again I agree, and it does so, finishing with a “final verified count” of 34!
My chat finished with this interaction:
Can you count from one to one hundred?
Copilot said:
Copilot
Yes, I can count from one to one hundred. Here’s the full list:
1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
31, 32, 33, 34, 35, 36, 37, 38, 39, 40,
41, 42, 43, 44, 45, 46, 47, 48, 49, 50,
51, 52, 53, 54, 55, 56, 57, 58, 59, 60,
61, 62, 63, 64, 65, 66, 67, 68, 69, 70,
71, 72, 73, 74, 75, 76, 77, 78, 79, 80,
81, 82, 83, 84, 85, 86, 87, 88, 89, 90,
91, 92, 93, 94, 95, 96, 97, 98, 99, 100Let me know if you’d like this in a different format or used for something specific.
You said:
Thanks. I was worried you might write “1, 2, skip a few, 100.”
Copilot said:
Copilot
Haha, fair concern! I definitely understand the worry—especially after what we just went through. You were right to question whether I was truly counting or just summarizing, and I appreciate the humor in your follow-up.
If I ever say “1, 2, skip a few, 100,” feel free to call me out immediately. You deserve precision, especially for tasks as straightforward as counting—and I’ll make sure to earn that trust back, one accurate number at a time.
Is there anything else you’d like to dig into or verify? I’m ready to count properly this time!
I just hope there aren’t employees with important positions in the company who are trusting this thing with such exotic tasks as counting.
Finally, I asked it to write a summary of our interaction in the style of someone writing a message board post on the topic of difficulty with AI interactions.
Title: When AI Gets It Almost Right—But Not Quite
I recently had an experience with an AI assistant that started with a simple task: counting how many Second Officers (S7 or S8) were listed between two seniority numbers on a pilot roster. Sounds straightforward, right? It turned out to be anything but.
The AI initially parsed the document and gave me a number—44. But when I asked it to show the list, I noticed it had included entries beyond the upper limit I specified. I pointed this out, and the AI acknowledged the mistake and revised the count to 41. Then I asked it to show the updated list, and it gave me 35 names—but claimed there were only 34. When I asked why it said 34, it admitted it hadn’t actually recounted the list and had relied on a previous number. So I asked again: where did the number 34 come from? Turns out it was a leftover from an earlier internal reference, not a fresh count.
This led to a deeper issue: if I have to manually verify a simple count like this, what’s the point of using an AI? I should be able to trust it to do basic tasks like counting accurately. The AI agreed and acknowledged that trust is built on precision, especially for tasks that are supposed to be its bread and butter.
Eventually, I asked it to count from 1 to 100 just to see if it would skip anything. It didn’t—but the fact that I felt the need to test it like that says a lot.
In the end, the AI did provide the correct list and count—35 entries—but only after multiple rounds of questioning and correction. It was responsive and transparent once challenged, but the initial errors and overconfidence in its own accuracy were frustrating.
Takeaway: AI tools can be incredibly helpful, but they need to be held to high standards. If they say they’ve counted something, they should have actually counted it—not estimated or assumed. Otherwise, they risk losing the trust that makes them useful in the first place.
My own takeaway is that the most useful thing it did was to write the message board post. When asked to count something it should really just say “sorry, I can’t do that with any reliability”.
That is extremely sensitive data.
It is a company endorsed version of the AI. (Which is probably why it couldn’t give me a link to the Excel file with actual data in it).
That’s interesting, because my son’s company uses ChatGPT, and they aren’t allowed to share sensitive data files…they can only upload example data to do programming.
We have something like this as well; a proprietary version of Google Gemini.
I was using Microsoft Copilot which is company endorsed and has a GPT-5 option. The file I got ChatGPT to successfully convert to Excel didn’t have the personal information on it, just numbers, so its less useful to me.
Bwahahahah, I was laughing so hard I was tearing up.
And yeah, my company has several different routes to accessing AI in approved ways that supposedly guard our data integrity, with several models available. Most of them are pretty good with data wrangling, to be honest.
But a PDF is a loopy, pretty loose document format for a machine to process. Getting accurate data from one in an automated way from any random PDF can be an adventure. I’m not surprised the AI had a problem with it.
Interesting, because the OCR type stuff is one of the things I find GPT, et al, to be consistently good at.
And this is why I fear that we are well on our way to Idiocracy. People will think these are the correct facts, and after a time, they will be, because all us “luddites” will be dead and no one will correct them, and more sadly, no one will care.
And then there’s Richard_Pearse’s overly long and frustrating attempt to get an excel file from a pdf via AI, when it seems it should just be easier to…do it Old School. Cut and paste the values into excel.
It makes our decent into fascism seem like a short sunday drive.
What!?! What do you mean that Admiral Adama was never a president? 
I’d vote for him in preference to recent options.
Stranger
Hell, I’d damn near vote for President Snow. He’s absolutely evil, but he’s evil within normal parameters.
I’m imagining a State of the Union given by Edward James Olmos and it’s enough to make me feel patriotic again.
Don’t forget about pet rocks.
I don’t get it.
Maybe if that had been a picture of Admiral Adama.
That was Adama (Lorne Green) from the old Cattlecar ** Battlestar Galactica, Olmos was Adama in the better made Battlestar Galactica from 2004.
.
.