Is the story set in the 1980’s, or is it an old computer?
In 1975 Martin Gardner published his “Hoax integer” e[sup]π √163[/sup] which is ridiculously close to, yet not equal to, the integer 640320[sup]3[/sup]+744. Perhaps your character tries to test whether the hoax integer is larger or smaller than its nearby integer, a task likely to overtax a computer since its first 12 digits to the right of the decimal point are .999999999999.
(I have some boring stories of real incidents from a previous lifetime. For example, when early 370/165’s were retrofitted to support the MVCL instruction, some couldn’t handle the activity and their power supplies crowbarred off!)