This is not about plagiarism, though. Plagiarism and copyright violation are two very different things. You can plagiarize without violating copyright, and you can violate copyright without plagiarizing. Or you can do both together. But plagiarism is basically irrelvant to the legal question here.
As echoreply suggests, it is somewhat difficult to come up with a single definitive answer, precisely because copyright lawsuits each have to be decided on their individual merits, and questions of fair use are decided by balancing four factors and weighing them together in order to reach a determination.
Also, echoing echoreply’s reply (:)), the biggest hurdle here is going to be the question of whether it constitutes fair use to quote the entire caption. Part of the court’s evaluation in such a case might be to try and assess the extent to which the caption can be separated out from the artwork. It seems plausible that a court might conclude that just quoting the caption without reproducing the artwork is fair use. After all, the caption is not the whole work; it is only part of the work, and one of the factors in determining fair use is the amount and substantiality of the portion taken. Also, reproducing the caption without the artwork is not likely to measurably affect the market for the original work, and that is another factor for the court to consider.
Another problem is that, if the court did take this approach, it would probably have to consider each and every cartoon separately, because some of Larson’s cartoons rely more heavily than others on the caption, and some have no caption at all.
If we take echoreply’s last sentence, that’s also going to be a factor, because one of the things that needs to be considered in a fair use case is the purpose and character of your use. That is, why are you copying (some of) the original material. If you can make a good case that your aim here is simply to make it easier for people to find the originals, that will weigh in your favor, but again, it is just one factor and needs to be balanced against the other factors.
One thing worth getting out of the way is that captions are probably the only real problem here. A written description of the artwork will almost certainly NOT violate copyright. This photograph from today’s New York Times is protected by copyright, and if I were to take the image and use it on my website or in a book, the copyright holder could sue me for damages. But there is nothing in copyright law that prevents me from describing the image, discussing who is in it, where they are, what they are wearing, and any other details that appear in the picture. Even if my description is so lengthy and precise that an artist could reproduce the image almost exactly, I still haven’t violated copyright with my description. And the same will be true of Larson’s artwork.
If i were willing to risk a venture like the OP’s, here’s how I might go about it. If you’re a Far Side fan, see if you can identify the cartoon in your mind just from my keywords and description, without looking at the image. I’m going to use this Far Side cartoon as my example.
Text keywords:
blond
conducting
Goodall
hair
Jane
research
tramp
Image keywords:
ape
branch
chimpanzee
gorilla
jungle
leaf or leaves
monkey
tree
vine
Image description:
Two primates, one female and one male, are sitting on a branch in a tree. There are vines hanging in the vicinity, and large leaves, indicating that the scene is most likely the jungle. The female ape, wearing 1950s-style horn-rim glasses, is grooming the male from behind, and is holding a thin item, apparently a hair, between her thumb and forefinger, inspecting it.
I don’t think I’ve violated copyright with those sets of keywords and that description. By removing ancillary words like pronouns, articles, conjunctions, etc., and by placing the text keywords in alphabetical order, rather than the order in which they appear in the caption, I have eliminated the unique arrangement that gives Larson’s caption its originality. He has no copyright claim over my alphabetical list.
It seems to me that such an index would work best in electronic format, allowing it to be searched by keyword, and also allowing the creation of an index without using the full captions from the cartoons. Obviously, if you could use the full and complete captions, the index would be more thorough and useful, but I think my method would work just fine in the vast majority of cases, and has the benefit of being far more careful about copyright.