It is important to remember that a great deal of what we know about Republican and early imperial Rome comes entirely from the surviving works of writers who came decades or even centuries after the events they chronicled. Furthermore, the motivations of these writers rarely had anything to do with what we would consider serious scholarly research.
Suetonius, for example, adored gossip and juicy tidbits. Living and writing during the age of the “Good” Emperors, he was allowed more literary license than one might normally expect, but, at the same time, had access to imperial archival materials. So while the more outrageous stories should be taken with a heavy dose of salt, Suetonius was evidently in a good position to get his basic facts correct.
Plutarch’s approach to history was not much concerned with the facts either. He was interested primarily in character. His biographies are as much philosophical inquiries into the nature of virtue as histories. Good reading, though.
Tacitus, on the other hand, was the Kitty Kelly of his day. He was from a senatorial family that had never gotten over the fact that they had no power under the imperial system. So his histories drip with poison. Probably alone among historians contemporary to the age, he paints a negative picture of Augustus, and his opinions of the Julio-Claudian dynasty only go downhill from there.
So as far as individual scandals, such as the possibility of Livia’s poisoning of Augustus, there is really little evidence other than what these men have to say. Since the Romans, with very few exceptions, cremated their dead, there’s no hope of finding Augustus’ remains to conduct forensic tests. Robert Graves did carry out an investigation of Claudius’ death based on the literary evidence, and concluded (somewhat tortuously) that Seneca was in on the plot.
Colleen McCullough is an interesting case. The research she had conducted in the writing of her Masters of Rome series has been tremendous. For the most part, her novels take the known historical record and simply fill in the gaps. Since these are novels and not scholarly volumes, McCullough is under no obligation to stick to the facts, and yet this is precisely what she does. Naturally a lot of the details she fills in cannot be proven as accurate, but nor can they be disproved. Even in cases where she diverts from the historical record, her reasons for doing so are based on solid reasoning and are defended in the reader’s notes that conclude each volume. In making these novels so historically accurate, McCullough has taken a bold step, because her weighty prose is a casual reader’s nightmare. But for those readers with long attention spans who don’t mind taking notes as they read, Masters of Rome is a veritable smorgasbord.