I have a need to be able to calculate a confidence interval for the estimated height of a peak in noisy data.
This is different from calculating a detection limit or a quantitation limit, which I think is what Hubaux-Vos is about. I can’t use go/no-go binary determinations and don’t need to estimate any limits; I actually need the confidence interval, peak by peak, for real data, for every single peak, including the ones that are undetectable. It is critical that these estimates are robust, continuous in all their derivatives, and reasonable in the limits of clean peaks and lost-in-the-noise peaks.
The data look like chromatography or astronomical spectra (I can’t share the data or say more about them and am pretty sure it doesn’t matter anyway). There’s a noisy background and a peak rising out of that background. In the ranges where this is interesting the peaks look interchangeably like Gaussian or Lorenzian peaks - that is, the difference between these is most detectable in the tails, which are lost in noise anyway. I already know what the width parameter and the central location are, and I am guaranteed that there must be some positive and nonzero peak there, but it could be far below the noise and thus show no trace. I have several nice means of estimating peak amplitude. What I don’t have is a way of estimating a confidence interval for that amplitude.
I do have another way out of this; I can synthesize a zillion noisy peaks with random amplitudes and random noise floors, and compare the estimate my methods give with the amplitude parameter that created each one. But I would like to know if there is an accepted approach for this.
I am far from an expert on this subject. However, it seems to me that to calculate a confidence interval you need to estimate a probability distribution. So unless you have some independent method of determining that, I believe you are stuck with repeated trials. As I understand your problem, you don’t care about the shape of the peaks, all you care about is the confidence interval of the peak value itself. What happens off the peak doesn’t matter. So you have a series of values (the magnitude of the peak) at a given point in x. To calculate the confidence interval, you need to estimate the variation of this time series. To do that, all I can think of is repeated trials. Do you have reason to believe that the probability distribution for all the peaks will be the same? That would give you the necessary information for the peaks lost in the noise.
This called the Monte Carlo method, and it is very common. If you don’t know the prior distribution for your peak amplitudes, you can use a Gaussian distribution and no one will complain. You don’t need random noise floors. You know what the noise of your detector is, right? What you want is a confidence level in your measured (calculated) peak amplitude, given the noise you have. You can get this with standard monte carlo techniques.
The Monte Carlo method has just given me 200 trials of 30 peaks each, with a per-trial random value for the peak amplitude relative to the noise amplitude (this amplitude random value is actually 10**z where z is uniformly distributed over the range -3 to 3). For each peak I used the estimation method to deliver an amplitude and an amplitude of the noise floor, to compare with the known amplitude used to generate the peak. 30 peaks should let me estimate the variability of the amplitude estimation errors. 200 trials should let me model how this variability depends on the peak to noise amplitude ratio (it is obvious that they scale together, for the same reason that if I were measuring geometrical bumps the analysis wouldn’t work differently for inches versus centimeters). It took about 2 hours to write the software and 15 minutes for the simulation to run on my almost 2 year old Dell laptop.
If there’s an analytic approach it would be nice to use, but if not this will do OK I think.
By the way, I don’t know what the noise of my detector is. Detectors don’t exist yet. The point of this thing is that I have to invent a detector, and there are various physical options available, whose noises depend on different combinations of different considerations. So I actually need to understand how noise and other behaviors interact in my use of possible detectors, as part of evaluating different detector design predictive models and physical prototypes.
In fact, because it is interesting and stimulating and clarifying, let me elaborate on the idea of inventing detectors. Suppose I needed to measure peaks in pressure measurements over time, and there were no such things as pressure transducers (this isn’t what I am working on but it is perfectly representative).
Could I have a diaphram push a spring, and measure position? Could I measure the density of air by its cooling capacity? Could I measure the air’s index of refraction? Could I make a feedback mechanism that holds a diaphragm in place, and then sample the regulation point of the feedback mechanism? Could I have a flat tube in a coil and watch it unwind (the Bourdon tube)?
Figuring out how to make an optimal transducer requires understanding the dimensions of optimality, knowing the performance metrics. This analysis is really an example of that category of problems.
More of an answer now. I’m just estimating a value by averaging a group of measurements. If my peak had a rectangular shape this would be more obvious, but as it is I’m weighting some measurements more than others, and this threw me. That’s really the only difference between Lorentzian peak estimation and the general practice of averaging a set of measurements. The estimate confidence interval is as Student describes, proportional to the noise and inversely proportional to the square root of the integral of my curve.
