Correlation coefficient in polynomial regression

[agiantdwarf]

I've figured out how to do multiple linear regression, i.e. regression to find the best the best fit polynomial (of a certain degree). For example, you can run quadratic regression and cubic regression.

For linear regression, there's a pretty straightforward way of finding the correlation coefficient. How does it work with polynomial regression? Sinusoidal regression? Logistic regression?

[ultrafilter]

Unless you're talking about something unusual when you say "correlation coefficient", it's exactly the same for any pair of variables.

[desdinova]

Originally Posted by ultrafilter

Unless you're talking about something unusual when you say "correlation coefficient", it's exactly the same for any pair of variables.

I'm guessing he's wondering how you handle it when you've got a regression with multiple regressors. As best as I understand coefficients of correlation, however, by definition they only measure correlation between exactly two variables. I think there were other tests that can be used, for example, to test if all of the coefficients on your regressors are jointly insignificant, but it's been little while since I've taken any statistics courses.

However, with a multiple regression, I'm pretty sure you would have to calculate r for each regressor against the regressand. That's my poor memory of it, anyway. I guess it will have to suffice (with a massive grain of salt) until someone with a little fresher knowledge of statistics happens by.

[desdinova]

Oh yeah, it would not, however, change anything if you did a logarithmic transformation or anything like that. r for X and log(Y) is still r, but of course is different from the coefficient of correlation between X and Y. Just imagine that all you did was swap out "Y" for "Q" except Q happens to be log(Y).

[ultrafilter]

Ah, so the OP is looking to test the hypothesis that b_i = 0 for each i. There is a test for that, but it's been a while since I did much of this, so I'm not sure I can find it.

[agiantdwarf]

Here's my question, stated a little more clearly:

When you do linear regression, you find a "least-squares function" that is the "best fit" line. There's a formula that tells you how well this "best fit line" correlates to the actual data ("R").

I'm wondering if there's a variable like this if you're running regression for, say, a quintic function. Or a sinusoidal function, or a logistic function, etc.

[agiantdwarf]

Originally Posted by agiantdwarf

There's a formula that tells you how well this "best fit line" correlates to the actual data ("R").

That is, "R" is the coefficient, and there's a formula to find R.

[ultrafilter]

You'd use R² in every case, or at least that's what my linear regression professor said. It wasn't an advanced class, so maybe there's something better, but I don't know of it.

Rather than thinking of the equation as y = ax² + bx + c, think of it as y = ax₁ + bx₀ + c, with x₁ = x₀².

[agiantdwarf]

Originally Posted by ultrafilter

You'd use R² in every case, or at least that's what my linear regression professor said. It wasn't an advanced class, so maybe there's something better, but I don't know of it.

Rather than thinking of the equation as y = ax² + bx + c, think of it as y = ax₁ + bx₀ + c, with x₁ = x₀².

Oh, I see. And to calculate R^2 you do (SSdev-SSres)/SSdev, right? Thanks!

[agiantdwarf]

Sorry to bump this old thread, but can anyone help me figure out how to do sinusoidal regression?

This makes some sense but I don't know how to apply it.

[askol]

I'd guess that you just pick the amplitude and phase of the sinusoid to minimize the RMS error between the sinusoid and the data. That's how you write a fourier series, at any rate.

You'd pick the sinusoid A*cos(Bx+c)+D.

But how do you do that? The EE in me would just find the fourier series representation, find the lowest frequency peak, and grab the constants from that. But I'm sure there're other ways.

[ultrafilter]

What you need to do is to take the expression sum((a * cos(bx_i + c) + d - y_i)², 1 < i < n), take the partial derivatives with respect to each coefficient, and set them simultaneously equal to 0. Solve, and you'll have the least-squares coefficients.

Please notice how you don't see me doing this. It's gonna be nasty.

[ultrafilter]

OK, so just for fun, I tried to find the coefficients. You'll need a computer algebra system of some kind to handle it--it's too complicated to handle by hand.