Suppose you have a function of two independent variables, Z = f(X,Y), and a graph comparing the two independent variables shows your (X,Y) point cloud is somewhat long and slender and somewhat crescent shaped, like a bit of scatter along and around the middle region of the positive branch of a hyperbola.
If you want to estimate f for a point (X,Y) that is well inside the cloud, certainly one way to do so is to interpolate between some close neighbors. At least, I think the term “interpolation” can apply to functions of more than one variable.
Suppose you want to estimate f for a point that is partly surrounded by the crescent, so that it is between the middle of the cloud and the line joining the ends of the cloud. All of its nearest neighbors are broadly in the same general direction. Is using those nearest neighbors interpolation or extrapolation? Do we call it interpolation because we’re evaluating f for values of X and Y that are respectively within the ranges of X and Y? Or do we call it extrapolation because it is outside the cluster of neighbors?
Suppose we pick a point far off of the crescent to estimate, but it’s still within the ranges of X and Y; that is, draw a rectangle to enclose the crescent, and estimate a point near that corner of the rectangle that is furthest from any point. What’s that?
This would be easier if I drew a plot, but there’s one that is useful enough at http://i.stack.imgur.com/hwxSW.jpg so imagine we’re talking about a point near the center of the circle that the points roughly follow.