GEOG 414/514:  Advanced Geographic Data Analysis
Scatter-diagram smoothing (Nonparametric regression)

Scatter-diagram smoothing involves drawing a smooth curve on a scatter diagram to summarize a relationship, in a fashion that makes few assumptions initially about the form or strength of the relationship.  It is related to (and is a special case of) nonparametric regression, in which the objective  is to represent the relationship between a response variable and one or more predictor variables, again in way that makes few assumptions about the form of the relationship.  In other words, in contrast to "standard" linear regression analysis, no assumption is made that the relationship is represented by a straight line (although one could certainly think of a straight line as a special case of nonparametric regression). 

Another way of looking at scatter diagram smoothing is as a way of depicting the "local" relationship between a response variable and a predictor variable over parts of their ranges, which may differ from a "global" relationship determined using the whole data set.  (And again, the idea of "local" as opposed to "global" relationships has an obvious geographical analogy.)

A review of global fitting (e.g. linear regression)

In ordinary linear regression analysis, the objective can be considered to be drawing a line through the data in an optimal way, where the parameters (regression coefficients) are determined using all of the data, i.e. they are globally determined.  However, it is possible to think of the line as connecting the points, that for each value of X, represent the local density maxima of Y--it just happens that these local maxima happen to be arranged along a straight line.

Loess curves

A bivariate smoother is a function or procedure for drawing a smooth curve through a scatter diagram.  Like linear regression (in which the "curve" is a straight line), the smooth curve is drawn in such a way as to have some desirable properties.  In general, the properties are that the curve indeed be smooth, and that locally, the curve minimize the variance of the residuals or prediction error.

The bivariate smoother used most frequently in practice is known as a "lowess" or "loess" curve.  The acronyms are meant to represent the notion of locally weighted regression--a curve- or function-fitting technique that provides a generally smooth curve, the value of which at a particular location along the x-axis is determined only by the points in that vicinity.  The method consequently makes no assumptions about the form of the relationship, and allows the form to be discovered using the data itself.  (The difference between the two acronyms or names is mostly superficial, but there is an actual difference in R--there are two different functions, lowess() and loess(), which will be explained below.)

The mechanics of loess:

  • the locally weighted "fitted value" at a particular point
  • what happens at the margin of the scatter diagram
  • the mathematics of loess
  • the parameters of an individual curve (in addition to smoothing type)
    • span -- a value between 0 and 1 controlling the amount of smoothing, with smaller values resulting in less smoothing.  Typical values lie in the range of .3 to .5.  Span can be considered to represent the width of the smoothing window.
    • degree -- the degree of the locally-fitted polynomial.  1 = locally linear fitting (i.e. a line), 2 = locally quadratic quadratic fitting (i.e. a parabola).
    • family -- either "Symmetric" or "Gaussian"  The symmetric option combines the local fitting with a "robustness" step that discounts the influence of unusual points, while the Gaussian option does not.

Scatter diagram smoothing in R

Note that there are actually two versions of the lowess or loess scatter-diagram smoothing approach implemented in R.  The former (lowess) was implemented first, while the latter (loess) is more flexible and powerful.
  • Simple "lowess"

A simple "lowess/loess" curve is constructed using the lowess() function, which finds a "fitted" value for each data point; these can be plotted as individual symbols, but they are usually connected with lines.  The lowess() function has a "span" argument (sometimes symbolized by λ) that represents the proportion of the total number of points that contribute to each local fitted value.

plot(elev, tann)
points(lowess(elev, tann), col="red")
lines(lowess(elev,tann), col="red")

lines(lowess(elev, tann, f=0.33), col="magenta")
lines(lowess(elev, tann, f=0.80), col="green")

  • "loess"

The newer loess() function uses a formula to specify the response (and in its application as a scatter-diagram smoother) a single predictor variable.  The loess() function creates an object that contains the results, and the predict() function retrieves the fitted values.  These can then be plotted along with the response variable.  However, the points must be plotted in increasing order of the predictor variable in order for the lines() function to draw the line in an appropriate fashion this is done by using the results of the order() function applied to the predictor variable values and the explicit subscripting (in square brackets [ ]) to arrange the observations  in ascending order.

plot(tann ~ elev)
loess.model <- loess(tann ~ elev)
loess.model
hat <- predict(loess.model)
lines(elev[order(elev)], hat[order(elev)], col="red")

loess.model <- loess(tann ~ elev, span=0.75, degree=2)

Other bivariate smoothers

Loess is one of a number of smoothers (including linear regression as an end-member) that can be used.  The different smoothers vary in the assumptions they make about

  • the form of the relationship
  • the influence of individual points

The other scatter diagram smoothers include a straight, or "least-squares" line, a low-order polynomial least-squares line, and the "smoothing spline".  Each can be viewed as special cases of the more flexible loess-type smoothers in which the curve is very simple.  The best way to understand these different smoothers is to compare them:

  • Least-squares line

plot(tann ~ elev)
abline(lm(tann ~ elev), col="red")

  • Least-squares polynomial

# second order polynomial
poly2.model <- lm(tann ~ elev + I(elev^2))
poly2.model
hat <- predict(poly2.model)
lines(elev[order(elev)], hat[order(elev)], col="purple")

  • Smoothing spline

library(modreg)
lines(smooth.spline(elev, tann), col="magenta")

The various smoothers can be summarized as follows:

  Smoother Form Influence of individual points
 
fewest assumptions loess no assumptions unusual points discounted
  smoothing spline smooth curve some discounting of unusual points
  robust, robust MM straight line unusual points discounted
  least squares (curvilinear) curve all points influential
most assumptions least squares (linear) straight line all points influential