In ordinary linear regression, the goal is to minimize the sum of the squared
vertical distances between the y data values and the corresponding
y values on the fitted line. In orthogonal regression the goal
is to minimize the orthogonal (perpendicular) distances from the data points
to the fitted line.
The slope-intercept equation for a line is:
Y = m*X + b
where m is the slope and b is the intercept.
A line perpendicular to this line will have -(1/m) slope, so the equation
Y' = -X/m + b'
If this line passes through some data point (X0,Y0), its equation will
Y' = -X/m + (X0/m + Y0)
The perpendicular line will intersect the fitted line at a point (Xi,Yi) where
Xi and Yi are defined by:
Xi = (X0 + m*Y0 - m*b) / (m^2 + 1)
Yi = m*Xi + b
So the orthogonal distance from (X0,X0) to the fitted line is the distance between
(X0,Y0) and (Xi,Yi) which is computed as:
distance = sqrt((X0-Xi)^2 + (Y0-Yi)^2)
So the goal of the NLREG program is to minimize the sum of these orthogonal distances.
Here is a NLREG program that does this:
Title "Fit a line to data points minimizing orthogonal distances";
Variables X0, Y0;
Parameters m, b;
Double Xi, Yi, distance;
Xi = (X0 + m*Y0 - m*b) / (m^2 + 1);
Yi = m*Xi + b;
distance = sqrt((X0-Xi)^2 + (Y0-Yi)^2);
For an example of a NLREG program that performs orthogonal regression to a 3D plane,
please click here.