Option FanaticOptions, stock, futures, and system trading, backtesting, money management, and much more!

Why is Curve Fitting Such a Bad Word?

I have been trying to get more organized this year by converting incomplete drafts into finished blog posts. Some of these are from out in left field, but I am publishing them anyway on the off chance someone might be able to benefit.

In the second paragraph here, I said “optimization” is sometimes a bad word. It is specifically a bad word when used to mean curve fitting, which relates back to this draft I wrote in July 2019.

—————————

Is it curve-fitting or curve fitting? Whatever it is, it’s a bad word when used in these parts.

The process of curve fitting, though, is not a bad process but rather a branch of mathematics.

Wikipedia gives us:

     > Curve fitting[1][2] is the process of constructing a curve, or
     > mathematical function, that has the best fit to a series of data
     > points,[3] possibly subject to constraints.[4][5] Curve fitting
     > can involve either interpolation,[6][7] where an exact fit to
     > the data is required, or smoothing,[8][9] in which a “smooth”
     > function is constructed that approximately fits the data. A
     > related topic is regression analysis,[10][11] which focuses
     > more on questions of statistical inference such as how much
     > uncertainty is present in a curve that is fit to data observed
     > with random errors. Fitted curves can be used as an aid for
     > data visualization,[12][13] to infer values of a function where
     > no data are available,[14] and to summarize the relationships
     > among two or more variables.[15] Extrapolation refers to the
     > use of a fitted curve beyond the range of the observed data,
     > [16] and is subject to a degree of uncertainty[17] since it
     > may reflect the method used to construct the curve as much
     > as it reflects the observed data.

When used without validity, curve fitting turns bad. Looking to the past for viable strategies with the hope of applying them in the future makes sense. However, the past is unlikely to match the future and I have no idea of knowing how similar the two may be. The more I try to curve fit a strategy to the past, the less likely it is to work in the future.

The true goal of trading system development is to provide enough confidence to stick with a strategy through times of drawdown in order to realize future profit (see end of third paragraph here). However, if the strategy is poorly suited for the future because it is curve fit to the past then I will definitely lose money trading it. Curve fitting is a bad word in this respect.