Sigmoid function

This article needs additional citations for verification. Please help improve this article by adding reliable references. Unsourced material may be challenged and removed. (May 2008)

The logistic curve

Plot of the error function

Many natural processes and complex system learning curves display a history dependent progression from small beginnings that accelerates and approaches a climax over time. For lack of complex descriptions a sigmoid function is often used. A sigmoid curve is produced by a mathematical function having an "S" shape. Often, sigmoid function refers to the special case of the logistic function shown at right and defined by the formula

Another example is the Gompertz curve. It is used in modeling systems that saturate at large values of t.

Properties

In general, a sigmoid function is real-valued and differentiable, having either a non-negative or non-positive first derivative and exactly one inflection point. There are also a pair of horizontal asymptotes as . The logistic functions are characterized as the solutions of the differential equation^[1]

Examples

Besides the logistic function, sigmoid functions include the ordinary arctangent, the hyperbolic tangent, and the error function, but also the Gompertz function, the generalised logistic function, and algebraic functions like .

The integral of any smooth, positive, "bump-shaped" function will be sigmoidal, thus the cumulative distribution functions for many common probability distributions are sigmoidal. The most famous such example is the error function.

See also

• Logistic function
• Logistic distribution
• Logistic regression
• Logit
• Hyperbolic function
• Weibull distribution

References

• Tom M. Mitchell, Machine Learning, WCB-McGraw-Hill, 1997, ISBN 0-07-042807-7. In particular see "Chapter 4: Artificial Neural Networks" (in particular p. 96-97) where Mitchel uses the word "logistic function" and the "sigmoid function" synonymously -- this function he also calls the "squashing function" -- and the sigmoid (aka logistic) function is used to compress the outputs of the "neurons" in multi-layer neural nets.