Measuring Non-Gaussianity Using Negentropy

In signal processing and machine learning, negentropy serves as a widely adopted measure of non-Gaussianity, particularly suitable for tasks like Independent Component Analysis (ICA). The core principle of negentropy derives from information entropy, quantifying non-Gaussianity by comparing the divergence between a target signal's distribution and a Gaussian distribution.

Negentropy is defined based on the divergence between a signal's probability density function and a Gaussian distribution. For a random variable, entropy measures uncertainty, while a Gaussian distribution possesses the maximum entropy among all distributions with identical variance. Thus, negentropy calculates the difference between the entropy of the target signal and the entropy of a Gaussian signal with the same covariance. A higher negentropy value indicates stronger non-Gaussian characteristics in the signal.

In signal separation tasks, the advantage of negentropy lies in its sensitivity to non-Gaussian signals. Traditional ICA algorithms typically leverage this property to identify independent components with maximum non-Gaussianity. Compared to other non-Gaussianity measures like kurtosis, negentropy demonstrates superior robustness, especially when handling heavy-tailed or asymmetric distributions.

In practical implementations, negentropy can be efficiently computed using approximation methods, such as employing higher-order cumulants or nonlinear function approximations. For example, a common implementation in Python using FastICA would involve:

from sklearn.decomposition import FastICA
# Initialize ICA with negentropy-based contrast function
ica = FastICA(algorithm='parallel', fun='logcosh')
components = ica.fit_transform(mixed_signals)

This makes negentropy a core tool in many blind source separation algorithms, effectively isolating independent components from mixed signals.

In summary, negentropy provides a reliable method for measuring non-Gaussianity, holding significant application value in signal separation and feature extraction. Its successful implementation not only enhances separation performance but also advances related theoretical developments.