Entropy measuring algorithms

List of algorithms from NIST 800-90B, Schürmann and Grassberger, Hutter Prize entries and Gupta and Agarwal deep learning techniques for entropy measurement in the general (correlated/ non-IID) case:-

List (incomplete) of entropy measuring algorithms:-

  1. Most Common Value Estimate.

  2. Collision Estimate.

  3. Markov Estimate.

  4. Compression Estimate.

  5. t-Tuple Estimate.

  6. Longest Repeated Substring (LRS) Estimate.

  7. Multi Most Common in Window Prediction Estimate.

  8. Lag Prediction Estimate.

  9. MultiMMC Prediction Estimate.

  10. LZ78Y Prediction Estimate.

  11. Ziv - Lempel.

  12. Gambling & suffix trees.

  13. Bayesian probability estimation.

  14. Rissanen’s method.

  15. Superposition of probabilities.

  16. Global probability estimates.

  17. Hutter Prize entries.

  18. Ouija technique for asking Shannon himself.

  19. And a shed load of deep learning/artificial intelligence techniques…

Deep learning compression methods for complex and long range correlations.

Deep learning compression methods for complex and long range correlations.

And there are others still, with a significant proportion based around compression theory. So what to do?