Chaubey, Yogendra P. 
ORCID: https://orcid.org/0000-0002-0234-1429 and Vu, Nhat Linh
(2020)
A Numerical Study of Entropy and Residual Entropy Estimators Based on Smooth Density Estimators for Non-negative Random Variables.
Technical Report.
Concordia University, Montreal.
(Submitted)
Preview |
Text (application/pdf)
1MB2020_1_TR.pdf - Submitted Version Available under License Spectrum Terms of Access. |
Abstract
In this paper, we are interested in the entropy of a non-negative random variable. Since the underlying probability density function is unknown, we propose the use of Poisson smoothed histogram density estimator in order to estimate the entropy. To study the performance of our estimator, we run simulations on a wide range of densities and compare our entropy estimators with the existing estimators that based on different approaches such as spacing estimators. Furthermore, we extend our study to residual entropy estimators which is the entropy of a random variable given that it has been survived up to time t.
| Divisions: | Concordia University > Faculty of Arts and Science > Mathematics and Statistics |
|---|---|
| Item Type: | Monograph (Technical Report) |
| Authors: | Chaubey, Yogendra P. and Vu, Nhat Linh |
| Series Name: | Technical Report Series,Dep. of Mathematics and Statistics |
| Institution: | Concordia Univeristy |
| Date: | 3 March 2020 |
| Funders: |
|
| Identification Number: | Technical Report No. 1/20, March 2020 |
| Keywords: | Entropy estimator, information theory, residual entropy, density estimator, survival function. |
| ID Code: | 986481 |
| Deposited By: | Yogen Chaubey |
| Deposited On: | 21 Apr 2020 19:24 |
| Last Modified: | 23 Apr 2020 18:22 |
References:
[1] Belzunce, F., Guillamon, A., Navarro, J., and Ruiz, J. M. (2001) Kernel esitmation of residualentropy. Communications in Statistics-Theory and Methods 30:1243-1255.
[2] Bouzebda, S., Elhattab, I., Keziou, A., and Lounis, T. (2013) New entropy estimator with an
application to test of normality. Communication in Statistics-Theory and Methods 42:2245-2270.
[3] Bouezmarni, T. and Scaillet, O. (2005) Consitency of asymmetric kernel density estimators and
smoothed histograms with application to income data. Econometric Theory 21:390-412.
[4] Chaubey, Y. P., Mudholkar, G. S. and Smethurst, P. A. (1993). On entropy-based goodness-of-fit
tests: a practical strategy. In Probability and Statistics, S. K. Basu and B. K. Sinha, eds., Narosa
Publishing House, New Delhi, 116-120.
[5] Chaubey, Y. P. and Sen, P. K. (2009). On The Selection Of The Smoothing Parameter In Poisson
Smoothing Of Histogram Estimator: Computational Aspects. Pakistan Journal of Statistics 25(4):
385-401.
[6] Chaubey, Y. P, Sen, P., and Li, J. (2010) Smooth density estimation for length-biased data. Journal
of The Indian Society of Agricultural Statistics 64(2):145-155.
[7] Cheng, C. and Parzen, E. (1997) Unified estimators of smooth quantile and quantile density funnc-
tions. Journal of Statistical Planning and Inference 59:291-307.
[8] Correa, J. C. (1995). A new estimator of entropy. Communications in Statistics: Theory and Methods
24:2439-2449.
[9] Cover, T. M. and Thomas J. A. (1991). Elements of Information Theory. Wiley, New York, USA.
[10] Ebrahimi, N., Habibullah, M., and Soofi, E. (1992) Testing exponentiality based on KullbackLeibler
information. Journal of the Royal Statistical Society: Series B, 54:739-748.
[11] Ebrahimi, N. (1996). How to measure uncertainty in the life time distributions. The Indian Journal
of Statistics Series A 58:48-56.
[12] Eggermont, P.B. and LaRiccia V. N. (1999). Best Asymptotic Normality of the Kernel Density
Entropy Estimator for Smooth Densities. IEEE Transactions On Information Theory 45:1321-1326
[13] Hall, P. and Morton, S. C. (1993). On the estimation of entropy. Ann. Inst. Statist. Math. 45:69-88.
[14] Kapur, J. N. (1993). Maximum Entropy Models in Science and Engineering. John Wiley & Sons,
Inc., New York, USA.
[15] Kapur, J. N., Kesavan, H. K. (1992). Entropy optimization principles with applications. Academic
press, Inc., New York.
[16] Noughabi, H. A. (2010). A new estimator of entropy and its application in testing normality.Journal
of Statistical Computation And Simulation 80:1151-1162.
[17] Paninski, L. (2003). Estimation of entropy and mutual information. Neural Computation 15:1191-
1253.
[18] Shannon, C.E. (1948). A mathematical theory of ommunication. Bell System Tech. J. 27:379-423 &
623-656.
[19] Van Es, B. (1992). Estimating functionals related to a density by a class of statistics based on
spacings. Scandinavian Journal of Statistics 19:61-72.
[20] Vasicek, O. (1976). A test for normality based on sample entropy. Journal of the Royal Statistical
Society 38:54-59.
[21] Wieczorkowski, R. and Grzegorzewski, P. (1999). Entropy estimators improvements and comparisons.
Communications in Statistics-Simulation and Computation 28(2):541-567.
All items in Spectrum are protected by copyright, with all rights reserved. The use of items is governed by Spectrum's terms of access.
Repository Staff Only: item control page
Download Statistics
Download Statistics