(Solved): Consider a stationary stochastic process X1,X2, with XiX, a finite set. Suppose H is ...

Consider a stationary stochastic process $X_1,X_2,\dots$ with $X_i?\mathcal{X}$, a finite set. Suppose $\vec{H}$ is a constant so that $?\frac{1}{n}\log p\left(X_1,X_2,\dots ,X_n\right)?\stackrel{?}{H}$ in probability, i.e., for every $?>0$ ${\lim }_{n??}\mathrm{Pr}\left\{??\frac{1}{n}\log p\left(X_1,X_2,\dots ,X_n\right)?\stackrel{?}{H}?>?\right\}=0.$ For any integer $n>0$ and real number $?>0$, define $A_{?}^{(n)}$ as the set of sequences $\mathrm{x}?{\mathcal{X}}^n$ such that $2^{?n(\stackrel{?}{H}+?)}?p(\mathbf{x})?2^{?n(\stackrel{?}{H}??)}.$ (a) Show that ${\lim }_{n??}\mathrm{Pr}\left\{\left(X_1,\dots ,X_n\right)?A_{?}^{(n)}\right\}=1$. (b) Show that $?A_{?}^{(n)}??2^{n(\hat{H}+?)}$. (c) (Optional) A length $n$ code of this process is a subset $C?{\mathcal{X}}^n$. The error probability of the code is $P_e=1?{?}_{\mathbf{x}?C}p(\mathbf{x})$. Let $M^{?}(n,?)$ be the smallest cardinality of a length $n$ code with $P_e??$. Using the above two properties of $A_{?}^{(n)}$, show that for every $??(0,1)$, ${\lim }_{n??}\frac{\log M^{?}(n,?)}{n}=\stackrel{?}{H}.$ (Prove both the achievability and converse.)