Under the condition of unauthorized receivers and cognitive radios

the problem of detecting the number of transmit antennas in MIMO system with multi-path channel is to be solved.To achieve this goal

we analyze the invalidation of presenting model and transform the MIMO multi-path channel model into virtual channel matrix to create the model of detecting the number of transmit antennas in MIMO system with multi-path channel.Then

the result of least eigen-value of covariance in random matrix theory is applied to the proof

which shows that the least eigen-value of channel covariance whose element obeys the Rayleigh distribution

converges in distribution to the Tracy-Widom law TW2.The effect on detecting the number of transmit antennas of this property is analyzed and an improved algorithm based on RMT

which could detect the number of transmit antennas in MIMO system with multi-path channel

is proposed.At last

the simulation shows that compared with the RMT algorithm the improved algorithm has better performance

in the condition of low SNR and small data size.