The class of locally stationary processes assumes a time-varying (tv) spectral representation and finite second moment. Different areas have observed phenomena with heavy tail distributions or infinite variance. Using stable distribution as a heavy-tailed innovation is an attractive option. However, its estimation is difficult due to the absence of a closed expression for the density function and the non-existence of second moment. In this paper, we propose the tvARMA model with tempered stable innovations, which have lighter tails than the stable distribution and have finite moments. A two-step method is proposed to estimate this parametric model. In the first step, we use the blocked Whittle estimation to estimate the time-varying structure of the process. In the second step, we recover residuals from the first step and use the maximum likelihood method to estimate the rest of the parameters related to the standardized classical tempered stable (stdCTS) innovations. We perform simulation studies to evaluate the consistency of the maximum likelihood estimation of independent stdCTS samples. Then, we execute simulations to study the two-step estimation of our model. Finally, an empirical application is illustrated.