Abstract—In video compression, it can reduce the redundant information efficiently by motion estimation (ME) and motion compensation (MC), and less code is used to encode as much information as possible. In this paper, I frame encoding adopts wavelet transform and set partitioning in hierarchical trees (SPIHT) algorithm; for P frames, each frame sets the reconstructed frame of its previous frame as a reference frame, and then P frames proceed to code with ME and MC. In the step of ME, trading off between accuracy and computational complexity, adaptive fast search (AFS) algorithm combining with nodal search-based deformable block matching algorithm (NS-DBMA) is adopted to search for the matched block; and then in the MC process, wavelet transform combining with zerotree entropy (ZTE) algorithm is adopted according to the characteristics of residual image data. Meanwhile rate control is carried out in ZTE algorithm. Experimental results show that the proposed algorithm performs well for video sequences with complex motion and rich details, and reduces blocking artifacts obviously.