The complexity of H.264 hardware intra encoder is remarkable due to 17 prediction modes and Rate-distortion optimization mode decision.Although reduce coding complexity greatly

most of current intra prediction mode decision algorithms are hard to implement in hardware.A new mode decision algorithm for VLSI implementation is proposed based on Sobel edge detection mode decision algorithm.Our algorithm decreases memory bandwidth by modifying Sobel edge detector of part pixels

reduces hardware design complexity by optimizing mode region range

and simplifies cost computations of each prediction mode by using SATD.The results of experiment and hardware design show that our algorithm can greatly reduce the complexity of hardware implementation and significantly improve the efficiency of coder while coding quality is decreased slightly.