A Programmable Heterogeneous Chip Designed for Video Bridging

WANG Pan-feng; CAI Yi-ci

doi:10.12263/DZXB.20240144

您当前的位置：

首页 >

文章列表页 >

A Programmable Heterogeneous Chip Designed for Video Bridging

PAPERS | 更新时间：2025-12-08

- A Programmable Heterogeneous Chip Designed for Video Bridging
- ACTA ELECTRONICA SINICA Vol. 53, Issue 1, Pages: 72-83(2025)
- 作者机构：
  
  1.清华大学计算机科学与技术系，北京 100084
  2.京微齐力（北京）科技股份有限公司，北京 100190
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(62220106011)
- DOI：10.12263/DZXB.20240144
  CLC： TN402;
- Received：05 February 2024，
  
  Revised：2024-05-17，
  
  Published：25 January 2025
- 稿件说明：
移动端阅览
王潘丰, 蔡懿慈. 一种可编程异构芯片设计方法应用于视频桥接[J]. 电子学报, 2025, 53(01): 72-83.

WANG Pan-feng, CAI Yi-ci. A Programmable Heterogeneous Chip Designed for Video Bridging[J]. Acta Electronica Sinica, 2025, 53(01): 72-83.
王潘丰, 蔡懿慈. 一种可编程异构芯片设计方法应用于视频桥接[J]. 电子学报, 2025, 53(01): 72-83. DOI：10.12263/DZXB.20240144

WANG Pan-feng, CAI Yi-ci. A Programmable Heterogeneous Chip Designed for Video Bridging[J]. Acta Electronica Sinica, 2025, 53(01): 72-83. DOI：10.12263/DZXB.20240144

摘要

随着智能时代的到来，越来越多的设备拥有摄像头和显示屏，而它们具有各种各样不同接口和视频格式，视频桥接面临新的挑战.以往的解决方案是根据接口和视频格式的需求采用不同的电路，如现场可编程门阵列（Field Programmable Gate Array，FPGA）、图形处理器（Graphics Processing Unit，GPU）和专用集成电路（Application Specific Integrated Circuit，ASIC）等.但这种单一的电路模式难以同时满足低成本、超低功耗和小型化的要求，尤其是在移动显示领域.本文提出了一种新的异构体系架构，它将FPGA、微控制单元（MicroController Unit，MCU）、ASIC和存储器无缝集成到一个芯片中.该芯片不仅实现了小型化，而且具有低成本和低功耗的优势；更重要的是该款芯片可以支持不同接口和视频格式的桥接需求.针对不同算法的应用，本文给出了使用该芯片的评估方法和解决方案，为架构设计提供了依据.该芯片已成功在22 nm工艺流片，整体尺寸约为4 mm×4 mm，总功耗约为200 mW.它可以支持3 840 × 2 160分辨率和144 Hz刷新率的视频输入格式，1 080 × 2 340分辨率和90 Hz刷新率的视频输出格式.在实现同样视频桥接功能的应用时，本文所提芯片的面积和功耗均小于AMD芯片XC7K325T和Zynq Z7035的1/10.换而言之，针对此类场景的应用，本文方案在成本和功耗方面相比于传统商业FPGA有显著优化.

Abstract

With the development of intelligent era

more and more devices have cameras and display screens

which are in various video formats with different interfaces. To fill the gaps

video bridging is widely required. The previous solutions adopted field programmable gate array (FPGA)

graphics processing unit (GPU)

and application specific integrated circuits (ASIC). However

it is difficult to meet the requirements of low cost and ultra-low power consumption and miniaturization

especially in the field of mobile display. This paper proposes a novel heterogeneous architecture which seamlessly integrates FPGA

microcontrol unit (MCU)

ASIC

and memory into a single silicon chip. This chip not only achieves miniaturization

but also has the advantages of low cost and low power consumption; More importantly

this chip can support bridging requirements for different interfaces and video formats. At the same time

this paper provides evaluation methods and solutions for different algorithm applications

and provides a basis for architecture design. The chip has been successfully taped out in an industrial 22 nm process. It can support video input formats with a resolution of 3 840×2 160 and a refresh rate of 144 Hz

as well as video output formats with a resolution of 1 080×2 340 and a refresh rate of 90 Hz. The experimental results show that

in supporting the similar function

the overall chip size is about 4 mm×4 mm and the total power consumption is about only 200 mW

both of which are less than one tenth of AMD XC7K325T and Zynq Z7035. In other words

for applications in video bridging scenarios

our solution has significant optimization compared to traditional commercial FPGAs in terms of cost and power consumption.

关键词

Keywords

references

WANG Y Q , WANG C , ZHAO X Y , et al . A PCIe-based hardware acceleration architecture of the communication protocol stack [C ] // 2021 7th International Conference on Computer and Communications (ICCC) . Piscataway : IEEE , 2021 : 192 - 198 .

BASLI A , KAHRAMAN C , ATES B , et al . Field programmable gate arrays based multi camera thermal and day vision video processing and distribution system [C ] // 2021 International Conference on Innovations in Intelligent SysTems and Applications (INISTA) . Piscataway : IEEE , 2021 : 1 - 5 .

ELNAWAWY M , FARHAN A , AL NABULSI A , et al . Role of FPGA in internet of things applications [C ] // 2019 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT) . Piscataway : IEEE , 2019 : 1 - 6 .

MIELCZAREK A , MAKOWSKI D , PEREK P , et al . Framework for high-performance video acquisition and processing in MTCA.4 form factor [J ] . IEEE Transactions on Nuclear Science , 2019 , 66 ( 7 ): 1144 - 1150 .

BEN AMARA A , PISSALOUX E , ATRI M . Sobel edge detection system design and integration on an FPGA based HD video streaming architecture [C ] // 2016 11th International Design & Test Symposium (IDT) . Piscataway : IEEE , 2016 : 160 - 164 .

RANASINGHE N , BANGAMUARACHCHI R , SENEVIRATNE J , et al . SMPTE ST 2110 compliant scalable architecture on FPGA for end to end uncompressed professional video transport over IP networks [C ] // 2019 IEEE 30th International Conference on Application-specific Systems, Architectures and Processors (ASAP) . Piscataway : IEEE , 2019 : 235 - 238 .

GEIER M , PITZL F , CHAKRABORTY S . GigE vision data acquisition for visual servoing using SG/DMA proxying [C ] // Proceedings of the 14th ACM/IEEE Symposium on Embedded Systems for Real-Time Multimedia . New York : ACM , 2016 : 1 - 10 .

BAEK A R , LEE K , CHOI H . Speed-up image processing on mobile CPU and GPU [C ] // 2015 Asia Pacific Conference on Multimedia and Broadcasting . Piscataway : IEEE , 2015 : 1 - 3 .

ZHANG T , LIU X Y , WANG X D . High performance GPU tensor completion with tubal-sampling pattern [J ] . IEEE Transactions on Parallel and Distributed Systems , 2020 , 31 ( 7 ): 1724 - 1739 .

IGARASHI H , TAKANO F , MORIYOSHI T . Highly parallel transformation and quantization for HEVC encoder on GPUs [C ] // 2016 Visual Communications and Image Processing (VCIP) . Piscataway : IEEE , 2016 : 1 - 4 .

ZHANG X F , MA Y , XIONG J J , et al . Exploring HW/SW co-design for video analysis on CPU-FPGA heterogeneous systems [J ] . IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , 2022 , 41 ( 6 ): 1606 - 1619 .

LIN Y C , ZHANG B Y , PRASANNA V , et al . HP-GNN: Generating high throughput GNN training implementation on CPU-FPGA heterogeneous platform [C ] // Proceedings of the 2022 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays . New York : ACM , 2022 : 123 - 133 .

HUANG Q J , WANG D Q , DONG Z , et al . CoDeNet: Efficient deployment of input-adaptive object detection on embedded FPGAs [C ] // The 2021 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays . New York : ACM , 2021 : 206 - 216 .

ZUCHOWSKI P S , REYNOLDS C B , GRUPP R J , et al . A hybrid ASIC and FPGA architecture [C ] // IEEE/ACM International Conference on Computer Aided Design , 2002 . Piscataway : IEEE , 2002: 187 - 194 .

HO C H , YU C W , LEONG P H W , et al . Domain-specific hybrid FPGA: Architecture and floating point applications [C ] // 2007 International Conference on Field Programmable Logic and Applications . Piscataway : IEEE , 2007 : 196 - 201 .

JAIN A K , OMIDIAN H , FRAISSE H , et al . A domain-specific architecture for accelerating sparse matrix vector multiplication on FPGAs [C ] // 2020 30th International Conference on Field-Programmable Logic and Applications (FPL) . Piscataway : IEEE , 2020 : 127 - 132 .

PARVEZ H , MARRAKCHI Z , MEHREZ H . Application specific FPGA using heterogeneous logic blocks [C ] // Reconfigurable Computing: Architectures, Tools and Applications . Berlin : Springer , 2010 : 92 - 109 .

ZHAO R Z , NG H C , LUK W , et al . Towards efficient convolutional neural network for domain-specific applications on FPGA [C ] // 2018 28th International Conference on Field Programmable Logic and Applications (FPL) . Piscataway : IEEE , 2018 : 147 - 1477 .

CUI G C , MENG T , SHI Y J , et al . Design of heterogeneous reconfigurable cipher engine basing on FPGA ASIC [C ] // 2019 5th International Conference on Big Data Computing and Communications (BIGCOM) . Piscataway : IEEE , 2019 : 261 - 265 .

RAMACHANDRA C N , NAG A , BALASUBRAMONION R , et al . ONT-X: An FPGA approach to real-time portable genomic analysis [C ] // 2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) . Piscataway : IEEE , 2021 : 268 - 269 .

KEFALAS N , THEODORIDIS G . Implementing VESA display stream compression encoder in FPGAs [C ] // 2019 29th International Symposium on Power and Timing Modeling, Optimization and Simulation (PATMOS) . Piscataway : IEEE , 2019 : 35 - 40 .

朱秀昌 , 唐贵进 . 视频显示流压缩的技术与标准 [J ] . 数据采集与处理 , 2017 , 32 ( 3 ): 469 - 478 .

ZHU X C , TANG G J . Video display stream compression technologies and standards [J ] . Journal of Data Acquisition and Processing , 2017 , 32 ( 3 ): 469 - 478 . (in Chinese)

Vedio Electronics Standards Association . VESA display stream compression (DSC) standard [S/OL ] . ( 2017-01-18 )[ 2024-05-26 ] . https://glenwing.github.io/docs/VESA-DSC-1.2a.pdf https://glenwing.github.io/docs/VESA-DSC-1.2a.pdf .

WALLS F , MACINNIS A S . 27.4L: Late-news paper : VESA display stream compression: An overview [J ] . SID Symposium Digest of Technical Papers , 2014 , 45 ( 1 ): 360 - 363 .

彭春萍 . 颜色空间转换在视频采集显示系统中的应用 [J ] . 工业控制计算机 , 2012 , 25 ( 8 ): 22 - 23, 25 .

PENG C P . Color space conversion in video sampling and display system [J ] . Industrial Control Computer , 2012 , 25 ( 8 ): 22 - 23, 25 . (in Chinese)

MAEDA Y , FUKUSHIMA N , HAMAMOTO T . Color transformation for compressive computing in image filtering [C ] // 2021 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) . Piscataway : IEEE , 2021 : 88 - 92 .

ITU-R Sector . Studio encoding parameters of digital television for standard 4:3 and wide-screen 16:9 aspect ratios [S/OL ] . ( 1995-10-01 )[ 2024-05-26 ] . https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.601-6-200701-S!!PDF-E.pdf .

SURESH C , SINGH S , SAINI R , et al . A comparative analysis of image scaling algorithms [J ] . International Journal of Image, Graphics and Signal Processing , 2013 , 5 ( 5 ): 55 - 62 .

LIN C C , SHEU M H , CHIANG H K , et al . An efficient architecture of extended linear interpolation for image processing [J ] . Journal of Information Science and Engineering , 2010 , 26 ( 2 ): 631 - 648 .

MOON H M , KO K R , SHIN J , et al . A novel double linear-cubic convolution interpolation for digital image scaling [C ] // Proceedings of the 29th Annual ACM Symposium on Applied Computing . New York : ACM , 2014 : 1733 - 1734 .

CHEN S L . VLSI implementation of a low-cost high-quality image scaling processor [J ] . IEEE Transactions on Circuits and Systems II: Express Briefs , 2013 , 60 ( 1 ): 31 - 35 .

RAMESH G , PRASATH T A . An aphoristic study on different interpolation techniques for medical image scaling and its comparative analysis [C ] // 2021 International Conference on Computer Communication and Informatics (ICCCI) . Piscataway : IEEE , 2021 : 1 - 4 .

王莹 , 王晶 , 高岚 , 等 . 一种注意力机制优化方法及硬件加速设计 [J ] . 电子学报 , 2023 , 51 ( 4 ): 1021 - 1029 .

WANG Y , WANG J , GAO L , et al . An improved attention mechanism algorithm model and hardware aceleration design method [J ] . Acta Electronica Sinica , 2023 , 51 ( 4 ): 1021 - 1029 . (in Chinese)

BOUTROS A , YAZDANSHENAS S , BETZ V . You cannot improve what you do not measure: FPGA vs. ASIC efficiency gaps for convolutional neural network inference [J ] . ACM Transactions on Reconfigurable Technology and Systems , 2018 , 11 ( 3 ): 1 - 23 .

KUON I , ROSE J . Measuring the gap between FPGAs and ASICs [J ] . IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , 2007 , 26 ( 2 ): 203 - 215 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Pseudo Third-Order Delta-Sigma Modulator Applied to Internet of Things Sensors

A Sub-1 V 10 bit SAR ADC Robust Against Process-Voltage-Temperature Variation

Low Power Test Data Compression Technique Based on Reconfigurable MUXs Network

An Dynamic Frequency Scaling Low Power Architecture Based on Symbol Rate for DOCSIS Upstream Transmitter

Feature Masking and Contrastive Learning Integrating Multi-Dimensional Decorrelation in Sequential Recommendation

Related Author

WEI Cong

HUANG Li-jie

HU Wei

WEI Rong-shan

ZHANG Chang

TONG Xing-yuan

LIU Jun

WU Xi

Related Institution

School of Physics and Information Engineering, Fuzhou University

Shaanxi Provincial Research Center for Telecommunication ASIC Design， Xi’an University of Posts and Telecommunications

Graduate University of Chinese Academy of SciencesBeijing 100490China

School of Computer and Information, Hefei University of Technology

Graduate University of Chinese Academy of Sciences

⁰