一种可编程异构芯片设计方法应用于视频桥接

王潘丰; 蔡懿慈

doi:10.12263/DZXB.20240144

您当前的位置：

首页 >

文章列表页 >

一种可编程异构芯片设计方法应用于视频桥接

学术论文 | 更新时间：2025-12-08

- 一种可编程异构芯片设计方法应用于视频桥接
- A Programmable Heterogeneous Chip Designed for Video Bridging
- 电子学报 2025年53卷第1期页码：72-83
- 作者机构：
  
  1.清华大学计算机科学与技术系，北京 100084
  2.京微齐力（北京）科技股份有限公司，北京 100190
- 作者简介：
  
  [ "王潘丰男，1980年10月出生于江苏省徐州市.清华大学计算机科学与技术系创新领军工程博士生.主要研究领域为智能卡、手机基带芯片、可穿戴芯片、FPGA芯片等.E-mail: wpf20@mails.tsinghua.edu.cn" ]
  [ "蔡懿慈女，1960年7月出生于天津市.清华大学计算机科学与技术系教授、博士生导师.主要研究领域为集成电路计算机辅助设计软件、算法及软件系统.E-mail: caiyc@mail.tsinghua.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(62220106011)
- DOI：10.12263/DZXB.20240144
  中图分类号： TN402;
- 收稿：2024-02-05，
  
  修回：2024-05-17，
  
  纸质出版：2025-01-25
- 稿件说明：
移动端阅览
王潘丰, 蔡懿慈. 一种可编程异构芯片设计方法应用于视频桥接[J]. 电子学报, 2025, 53(01): 72-83.

WANG Pan-feng, CAI Yi-ci. A Programmable Heterogeneous Chip Designed for Video Bridging[J]. Acta Electronica Sinica, 2025, 53(01): 72-83.
王潘丰, 蔡懿慈. 一种可编程异构芯片设计方法应用于视频桥接[J]. 电子学报, 2025, 53(01): 72-83. DOI：10.12263/DZXB.20240144

WANG Pan-feng, CAI Yi-ci. A Programmable Heterogeneous Chip Designed for Video Bridging[J]. Acta Electronica Sinica, 2025, 53(01): 72-83. DOI：10.12263/DZXB.20240144

摘要

随着智能时代的到来，越来越多的设备拥有摄像头和显示屏，而它们具有各种各样不同接口和视频格式，视频桥接面临新的挑战.以往的解决方案是根据接口和视频格式的需求采用不同的电路，如现场可编程门阵列（Field Programmable Gate Array，FPGA）、图形处理器（Graphics Processing Unit，GPU）和专用集成电路（Application Specific Integrated Circuit，ASIC）等.但这种单一的电路模式难以同时满足低成本、超低功耗和小型化的要求，尤其是在移动显示领域.本文提出了一种新的异构体系架构，它将FPGA、微控制单元（MicroController Unit，MCU）、ASIC和存储器无缝集成到一个芯片中.该芯片不仅实现了小型化，而且具有低成本和低功耗的优势；更重要的是该款芯片可以支持不同接口和视频格式的桥接需求.针对不同算法的应用，本文给出了使用该芯片的评估方法和解决方案，为架构设计提供了依据.该芯片已成功在22 nm工艺流片，整体尺寸约为4 mm×4 mm，总功耗约为200 mW.它可以支持3 840 × 2 160分辨率和144 Hz刷新率的视频输入格式，1 080 × 2 340分辨率和90 Hz刷新率的视频输出格式.在实现同样视频桥接功能的应用时，本文所提芯片的面积和功耗均小于AMD芯片XC7K325T和Zynq Z7035的1/10.换而言之，针对此类场景的应用，本文方案在成本和功耗方面相比于传统商业FPGA有显著优化.

Abstract

With the development of intelligent era

more and more devices have cameras and display screens

which are in various video formats with different interfaces. To fill the gaps

video bridging is widely required. The previous solutions adopted field programmable gate array (FPGA)

graphics processing unit (GPU)

and application specific integrated circuits (ASIC). However

it is difficult to meet the requirements of low cost and ultra-low power consumption and miniaturization

especially in the field of mobile display. This paper proposes a novel heterogeneous architecture which seamlessly integrates FPGA

microcontrol unit (MCU)

ASIC

and memory into a single silicon chip. This chip not only achieves miniaturization

but also has the advantages of low cost and low power consumption; More importantly

this chip can support bridging requirements for different interfaces and video formats. At the same time

this paper provides evaluation methods and solutions for different algorithm applications

and provides a basis for architecture design. The chip has been successfully taped out in an industrial 22 nm process. It can support video input formats with a resolution of 3 840×2 160 and a refresh rate of 144 Hz

as well as video output formats with a resolution of 1 080×2 340 and a refresh rate of 90 Hz. The experimental results show that

in supporting the similar function

the overall chip size is about 4 mm×4 mm and the total power consumption is about only 200 mW

both of which are less than one tenth of AMD XC7K325T and Zynq Z7035. In other words

for applications in video bridging scenarios

our solution has significant optimization compared to traditional commercial FPGAs in terms of cost and power consumption.

关键词

Keywords

references

WANG Y Q , WANG C , ZHAO X Y , et al . A PCIe-based hardware acceleration architecture of the communication protocol stack [C ] // 2021 7th International Conference on Computer and Communications (ICCC) . Piscataway : IEEE , 2021 : 192 - 198 .

BASLI A , KAHRAMAN C , ATES B , et al . Field programmable gate arrays based multi camera thermal and day vision video processing and distribution system [C ] // 2021 International Conference on Innovations in Intelligent SysTems and Applications (INISTA) . Piscataway : IEEE , 2021 : 1 - 5 .

ELNAWAWY M , FARHAN A , AL NABULSI A , et al . Role of FPGA in internet of things applications [C ] // 2019 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT) . Piscataway : IEEE , 2019 : 1 - 6 .

MIELCZAREK A , MAKOWSKI D , PEREK P , et al . Framework for high-performance video acquisition and processing in MTCA.4 form factor [J ] . IEEE Transactions on Nuclear Science , 2019 , 66 ( 7 ): 1144 - 1150 .

BEN AMARA A , PISSALOUX E , ATRI M . Sobel edge detection system design and integration on an FPGA based HD video streaming architecture [C ] // 2016 11th International Design & Test Symposium (IDT) . Piscataway : IEEE , 2016 : 160 - 164 .

RANASINGHE N , BANGAMUARACHCHI R , SENEVIRATNE J , et al . SMPTE ST 2110 compliant scalable architecture on FPGA for end to end uncompressed professional video transport over IP networks [C ] // 2019 IEEE 30th International Conference on Application-specific Systems, Architectures and Processors (ASAP) . Piscataway : IEEE , 2019 : 235 - 238 .

GEIER M , PITZL F , CHAKRABORTY S . GigE vision data acquisition for visual servoing using SG/DMA proxying [C ] // Proceedings of the 14th ACM/IEEE Symposium on Embedded Systems for Real-Time Multimedia . New York : ACM , 2016 : 1 - 10 .

BAEK A R , LEE K , CHOI H . Speed-up image processing on mobile CPU and GPU [C ] // 2015 Asia Pacific Conference on Multimedia and Broadcasting . Piscataway : IEEE , 2015 : 1 - 3 .

ZHANG T , LIU X Y , WANG X D . High performance GPU tensor completion with tubal-sampling pattern [J ] . IEEE Transactions on Parallel and Distributed Systems , 2020 , 31 ( 7 ): 1724 - 1739 .

IGARASHI H , TAKANO F , MORIYOSHI T . Highly parallel transformation and quantization for HEVC encoder on GPUs [C ] // 2016 Visual Communications and Image Processing (VCIP) . Piscataway : IEEE , 2016 : 1 - 4 .

ZHANG X F , MA Y , XIONG J J , et al . Exploring HW/SW co-design for video analysis on CPU-FPGA heterogeneous systems [J ] . IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , 2022 , 41 ( 6 ): 1606 - 1619 .

LIN Y C , ZHANG B Y , PRASANNA V , et al . HP-GNN: Generating high throughput GNN training implementation on CPU-FPGA heterogeneous platform [C ] // Proceedings of the 2022 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays . New York : ACM , 2022 : 123 - 133 .

HUANG Q J , WANG D Q , DONG Z , et al . CoDeNet: Efficient deployment of input-adaptive object detection on embedded FPGAs [C ] // The 2021 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays . New York : ACM , 2021 : 206 - 216 .

ZUCHOWSKI P S , REYNOLDS C B , GRUPP R J , et al . A hybrid ASIC and FPGA architecture [C ] // IEEE/ACM International Conference on Computer Aided Design , 2002 . Piscataway : IEEE , 2002: 187 - 194 .

HO C H , YU C W , LEONG P H W , et al . Domain-specific hybrid FPGA: Architecture and floating point applications [C ] // 2007 International Conference on Field Programmable Logic and Applications . Piscataway : IEEE , 2007 : 196 - 201 .

JAIN A K , OMIDIAN H , FRAISSE H , et al . A domain-specific architecture for accelerating sparse matrix vector multiplication on FPGAs [C ] // 2020 30th International Conference on Field-Programmable Logic and Applications (FPL) . Piscataway : IEEE , 2020 : 127 - 132 .

PARVEZ H , MARRAKCHI Z , MEHREZ H . Application specific FPGA using heterogeneous logic blocks [C ] // Reconfigurable Computing: Architectures, Tools and Applications . Berlin : Springer , 2010 : 92 - 109 .

ZHAO R Z , NG H C , LUK W , et al . Towards efficient convolutional neural network for domain-specific applications on FPGA [C ] // 2018 28th International Conference on Field Programmable Logic and Applications (FPL) . Piscataway : IEEE , 2018 : 147 - 1477 .

CUI G C , MENG T , SHI Y J , et al . Design of heterogeneous reconfigurable cipher engine basing on FPGA ASIC [C ] // 2019 5th International Conference on Big Data Computing and Communications (BIGCOM) . Piscataway : IEEE , 2019 : 261 - 265 .

RAMACHANDRA C N , NAG A , BALASUBRAMONION R , et al . ONT-X: An FPGA approach to real-time portable genomic analysis [C ] // 2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) . Piscataway : IEEE , 2021 : 268 - 269 .

KEFALAS N , THEODORIDIS G . Implementing VESA display stream compression encoder in FPGAs [C ] // 2019 29th International Symposium on Power and Timing Modeling, Optimization and Simulation (PATMOS) . Piscataway : IEEE , 2019 : 35 - 40 .

朱秀昌 , 唐贵进 . 视频显示流压缩的技术与标准 [J ] . 数据采集与处理 , 2017 , 32 ( 3 ): 469 - 478 .

ZHU X C , TANG G J . Video display stream compression technologies and standards [J ] . Journal of Data Acquisition and Processing , 2017 , 32 ( 3 ): 469 - 478 . (in Chinese)

Vedio Electronics Standards Association . VESA display stream compression (DSC) standard [S/OL ] . ( 2017-01-18 )[ 2024-05-26 ] . https://glenwing.github.io/docs/VESA-DSC-1.2a.pdf https://glenwing.github.io/docs/VESA-DSC-1.2a.pdf .

WALLS F , MACINNIS A S . 27.4L: Late-news paper : VESA display stream compression: An overview [J ] . SID Symposium Digest of Technical Papers , 2014 , 45 ( 1 ): 360 - 363 .

彭春萍 . 颜色空间转换在视频采集显示系统中的应用 [J ] . 工业控制计算机 , 2012 , 25 ( 8 ): 22 - 23, 25 .

PENG C P . Color space conversion in video sampling and display system [J ] . Industrial Control Computer , 2012 , 25 ( 8 ): 22 - 23, 25 . (in Chinese)

MAEDA Y , FUKUSHIMA N , HAMAMOTO T . Color transformation for compressive computing in image filtering [C ] // 2021 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) . Piscataway : IEEE , 2021 : 88 - 92 .

ITU-R Sector . Studio encoding parameters of digital television for standard 4:3 and wide-screen 16:9 aspect ratios [S/OL ] . ( 1995-10-01 )[ 2024-05-26 ] . https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.601-6-200701-S!!PDF-E.pdf .

SURESH C , SINGH S , SAINI R , et al . A comparative analysis of image scaling algorithms [J ] . International Journal of Image, Graphics and Signal Processing , 2013 , 5 ( 5 ): 55 - 62 .

LIN C C , SHEU M H , CHIANG H K , et al . An efficient architecture of extended linear interpolation for image processing [J ] . Journal of Information Science and Engineering , 2010 , 26 ( 2 ): 631 - 648 .

MOON H M , KO K R , SHIN J , et al . A novel double linear-cubic convolution interpolation for digital image scaling [C ] // Proceedings of the 29th Annual ACM Symposium on Applied Computing . New York : ACM , 2014 : 1733 - 1734 .

CHEN S L . VLSI implementation of a low-cost high-quality image scaling processor [J ] . IEEE Transactions on Circuits and Systems II: Express Briefs , 2013 , 60 ( 1 ): 31 - 35 .

RAMESH G , PRASATH T A . An aphoristic study on different interpolation techniques for medical image scaling and its comparative analysis [C ] // 2021 International Conference on Computer Communication and Informatics (ICCCI) . Piscataway : IEEE , 2021 : 1 - 4 .

王莹 , 王晶 , 高岚 , 等 . 一种注意力机制优化方法及硬件加速设计 [J ] . 电子学报 , 2023 , 51 ( 4 ): 1021 - 1029 .

WANG Y , WANG J , GAO L , et al . An improved attention mechanism algorithm model and hardware aceleration design method [J ] . Acta Electronica Sinica , 2023 , 51 ( 4 ): 1021 - 1029 . (in Chinese)

BOUTROS A , YAZDANSHENAS S , BETZ V . You cannot improve what you do not measure: FPGA vs. ASIC efficiency gaps for convolutional neural network inference [J ] . ACM Transactions on Reconfigurable Technology and Systems , 2018 , 11 ( 3 ): 1 - 23 .

KUON I , ROSE J . Measuring the gap between FPGAs and ASICs [J ] . IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , 2007 , 26 ( 2 ): 203 - 215 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

一种应用于物联网传感器的伪三阶Delta-Sigma调制器

工艺-电压-温度综合稳健的亚1 V 10位SAR ADC

基于可重构MUXs网络的低功耗测试数据压缩方法

基于符号率动态变频的低功耗DOCSIS上行发射器设计

基于注意力惩罚和自适应学习的场景图增强联合多模态方面情感分析