Submit to Electronics Review for Electronics Propose a Special Issue

Journal Browser

Hardware Architectures for Real Time Image Processing

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Keywords
Published Papers

A special issue of Electronics (ISSN 2079-9292). This special issue belongs to the section "Circuit and Signal Processing".

Deadline for manuscript submissions: closed (15 June 2022) | Viewed by 33664

Share This Special Issue

Special Issue Editors

Prof. Gustavo Marrero-Callico

E-Mail Website
Guest Editor

Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), Las Palmas de Gran Canaria, Spain
Interests: hyperspectral image processing; artificial intelligence algorithms; hardware architectures for real-time image processing; super-resolution image enhancement; circuits for multimedia processing and video coding standards; computer microarchitecture; synthesis-based design for SOCs; hardware–software codesign
Special Issues, Collections and Topics in MDPI journals

Dr. Himar Fabelo

E-Mail Website
Co-Guest Editor

Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), 35017 Las Palmas de Gran Canaria, Spain
Interests: hyperspectral imaging; brain cancer; machine learning; algorithm development and acceleration; medical hyperspectral intraoperative instrumentation
Special Issues, Collections and Topics in MDPI journals

Dr. Lucana Santos

E-Mail Website
Co-Guest Editor

European Space Agency European Space Research and Technology Centre, 2312RD Leiden, The Netherlands
Interests: hardware architectures for on-board data processing; reconfigurable architectures and hardware/software co-design methodologies

Special Issue Information

Dear Colleagues,

The development of hardware architectures for real-time image processing is an important requirement when dealing with critical applications where data must be processed in a restricted timeframe. Moreover, it is an interdisciplinary field that gives support to a wide variety of disciplines, such as electronic engineering, computer science, industrial control, physics, mathematics, biology, food quality assessment, medicine, etc. Although these circumstances arise in several applications, and several solutions have been proposed in the past, the large amount of data per unit of time currently provided by image sensors supposes a major challenge for feasible hardware implementations. Not only have the frame-rates and the resolution of the images increased, but additionally, some image modalities incorporate spectral information, depth information, contextual information, metadata, etc., making the implementation requirements even more stringent. In addition to time processing, many systems must meet constraints on weight, size, power or cost, resulting in exhaustive and time-prohibitive design-space explorations. To cope with these complex situations, several solutions have been addressed, including (but not limited) compressive sensing for data reduction, multimodal image and video compression, efficient high-speed memory management at different levels for high-bandwidth communications, fast interfaces for data transactions, high-level synthesis methodologies for high-performance implementations, heterogeneous implementations combining ASICS, FPGAs, GPUs, manycore processors, etc. This Special Issue on “Hardware Architectures for Real-Time Image Processing” aims to include different implementation solutions for real-time image processing, provided that such solutions suppose a significant scientific contribution in the field.

Dr. Gustavo Marrero Callico
Dr. Himar Fabelo
Dr. Lucana Santos
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Electronics is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

High-level synthesis for image processing architectures
Hardware architectures for Deep Learning and Machine Learning
Hardware accelerators for real-time image processing
Fast prototyping of hardware architectures for image processing
Verification of hardware architectures for image processing
Heterogeneous system for real-time image processing
Hardware architectures for video processing
IoT architectures for image processing
Hardware architectures for medical applications
Hardware architectures for critical-time applications

Published Papers (10 papers)

Download All Papers

Research

Jump to: Review

16 pages, 2136 KiB

Open AccessArticle

Resource-Efficient Hardware Implementation of Perspective Transformation Based on Central Projection

by Zeying Li, Weijiang Wang, Chengbo Xue and Rongkun Jiang

Electronics 2022, 11(9), 1367; https://doi.org/10.3390/electronics11091367 - 25 Apr 2022

Cited by 2 | Viewed by 1481

Abstract

Perspective correction of images is an important preprocessing task in computer vision applications, which can resolve distortions caused by shooting angles, etc. This paper proposes a hardware implementation of perspective transformation based on central projection, which is simpler than the homography transformation method. In particular, it does not need to solve complex equations, thus no software assistance is required. The design can be flexibly configured with different degrees of parallelism to meet different speed requirements. Implemented on the Xilinx Zynq-7000 platform, 2893 Look-up Tables (LUTs) are required when the parallelism is one, and it can process a 20 Hz video with a resolution of 640 × 480 in real time. When the parallelism is eight, it can process 157 Hz video and requires 11,223 LUTs. The proposed design can well meet the actual needs. Full article

(This article belongs to the Special Issue Hardware Architectures for Real Time Image Processing)

► Show Figures

Figure 1

22 pages, 4994 KiB

Open AccessArticle

An Efficient Dual-Channel Data Storage and Access Method for Spaceborne Synthetic Aperture Radar Real-Time Processing

by Guoqing Wang, He Chen and Yizhuang Xie

Electronics 2021, 10(6), 662; https://doi.org/10.3390/electronics10060662 - 12 Mar 2021

Cited by 3 | Viewed by 2194

Abstract

With the development of remote sensing technology and very large-scale integrated circuit (VLSI) technology, the real-time processing of spaceborne Synthetic Aperture Radar (SAR) has greatly improved the ability of Earth observation. However, the characteristics of external memory have led to matrix transposition becoming a technical bottleneck that limits the real-time performance of the SAR imaging system. In order to solve this problem, this paper combines the optimized data mapping method and reasonable hardware architecture to implement a data controller based on the Field-Programmable Gate Array (FPGA). First of all, this paper proposes an optimized dual-channel data storage and access method, so that the two-dimensional data access efficiency can be improved. Then, a hardware architecture is designed with register manager, simplified address generator and dual-channel Double-Data-Rate Three Synchronous Dynamic Random-Access Memory (DDR3 SDRAM) access mode. Finally, the proposed data controller is implemented on the Xilinx XC7VX690T FPGA chip. The experimental results show that the reading efficiency of the data controller proposed is 80% both in the range direction and azimuth direction, and the writing efficiency is 66% both in the range direction and azimuth direction. The results of a comparison with the recent implementations show that the proposed data controller has a higher data bandwidth, is more flexible in its design, and is suitable for use in spaceborne scenarios. Full article

(This article belongs to the Special Issue Hardware Architectures for Real Time Image Processing)

► Show Figures

Figure 1

20 pages, 11932 KiB

Open AccessEditor’s ChoiceArticle

An Efficient FPGA Implementation of Richardson-Lucy Deconvolution Algorithm for Hyperspectral Images

by Karine Avagian and Milica Orlandić

Electronics 2021, 10(4), 504; https://doi.org/10.3390/electronics10040504 - 21 Feb 2021

Cited by 4 | Viewed by 2768

Abstract

This paper proposes an implementation of a Richardson-Lucy (RL) deconvolution method to reduce the spatial degradation in hyperspectral images during the image acquisition process. The degradation, modeled by convolution with a point spread function (PSF), is reduced by applying both standard and accelerated RLdeconvolution algorithms on the individual images in spectral bands. Boundary conditions are introduced to maintain a constant image size without distorting the estimated image boundaries. The RL deconvolution algorithm is implemented on a field-programmable gate array (FPGA)-based Xilinx Zynq-7020 System-on-Chip (SoC). The proposed architecture is parameterized with respect to the image size and configurable with respect to the algorithm variant, the number of iterations, and the kernel size by setting the dedicated configuration registers. A speed-up by factors of 61 and 21 are reported compared to software-only and FPGA-based state-of-the-art implementations, respectively. Full article

(This article belongs to the Special Issue Hardware Architectures for Real Time Image Processing)

► Show Figures

Figure 1

15 pages, 2039 KiB

Open AccessArticle

Reliability Analysis of the SHyLoC CCSDS123 IP Core for Lossless Hyperspectral Image Compression Using COTS FPGAs

by Luis Alberto Aranda, Antonio Sánchez, Francisco Garcia-Herrero, Yubal Barrios, Roberto Sarmiento and Juan Antonio Maestro

Electronics 2020, 9(10), 1681; https://doi.org/10.3390/electronics9101681 - 14 Oct 2020

Cited by 6 | Viewed by 2043

Abstract

Hyperspectral images can comprise hundreds of spectral bands, which means that they can represent a large volume of data difficult to manage with the available on-board resources. Lossless compression solutions are interesting for reducing the amount of information stored or transmitted while preserving it at the same time. The Hyperspectral Lossless Compressor for space applications (SHyLoC), which is part of the European Space Agency (ESA) IP core’s library, has been demonstrated to meet the requirements of space missions in terms of compression efficiency, low complexity and high throughput. Currently, there is a trend to use Commercial Off-The-Shelf (COTS) on-board electronic devices on small satellites. Moreover, commercial Field-Programmable Gate Arrays (FPGAs) have been used in a number of them. Hence, a reliability analysis is required to ensure the robustness of the applications to Single Event Upsets (SEUs) in the configuration memory. In this work, we present a reliability analysis of this hyperspectral image compression module as a first step towards the development of ad-hoc fault-tolerant protection techniques for the SHyLoC IP core. The reliability analysis is performed using a fault-injection-based experimental set-up in which a hardware implementation of the Consultative Committee for Space Data Systems (CCSDS) 123.0-B-1 lossless compression standard is tested against configuration memory errors in a Xilinx Zynq XC7Z020 System-on-Chip. The results obtained for unhardened and redundancy-based protected versions of the module are put into perspective in terms of area/power consumption and availability/protection coverage gained to provide insight into the development of more efficient knowledge-based protection schemes. Full article

(This article belongs to the Special Issue Hardware Architectures for Real Time Image Processing)

► Show Figures

Figure 1

12 pages, 2015 KiB

Open AccessArticle

A Methodology to Analyze the Fault Tolerance of Demosaicking Methods against Memory Single Event Functional Interrupts (SEFIs)

by Luis Alberto Aranda, Alfonso Sánchez-Macián and Juan Antonio Maestro

Electronics 2020, 9(10), 1619; https://doi.org/10.3390/electronics9101619 - 02 Oct 2020

Cited by 2 | Viewed by 1803

Abstract

Electronic circuits in harsh environments, such as space, are affected by soft errors produced by radiation. A single event functional interrupt (SEFI) can affect the behavior of a memory chip, with one or more rows, columns or even the whole device producing a wrong value when reading a set of stored bits. This problem may affect raw Bayer images stored in satellites and other spacecraft. In this paper, we present a methodology to analyze how different interpolation algorithms behave when they try to reconstruct the affected Bayer images into standard red, green and blue (RGB) images. This methodology can be used to compare and develop new fault-tolerant algorithms. The proposed methodology has been illustrated by studying a subset of interpolation algorithms. The results obtained from this example show that the interpolation algorithms that traditionally offer better results in a normal operation (in the absence of errors) are not always the best when SEFI errors are present in the Bayer images. Full article

(This article belongs to the Special Issue Hardware Architectures for Real Time Image Processing)

► Show Figures

Figure 1

23 pages, 982 KiB

Open AccessArticle

Lossy Hyperspectral Image Compression on a Reconfigurable and Fault-Tolerant FPGA-Based Adaptive Computing Platform

by Yubal Barrios, Alfonso Rodríguez, Antonio Sánchez, Arturo Pérez, Sebastián López, Andrés Otero, Eduardo de la Torre and Roberto Sarmiento

Electronics 2020, 9(10), 1576; https://doi.org/10.3390/electronics9101576 - 26 Sep 2020

Cited by 7 | Viewed by 2726

Abstract

This paper describes a novel hardware implementation of a lossy multispectral and hyperspectral image compressor for on-board operation in space missions. The compression algorithm is a lossy extension of the Consultative Committee for Space Data Systems (CCSDS) 123.0-B-1 lossless standard that includes a bit-rate control stage, which in turn manages the losses the compressor may introduce to achieve higher compression ratios without compromising the recovered image quality. The algorithm has been implemented using High-Level Synthesis (HLS) techniques to increase design productivity by raising the abstraction level. The proposed lossy compression solution is deployed onto ARTICo³, a dynamically reconfigurable multi-accelerator architecture, obtaining a run-time adaptive solution that enables user-selectable performance (i.e., load more hardware accelerators to transparently increase throughput), power consumption, and fault tolerance (i.e., group hardware accelerators to transparently enable hardware redundancy). The whole compression solution is tested on a Xilinx Zynq UltraScale+ Field-Programmable Gate Array (FPGA)-based MPSoC using different input images, from multispectral to ultraspectral. For images acquired by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS), the proposed implementation renders an execution time of approximately 36 s when 8 accelerators are compressing concurrently at 100 MHz, which in turn uses around 20% of the LUTs and 17% of the dedicated memory blocks available in the target device. In this scenario, a speedup of 15.6× is obtained in comparison with a pure software version of the algorithm running in an ARM Cortex-A53 processor. Full article

(This article belongs to the Special Issue Hardware Architectures for Real Time Image Processing)

► Show Figures

Figure 1

21 pages, 3602 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

Parallel Classification Pipelines for Skin Cancer Detection Exploiting Hyperspectral Imaging on Hybrid Systems

by Emanuele Torti, Raquel Leon, Marco La Salvia, Giordana Florimbi, Beatriz Martinez-Vega, Himar Fabelo, Samuel Ortega, Gustavo M. Callicó and Francesco Leporati

Electronics 2020, 9(9), 1503; https://doi.org/10.3390/electronics9091503 - 13 Sep 2020

Cited by 19 | Viewed by 4694

Abstract

The early detection of skin cancer is of crucial importance to plan an effective therapy to treat the lesion. In routine medical practice, the diagnosis is based on the visual inspection of the lesion and it relies on the dermatologists’ expertise. After a first examination, the dermatologist may require a biopsy to confirm if the lesion is malignant or not. This methodology suffers from false positives and negatives issues, leading to unnecessary surgical procedures. Hyperspectral imaging is gaining relevance in this medical field since it is a non-invasive and non-ionizing technique, capable of providing higher accuracy than traditional imaging methods. Therefore, the development of an automatic classification system based on hyperspectral images could improve the medical practice to distinguish pigmented skin lesions from malignant, benign, and atypical lesions. Additionally, the system can assist general practitioners in first aid care to prevent noncritical lesions from reaching dermatologists, thereby alleviating the workload of medical specialists. In this paper is presented a parallel pipeline for skin cancer detection that exploits hyperspectral imaging. The computational times of the serial processing have been reduced by adopting multicore and many-core technologies, such as OpenMP and CUDA paradigms. Different parallel approaches have been combined, leading to the development of fifteen classification pipeline versions. Experimental results using in-vivo hyperspectral images show that a hybrid parallel approach is capable of classifying an image of 50 × 50 pixels with 125 bands in less than 1 s. Full article

(This article belongs to the Special Issue Hardware Architectures for Real Time Image Processing)

► Show Figures

Figure 1

20 pages, 5746 KiB

Open AccessArticle

An Efficient FPGA-Based Implementation for Quantized Remote Sensing Image Scene Classification Network

by Xiaoli Zhang, Xin Wei, Qianbo Sang, He Chen and Yizhuang Xie

Electronics 2020, 9(9), 1344; https://doi.org/10.3390/electronics9091344 - 20 Aug 2020

Cited by 13 | Viewed by 2794

Abstract

Deep Convolutional Neural Network (DCNN)-based image scene classification models play an important role in a wide variety of remote sensing applications and achieve great success. However, the large-scale remote sensing images and the intensive computations make the deployment of these DCNN-based models on low-power processing systems (e.g., spaceborne or airborne) a challenging problem. To solve this problem, this paper proposes a high-performance Field-Programmable Gate Array (FPGA)-based DCNN accelerator by combining an efficient network compression scheme and reasonable hardware architecture. Firstly, this paper applies the network quantization to a high-accuracy remote sensing scene classification network, an improved oriented response network (IORN). The volume of the parameters and feature maps in the network is greatly reduced. Secondly, an efficient hardware architecture for network implementation is proposed. The architecture employs dual-channel Double Data Rate Synchronous Dynamic Random-Access Memory (DDR) access mode, rational on-chip data processing scheme and efficient processing engine design. Finally, we implement the quantized IORN (Q-IORN) with the proposed architecture on a Xilinx VC709 development board. The experimental results show that the proposed accelerator has 88.31% top-1 classification accuracy and achieves a throughput of 209.60 Giga-Operations Per Second (GOP/s) with a 6.32 W on-chip power consumption at 200 MHz. The comparison results with off-the-shelf devices and recent state-of-the-art implementations illustrate that the proposed accelerator has obvious advantages in terms of energy efficiency. Full article

(This article belongs to the Special Issue Hardware Architectures for Real Time Image Processing)

► Show Figures

Figure 1

13 pages, 482 KiB

Open AccessArticle

A 13.3 Gbps 9/7M Discrete Wavelet Transform for CCSDS 122.0-B-1 Image Data Compression on a Space-Grade SRAM FPGA

by Elias Machairas and Nektarios Kranitis

Electronics 2020, 9(8), 1234; https://doi.org/10.3390/electronics9081234 - 31 Jul 2020

Cited by 6 | Viewed by 3943

Abstract

Remote sensing is recognized as a cornerstone monitoring technology. The latest high-resolution and high-speed spaceborne imagers provide an explosive growth in data volume and instrument data rates in the range of several Gbps. This competes with the limited on-board storage resources and downlink bandwidth, making image data compression a mission-critical on-board processing task. The Consultative Committee for Space Data Systems (CCSDS) Image Data Compression (IDC) standard CCSDS-122.0-B-1 is a transform-based 2D image compression algorithm designed specifically for use on-board a space platform. In this paper, we introduce a high-performance architecture for a key-part of the CCSDS-IDC algorithm, the 9/7M Integer Discrete Wavelet Transform (DWT). The proposed parallel architecture achieves 2 samples/cycle while the very deep pipeline enables very high clock frequencies. Moreover, it exploits elastic pipeline principles to provide modularity, latency insensitivity and distributed control. The implementation of the proposed architecture on a Xilinx Kintex Ultrascale XQRKU060 space-grade SRAM FPGA achieves state-of-the-art throughput performance of 831 MSamples/s (13.3 Gbps @ 16bpp) allowing seamless integration with next-generation high-speed imagers and on-board data handling networking technology. To the best of our knowledge, this is the fastest implementation of the 9/7M Integer DWT on a space-grade FPGA, outperforming previous implementations. Full article

(This article belongs to the Special Issue Hardware Architectures for Real Time Image Processing)

► Show Figures

Figure 1

Review

Jump to: Research

34 pages, 2313 KiB

Open AccessReview

Hardware Architectures for Real-Time Medical Imaging

by Eduardo Alcaín, Pedro R. Fernández, Rubén Nieto, Antonio S. Montemayor, Jaime Vilas, Adrian Galiana-Bordera, Pedro Miguel Martinez-Girones, Carmen Prieto-de-la-Lastra, Borja Rodriguez-Vila, Marina Bonet, Cristina Rodriguez-Sanchez, Imene Yahyaoui, Norberto Malpica, Susana Borromeo, Felipe Machado and Angel Torrado-Carvajal

Electronics 2021, 10(24), 3118; https://doi.org/10.3390/electronics10243118 - 15 Dec 2021

Cited by 13 | Viewed by 7193

Abstract

Medical imaging is considered one of the most important advances in the history of medicine and has become an essential part of the diagnosis and treatment of patients. Earlier prediction and treatment have been driving the acquisition of higher image resolutions as well as the fusion of different modalities, raising the need for sophisticated hardware and software systems for medical image registration, storage, analysis, and processing. In this scenario and given the new clinical pipelines and the huge clinical burden of hospitals, these systems are often required to provide both highly accurate and real-time processing of large amounts of imaging data. Additionally, lowering the prices of each part of imaging equipment, as well as its development and implementation, and increasing their lifespan is crucial to minimize the cost and lead to more accessible healthcare. This paper focuses on the evolution and the application of different hardware architectures (namely, CPU, GPU, DSP, FPGA, and ASIC) in medical imaging through various specific examples and discussing different options depending on the specific application. The main purpose is to provide a general introduction to hardware acceleration techniques for medical imaging researchers and developers who need to accelerate their implementations. Full article

(This article belongs to the Special Issue Hardware Architectures for Real Time Image Processing)

► Show Figures