arXiv Open Access 2023

Automated Heterogeneous Low-Bit Quantization of Multi-Model Deep Learning Inference Pipeline

Jayeeta Mondal Swarnava Dey Arijit Mukherjee

Lihat Sumber

Abstrak

Multiple Deep Neural Networks (DNNs) integrated into single Deep Learning (DL) inference pipelines e.g. Multi-Task Learning (MTL) or Ensemble Learning (EL), etc., albeit very accurate, pose challenges for edge deployment. In these systems, models vary in their quantization tolerance and resource demands, requiring meticulous tuning for accuracy-latency balance. This paper introduces an automated heterogeneous quantization approach for DL inference pipelines with multiple DNNs.

Topik & Kata Kunci

cs.CV math.OC

Penulis (3)

Jayeeta Mondal

Swarnava Dey

Arijit Mukherjee

Format Sitasi

APA MLA BibTeX

Mondal, J., Dey, S., Mukherjee, A. (2023). Automated Heterogeneous Low-Bit Quantization of Multi-Model Deep Learning Inference Pipeline. https://arxiv.org/abs/2311.05870

Akses Cepat

Lihat di Sumber

Informasi Jurnal

Tahun Terbit: 2023
Bahasa: en
Sumber Database: arXiv
Akses: Open Access ✓