Previous works on the Recurrent Neural Network-Transducer (RNN-T) models have shown that, under some conditions, it is possible to simplify its prediction network with little or no loss in recognition accuracy (arXiv:2003.07705 [eess.AS], [2], arXiv:2012.06749 [cs.CL]). This is done by limiting the context size of previous labels and/or using a simpler architecture for its layers instead of LSTMs. The benefits of such changes include reduction in model size, faster inference and power savings, which are all useful for on-device applications. In this work, we study ways to make the RNN-T decoder (prediction network + joint network) smaller and faster without degradation in recognition performance. Our prediction network performs a simple weighted averaging of the input embeddings, and shares its embedding matrix weights with the joint network's output layer (a.k.a. weight tying, commonly used in language modeling arXiv:1611.01462 [cs.LG]). This simple design, when used in conjunction with additional Edit-based Minimum Bayes Risk (EMBR) training, reduces the RNN-T Decoder from 23M parameters to just 2M, without affecting word-error rate (WER).
Sie haben es wieder getan: Nachdem arXiv erst im April um Applied Physics erweitert wurde, sind jetzt mit Electrical Engineering and Systems Science EESS und Economics Econ gleich zwei weitere Fachgebiete hinzugekommen.
Sie haben es wieder getan: Nachdem arXiv erst im April um Applied Physics erweitert wurde, sind jetzt mit Electrical Engineering and Systems Science EESS und Economics Econ gleich zwei weitere Fachgebiete hinzugekommen.
USDOE Office of Science (SC), Biological and Environmental Research (BER). Earth & Environmental Systems Science (EESS), Yang Wang, University of Miami Coral Gables
et al.
USDOE Office of Science (SC), Biological and Environmental Research (BER). Earth & Environmental Systems Science (EESS), Wei Cheng, UNIVERSITY OF WASHINGTON
et al.
USDOE Office of Science (SC), Biological and Environmental Research (BER). Earth & Environmental Systems Science (EESS), William Gardner, University of Montana
USDOE Office of Science (SC), Biological and Environmental Research (BER). Earth & Environmental Systems Science (EESS), Jefferson Moore, Regents of the University of California, Irvine
USDOE Office of Science (SC), Biological and Environmental Research (BER). Earth & Environmental Systems Science (EESS), Matthew Shupe, Univ. of Colorado, Boulder, CO (United States)
et al.
USDOE Office of Science (SC), Biological and Environmental Research (BER). Earth & Environmental Systems Science (EESS), James Smith, Univ. of California, Irvine, CA (United States)
et al.
The article proposes an original approach to building tools for knowledge testing using production expert systems. The presence of an inference mechanism and an apparatus for working with inaccurately presented information ensures the non-deterministic nature of the question-answer system. The ability to apply the approach in various fields is provided by the expert system shell. Knowledge testing tasks require the extension of traditional ideas about production expert systems. The article formulates requirements for such systems and defines a knowledge model of EESS (Extended Expert System Shell). The development ideas were mainly born in application to testing students' knowledge, but the proposed capabilities will also be useful in other areas, for example, in medical diagnos-tics – a traditional subject area of expert systems.
USDOE Office of Science (SC), Biological and Environmental Research (BER). Earth & Environmental Systems Science (EESS), Ian Williams, IOWA STATE UNIVERSITY OF SCIENCE AND TECHNOLOGY
et al.
USDOE Office of Science (SC), Biological and Environmental Research (BER). Earth & Environmental Systems Science (EESS), Umberto Ciri, Univ. of Puerto Rico, Mayaguez, PR (United States)
USDOE Office of Science (SC), Biological and Environmental Research (BER). Earth & Environmental Systems Science (EESS), Paul DeMott, Colorado State University
et al.
USDOE Office of Science (SC), Biological and Environmental Research (BER). Earth & Environmental Systems Science (EESS), Jeffrey Wilcox, University of North Carolina, Asheville, NC (United States)
USDOE Office of Science (SC), Biological and Environmental Research (BER). Earth & Environmental Systems Science (EESS), Peter Blossey, Univ. of Washington, Seattle, WA (United States)
Murray State University, Bassil El Masri, USDOE Office of Science (SC), Biological and Environmental Research (BER). Earth & Environmental Systems Science (EESS)
et al.
USDOE Office of Science (SC), Biological and Environmental Research (BER). Earth & Environmental Systems Science (EESS), Paul DeMott, Colorado State Univ., Fort Collins, CO (United States)
et al.