Fig 2 - uploaded by Adam Frémund
Content may be subject to copyright.
![Example of processing the original text and creating input/output text pairs. Figure taken from [10].](profile/Adam-Fremund/publication/363600075/figure/fig1/AS:11431281095712510@1667989218040/Example-of-processing-the-original-text-and-creating-input-output-text-pairs-Figure.png)
Example of processing the original text and creating input/output text pairs. Figure taken from [10].
Source publication
Pre-trained models used in the transfer-learning scenario are recently becoming very popular. Such models benefit from the availability of large sets of unlabeled data. Two kinds of such models include the Wav2Vec 2.0 speech recognizer and T5 text-to-text transformer. In this paper, we describe a novel application of such models for dialog systems,...
Context in source publication
Context 1
... to construct the internal representation of input on many linguistic layers: starting from phonetic and syntactic through semantic to the pragmatic layer. The T5 model is pre-trained in a self-supervised manner by generating a text restoration task from unlabelled training data. An example of the pre-training input/output text pair is shown in Fig. ...
Similar publications
Conformer-based models have become the most dominant end- to-end architecture for speech processing tasks. In this work, we propose a carefully redesigned Conformer with a new down- sampling schema. The proposed model, named Fast Con- former, is 2.8x faster than original Conformer, while preserv- ing state-of-the-art accuracy on Automatic Speech Re...
