Figure 1 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Download
View publication
Neural LMs with implicit cache pointers.

Neural LMs with implicit cache pointers.

Source publication
Figure 1: Neural LMs with implicit cache pointers.
Details of neural network dimensions for various LMs.
Perplexities on PTB and WikiText-2 (plain LSTMs).
Perplexities on PTB and WikiText-2 (Frage-AWD- LSTM setup).
Perplexities on SWBD.
Neural Language Modeling With Implicit Cache Pointers
Preprint
Full-text available
  • Sep 2020
A cache-inspired approach is proposed for neural language models (LMs) to improve long-range dependency and better predict rare words from long contexts. This approach is a simpler alternative to attention-based pointer mechanism that enables neural LMs to reproduce words from recent history. Without using attention and mixture structure, the metho...
Cite
Download full-text

Similar publications

Figure 1: Augmented vs Un-augmented model performance. The x-axis...
Figure 2: Model performance based on the nature of representation of...
Figure 3: Fine-tuning vs training only regression layer in the models....
Performance of 5 best Un-Augmented models.
+1 Performance of 5 best Sum models
Team ÚFAL at CMCL 2022 Shared Task: Figuring out the correct recipe for predicting Eye-Tracking features using Pretrained Language Models
Conference Paper
Full-text available
  • Apr 2022
Eye-Tracking data is a very useful source of information to study cognition and especially language comprehension in humans. In this paper, we describe our systems for the CMCL 2022 shared task on predicting eye-tracking information. We describe our experiments with pretrained models like BERT and XLM and the different ways in which we used those r...
View
Figure 1: Augmented vs Un-augmented model performance. The x-axis...
Figure 2: Model performance based on the nature of representation of...
Figure 3: Fine-tuning vs training only regression layer in the models....
Performance of 5 best Un-Augmented models.
+1 Performance of 5 best Sum models
Team \'UFAL at CMCL 2022 Shared Task: Figuring out the correct recipe for predicting Eye-Tracking features using Pretrained Language Models
Preprint
Full-text available
  • Apr 2022
Eye-Tracking data is a very useful source of information to study cognition and especially language comprehension in humans. In this paper, we describe our systems for the CMCL 2022 shared task on predicting eye-tracking information. We describe our experiments with pretrained models like BERT and XLM and the different ways in which we used those r...
View