Rare Tokens Degenerate All Tokens: Improving Neural Text Generation via Adaptive Gradient Gating for Rare Token Embeddings We find that the proposed method facilitates insights into causes of variation between reproductions, and as a result, allows conclusions to be drawn about what aspects of system and/or evaluation design need to be changed in order to improve reproducibility. The proposed QRA method produces degree-of-reproducibility scores that are comparable across multiple reproductions not only of the same, but also of different, original studies. We test QRA on 18 different system and evaluation measure combinations (involving diverse NLP tasks and types of evaluation), for each of which we have the original results and one to seven reproduction results.

QRA produces a single score estimating the degree of reproducibility of a given system and evaluation measure, on the basis of the scores from, and differences between, different reproductions. This paper describes and tests a method for carrying out quantified reproducibility assessment (QRA) that is based on concepts and definitions from metrology. Quantified Reproducibility Assessment of NLP Results In comparison to other widely used strategies for selecting important tokens, such as saliency and attention, our proposed method has a significantly lower false positive rate in generating rationales. We also validate the quality of the selected tokens in our method using human annotations in the ERASER benchmark. Our experiments on several diverse classification tasks show speedups up to 22x during inference time without much sacrifice in performance. To determine the importance of each token representation, we train a Contribution Predictor for each layer using a gradient-based saliency method. Our method dynamically eliminates less contributing tokens through layers, resulting in shorter lengths and consequently lower computational cost. In this work, we propose a novel approach for reducing the computational cost of BERT with minimal loss in downstream performance.

But, this usually comes at the cost of high latency and computation, hindering their usage in resource-limited settings. Pre-trained language models have shown stellar performance in various downstream tasks.

Jing du texas a n m