Nino Risteski 7d4e0c23c8 Update (#26198) 5 days ago
benchmarking 5e8c8eb5ba Apply ruff flake8-comprehensions (#21694) 7 months ago
contrastive-image-text d8e13b3e04 2 weeks ago
image-classification d8e13b3e04 2 weeks ago
language-modeling 5af2c62696 docs: add space to docs (#26067) 1 week ago
language-modeling-tpu 167a0d8f87 Add an option to reduce compile() console spam (#23938) 3 months ago
multiple-choice d8e13b3e04 2 weeks ago
question-answering d8e13b3e04 2 weeks ago
summarization d8e13b3e04 2 weeks ago
text-classification d8e13b3e04 2 weeks ago
token-classification 9c7b744795 Fix missing usage of `token` (#25382) 1 month ago
translation d8e13b3e04 2 weeks ago 7d4e0c23c8 Update (#26198) 5 days ago
_tests_requirements.txt 62396cff46 TF 2.14 compatibility (#25630) 1 month ago c34a525d2f Proposed fix for TF example now running on safetensors. (#23208) 4 months ago


This folder contains actively maintained examples of the use of 🤗 Transformers organized into different ML tasks. All examples in this folder are TensorFlow examples and are written using native Keras rather than classes like TFTrainer, which we now consider deprecated. If you've previously only used 🤗 Transformers via TFTrainer, we highly recommend taking a look at the new style - we think it's a big improvement!

In addition, all scripts here now support the 🤗 Datasets library - you can grab entire datasets just by changing one command-line argument!

A note on code folding

Most of these examples have been formatted with #region blocks. In IDEs such as PyCharm and VSCode, these blocks mark named regions of code that can be folded for easier viewing. If you find any of these scripts overwhelming or difficult to follow, we highly recommend beginning with all regions folded and then examining regions one at a time!

The Big Table of Tasks

Here is the list of all our examples:

Task Example datasets
language-modeling WikiText-2
multiple-choice SWAG
question-answering SQuAD
summarization XSum
text-classification GLUE
token-classification CoNLL NER
translation WMT

Coming soon

  • Colab notebooks to easily run through these scripts!