Masked Language Modeling, Next Sentence Prediction

from transformers import pipeline
unmasker = pipeline('fill-mask', model='bert-base-uncased')
unmasker(Hello I'm a [MASK] model.)

BERT base model (uncased) is a transformer model pretrained on a large corpus of English data using a masked language modeling (MLM) objective. It can be used for masked language modeling, next sentence prediction, and fine-tuning on downstream tasks such as sequence classification, token classification, or question answering.


from transformers import pipeline
image_classifier = pipeline('image-classification', 'martinezomg/vit-base-patch16-224-diabetic-retinopathy')
result = image_classifier('path/to/image.jpg')

This model is a fine-tuned version of google/vit-base-patch16-224 on the None dataset. It is designed for image classification tasks, specifically for diabetic retinopathy detection.


CLIPSeg model with reduce dimension 64, refined (using a more complex convolution). It was introduced in the paper Image Segmentation Using Text and Image Prompts by Lüddecke et al. and first released in this repository. This model is intended for zero-shot and one-shot image segmentation.


COCO
Conceptual Captions (CC3M)
SBU
Visual Genome (VG)
Conceptual Captions (CC12M)
ALT200M

Refer to the paper for evaluation results

GIT (short for GenerativeImage2Text) model, base-sized version. It was introduced in the paper GIT: A Generative Image-to-text Transformer for Vision and Language by Wang et al. and first released in this repository. The model is trained using 'teacher forcing' on a lot of (image, text) pairs. The goal for the model is simply to predict the next text token, giving the image tokens and previous text tokens. This allows the model to be used for tasks like image and video captioning, visual question answering (VQA) on images and videos, and even image classification (by simply conditioning the model on the image and asking it to generate a class for it in text).


Natural Language Processing Token Classification

nlp('My name is John and I live in New York.')

A pretrained BERT model for Named Entity Recognition (NER) on uncased text. It can be used to extract entities such as person names, locations, and organizations from text.


Table Transformer (DETR) model trained on PubTables1M for detecting the structure (like rows, columns) in tables.


nlp(https://templates.invoicehome.com/invoice-template-us-neat-750px.png, What is the invoice number?)

A fine-tuned version of the multi-modal LayoutLM model for the task of question answering on documents.


Natural Language Processing Summarization

from transformers import pipeline
summarizer = pipeline(summarization, model=facebook/bart-large-cnn)
ARTICLE = ...
print(summarizer(ARTICLE, max_length=130, min_length=30, do_sample=False))

BART (large-sized model), fine-tuned on CNN Daily Mail. BART is a transformer encoder-encoder (seq2seq) model with a bidirectional (BERT-like) encoder and an autoregressive (GPT-like) decoder. BART is pre-trained by (1) corrupting text with an arbitrary noising function, and (2) learning a model to reconstruct the original text. BART is particularly effective when fine-tuned for text generation (e.g. summarization, translation) but also works well for comprehension tasks (e.g. text classification, question answering). This particular checkpoint has been fine-tuned on CNN Daily Mail, a large collection of text-summary pairs.


Natural Language Processing Table Question Answering

from transformers import pipeline

qa_pipeline = pipeline('table-question-answering', model='google/tapas-large-finetuned-wtq')

result = qa_pipeline(table=table, query=query)

TAPAS large model fine-tuned on WikiTable Questions (WTQ). This model was pre-trained on MLM and an additional step which the authors call intermediate pre-training, and then fine-tuned in a chain on SQA, WikiSQL and finally WTQ. It uses relative position embeddings (i.e. resetting the position index at every cell of the table).


Natural Language Processing Text2Text Generation

from transformers import pipeline

t5 = pipeline('text2text-generation', model='google/t5-v1_1-base')
t5('translate English to French: Hugging Face is a great company')

Google's T5 Version 1.1 is a state-of-the-art text-to-text transformer model that achieves high performance on various NLP tasks such as summarization, question answering, and text classification. It is pre-trained on the Colossal Clean Crawled Corpus (C4) and fine-tuned on downstream tasks.


A finetuned xlm-roberta-base model for punctuation prediction on twelve languages: English, German, French, Spanish, Bulgarian, Italian, Polish, Dutch, Czech, Portugese, Slovak, Slovenian.


vqa(image='path/to/image.jpg', question='What is in the image?')

GIT (short for GenerativeImage2Text) model, base-sized version, fine-tuned on VQAv2. It was introduced in the paper GIT: A Generative Image-to-text Transformer for Vision and Language by Wang et al. and first released in this repository.