from transformers import VideoMAEImageProcessor, VideoMAEForVideoClassification
import numpy as np
import torch
video = list(np.random.randn(16, 3, 224, 224))
processor = VideoMAEImageProcessor.from_pretrained('MCG-NJU/videomae-small-finetuned-kinetics')
model = VideoMAEForVideoClassification.from_pretrained('MCG-NJU/videomae-small-finetuned-kinetics')
inputs = processor(video, return_tensors='pt')
with torch.no_grad():
 outputs = model(**inputs)
 logits = outputs.logits
predicted_class_idx = logits.argmax(-1).item()
print('Predicted class:', model.config.id2label[predicted_class_idx])


This model is a fine-tuned version of MCG-NJU/videomae-large-finetuned-kinetics on an unknown dataset.


A tiny random VideoMAE model for video classification.


transformers==4.25.1, torch==1.10.0, datasets==2.7.1, tokenizers==0.12.1

This model is a fine-tuned version of MCG-NJU/videomae-base on an unknown dataset. It is used for video classification tasks.


from decord import VideoReader, cpu
import torch
import numpy as np
from transformers import VideoMAEFeatureExtractor, VideoMAEForVideoClassification
from huggingface_hub import hf_hub_download
np.random.seed(0)
def sample_frame_indices(clip_len, frame_sample_rate, seg_len):
 converted_len = int(clip_len * frame_sample_rate)
 end_idx = np.random.randint(converted_len, seg_len)
 start_idx = end_idx - converted_len
 indices = np.linspace(start_idx, end_idx, num=clip_len)
 indices = np.clip(indices, start_idx, end_idx - 1).astype(np.int64)
 return indices
file_path = hf_hub_download(
 repo_id=nateraw/dino-clips, filename=archery.mp4, repo_type=space
)
videoreader = VideoReader(file_path, num_threads=1, ctx=cpu(0))
videoreader.seek(0)
indices = sample_frame_indices(clip_len=16, frame_sample_rate=4, seg_len=len(videoreader))
video = videoreader.get_batch(indices).asnumpy()
feature_extractor = VideoMAEFeatureExtractor.from_pretrained(nateraw/videomae-base-finetuned-ucf101)
model = VideoMAEForVideoClassification.from_pretrained(nateraw/videomae-base-finetuned-ucf101)
inputs = feature_extractor(list(video), return_tensors=pt)
with torch.no_grad():
 outputs = model(**inputs)
 logits = outputs.logits
predicted_label = logits.argmax(-1).item()
print(model.config.id2label[predicted_label])

VideoMAE Base model fine tuned on UCF101 for Video Action Recognition


Adam with betas=(0.9,0.999) and epsilon=1e-08

from transformers import AutoModelForVideoClassification, AutoTokenizer
model = AutoModelForVideoClassification.from_pretrained('sayakpaul/videomae-base-finetuned-ucf101-subset')
tokenizer = AutoTokenizer.from_pretrained('sayakpaul/videomae-base-finetuned-ucf101-subset')

This model is a fine-tuned version of MCG-NJU/videomae-base on an unknown dataset. It achieves the following results on the evaluation set: Loss: 0.3992, Accuracy: 0.8645.


Computer Vision Zero-Shot Image Classification

Transformers 4.21.3, TensorFlow 2.8.2, Tokenizers 0.12.1

This model was trained from scratch on an unknown dataset.


Natural Language Processing Zero-Shot Classification

from transformers import pipeline; classifier = pipeline('zero-shot-classification', model='laion/CLIP-ViT-B-32-laion2B-s34B-b79K'); classifier(image='path/to/image', class_names=['class1', 'class2', 'class3'])

A CLIP ViT-B/32 model trained with the LAION-2B English subset of LAION-5B using OpenCLIP. It enables researchers to better understand and explore zero-shot, arbitrary image classification. The model can be used for zero-shot image classification, image and text retrieval, among others.


from PIL import Image
import requests
from transformers import CLIPProcessor, CLIPModel
model = CLIPModel.from_pretrained(openai/clip-vit-base-patch32)
processor = CLIPProcessor.from_pretrained(openai/clip-vit-base-patch32)
url = http://images.cocodataset.org/val2017/000000039769.jpg
image = Image.open(requests.get(url, stream=True).raw)
inputs = processor(text=[a photo of a cat, a photo of a dog], images=image, return_tensors=pt, padding=True)
outputs = model(**inputs)
logits_per_image = outputs.logits_per_image
probs = logits_per_image.softmax(dim=1)

The CLIP model was developed by researchers at OpenAI to learn about what contributes to robustness in computer vision tasks. The model was also developed to test the ability of models to generalize to arbitrary image classification tasks in a zero-shot manner.


a photo of a cat
a photo of a dog

from PIL import Image
import requests
from transformers import CLIPProcessor, CLIPModel
model = CLIPModel.from_pretrained(openai/clip-vit-large-patch14)
processor = CLIPProcessor.from_pretrained(openai/clip-vit-large-patch14)
url = http://images.cocodataset.org/val2017/000000039769.jpg
image = Image.open(requests.get(url, stream=True).raw)
inputs = processor(text=[a photo of a cat, a photo of a dog], images=image, return_tensors=pt, padding=True)
outputs = model(**inputs)
logits_per_image = outputs.logits_per_image
probs = logits_per_image.softmax(dim=1)


A CLIP ViT L/14 model trained with the LAION-2B English subset of LAION-5B using OpenCLIP. Intended for research purposes and exploring zero-shot, arbitrary image classification. Can be used for interdisciplinary studies of the potential impact of such model.


A zero-shot image classification model based on OpenCLIP, which can classify images into various categories without requiring any training data for those categories.


from transformers import pipeline; classifier = pipeline('image-classification', model='laion/CLIP-ViT-bigG-14-laion2B-39B-b160k'); classifier(image, possible_class_names=['cat', 'dog'])

A CLIP ViT-bigG/14 model trained with the LAION-2B English subset of LAION-5B using OpenCLIP. The model is intended for research purposes and enables researchers to better understand and explore zero-shot, arbitrary image classification. It can be used for interdisciplinary studies of the potential impact of such models. The model achieves a 80.1 zero-shot top-1 accuracy on ImageNet-1k.


from PIL import Image
import requests
from transformers import CLIPProcessor, CLIPModel
model = CLIPModel.from_pretrained(openai/clip-vit-base-patch16)
processor = CLIPProcessor.from_pretrained(openai/clip-vit-base-patch16)
url = http://images.cocodataset.org/val2017/000000039769.jpg
image = Image.open(requests.get(url, stream=True).raw)
inputs = processor(text=[a photo of a cat, a photo of a dog], images=image, return_tensors=pt, padding=True)
outputs = model(**inputs)
logits_per_image = outputs.logits_per_image
probs = logits_per_image.softmax(dim=1)


List of possible class names (comma-separated)

from transformers import pipeline; classify = pipeline('image-classification', model='laion/CLIP-ViT-B-16-laion2B-s34B-b88K'); classify('/path/to/image.jpg', ['cat', 'dog'])

A CLIP ViT-B/16 model trained with the LAION-2B English subset of LAION-5B using OpenCLIP. This model is intended for research purposes and can be used for zero-shot image classification, image and text retrieval, and other related tasks.


from transformers import CLIPProcessor, CLIPModel; model = CLIPModel.from_pretrained('patrickjohncyh/fashion-clip'); processor = CLIPProcessor.from_pretrained('patrickjohncyh/fashion-clip'); inputs = processor(text='blue shoes', images=image, return_tensors='pt', padding=True); logits_per_image = model(**inputs).logits_per_image; probs = logits_per_image.softmax(dim=-1).tolist()[0]

FashionCLIP is a CLIP-based model developed to produce general product representations for fashion concepts. Leveraging the pre-trained checkpoint (ViT-B/32) released by OpenAI, it is trained on a large, high-quality novel fashion dataset to study whether domain specific fine-tuning of CLIP-like models is sufficient to produce product representations that are zero-shot transferable to entirely new datasets and tasks.


results = model(image_path, class_names='cat, dog, bird')

A series of CLIP ConvNeXt-Large (w/ extra text depth, vision MLP head) models trained on the LAION-2B (english) subset of LAION-5B using OpenCLIP. The models utilize the timm ConvNeXt-Large model (convnext_large) as the image tower, a MLP (fc - gelu - drop - fc) head in vision tower instead of the single projection of other CLIP models, and a text tower with same width but 4 layers more depth than ViT-L / RN50x16 models (depth 16, embed dim 768).


list of possible class names (comma-separated)

from transformers import pipeline; model = pipeline('image-classification', model='laion/CLIP-convnext_base_w-laion_aesthetic-s13B-b82K'); model('path/to/image.jpg', ['cat', 'dog'])

A series of CLIP ConvNeXt-Base (w/ wide embed dim) models trained on subsets LAION-5B using OpenCLIP. These models achieve between 70.8 and 71.7 zero-shot top-1 accuracy on ImageNet-1k. They can be used for zero-shot image classification, image and text retrieval, and other tasks.


results = classifier(image, class_names='cat, dog')

A series of CLIP ConvNeXt-XXLarge models trained on LAION-2B (English), a subset of LAION-5B, using OpenCLIP. These models achieve between 79.1 and 79.4 top-1 zero-shot accuracy on ImageNet-1k.


from transformers import pipeline; clip = pipeline('image-classification', model='laion/CLIP-convnext_base_w-laion2B-s13B-b82K'); clip('path/to/image.jpg', ['cat', 'dog'])

A series of CLIP ConvNeXt-Base (w/ wide embed dim) models trained on subsets LAION-5B using OpenCLIP. The models achieve between 70.8 and 71.7 zero-shot top-1 accuracy on ImageNet-1k. The models can be used for zero-shot image classification, image and text retrieval, and other related tasks.


Multimodal Zero-Shot Image Classification

from transformers import pipeline
clip = pipeline('zero-shot-image-classification', model='microsoft/BiomedCLIP-PubMedBERT_256-vit_base_patch16_224')
image = 'path/to/image.png'
possible_class_names = ['class1', 'class2', 'class3']
result = clip(image, possible_class_names)

BiomedCLIP is a biomedical vision-language foundation model pretrained on PMC-15M, a dataset of 15 million figure-caption pairs extracted from biomedical research articles in PubMed Central, using contrastive learning. It uses PubMedBERT as the text encoder and Vision Transformer as the image encoder, with domain-specific adaptations. It can perform various vision-language processing (VLP) tasks such as cross-modal retrieval, image classification, and visual question answering.


Comma-separated list of possible class names

from transformers import pipeline
image_classification = pipeline('image-classification', model='laion/CLIP-convnext_base_w_320-laion_aesthetic-s13B-b82K-augreg')
image_path = 'path/to/image.jpg'
class_names = 'dog, cat'
result = image_classification(image_path, class_names)
print(result)

A series of CLIP ConvNeXt-Base (w/ wide embed dim) models trained on subsets LAION-5B using OpenCLIP. The models utilize the timm ConvNeXt-Base model (convnext_base) as the image tower, and the same text tower as the RN50x4 (depth 12, embed dim 640) model from OpenAI CLIP.


a photo of a residential area
a photo of a playground
a photo of a stadium
a photo of a forest
a photo of an airport

from PIL import Image
import requests
from transformers import CLIPProcessor, CLIPModel
model = CLIPModel.from_pretrained(flax-community/clip-rsicd-v2)
processor = CLIPProcessor.from_pretrained(flax-community/clip-rsicd-v2)
url = https://raw.githubusercontent.com/arampacha/CLIP-rsicd/master/data/stadium_1.jpg
image = Image.open(requests.get(url, stream=True).raw)
labels = [residential area, playground, stadium, forest, airport]
inputs = processor(text=[fa photo of a {l} for l in labels], images=image, return_tensors=pt, padding=True)
outputs = model(**inputs)
logits_per_image = outputs.logits_per_image
probs = logits_per_image.softmax(dim=1)
for l, p in zip(labels, probs[0]):
 print(f{l:<16} {p:.4f})

This model is a fine-tuned CLIP by OpenAI. It is designed with an aim to improve zero-shot image classification, text-to-image and image-to-image retrieval specifically on remote sensing images.


import requests
import torch
from PIL import Image
from transformers import AlignProcessor, AlignModel
processor = AlignProcessor.from_pretrained(kakaobrain/align-base)
model = AlignModel.from_pretrained(kakaobrain/align-base)
url = http://images.cocodataset.org/val2017/000000039769.jpg
image = Image.open(requests.get(url, stream=True).raw)
candidate_labels = [an image of a cat, an image of a dog]
inputs = processor(text=candidate_labels, images=image, return_tensors=pt)
with torch.no_grad():
 outputs = model(**inputs)
logits_per_image = outputs.logits_per_image
probs = logits_per_image.softmax(dim=1)
print(probs)

on-par or outperforms Google ALIGN's reported metrics

The ALIGN model is a dual-encoder architecture with EfficientNet as its vision encoder and BERT as its text encoder. It learns to align visual and text representations with contrastive learning. This implementation is trained on the open source COYO dataset and can be used for zero-shot image classification and multi-modal embedding retrieval.


A tiny random CLIPSegModel for zero-shot image classification.


This model is a zero-shot image classification model based on OpenCLIP. It can be used for classifying images into various categories without any additional training.


from transformers import pipeline; clip = pipeline('zero-shot-classification', model='laion/CLIP-convnext_xxlarge-laion2B-s34B-b82K-augreg-rewind'); clip(image, class_names=['cat', 'dog', 'fish'])

A series of CLIP ConvNeXt-XXLarge models trained on LAION-2B (English), a subset of LAION-5B, using OpenCLIP. These models achieve between 79.1 and 79.4 top-1 zero-shot accuracy on ImageNet-1k. The models can be used for zero-shot image classification, image and text retrieval, and other related tasks.


from transformers import pipeline
clip = pipeline('image-classification', model='laion/CLIP-convnext_large_d.laion2B-s26B-b102K-augreg')
clip('./path/to/image.jpg', 'class1,class2,class3')

A series of CLIP ConvNeXt-Large (w/ extra text depth, vision MLP head) models trained on LAION-2B (english), a subset of LAION-5B, using OpenCLIP. The models are trained at 256x256 image resolution and achieve a 75.9 top-1 zero-shot accuracy on ImageNet-1k.


from transformers import pipeline; classifier = pipeline('image-classification', model='laion/CLIP-convnext_large_d_320.laion2B-s29B-b131K-ft'); classifier('path/to/image.jpg', ['class1', 'class2'])

A series of CLIP ConvNeXt-Large models trained on the LAION-2B (english) subset of LAION-5B using OpenCLIP. The models achieve between 75.9 and 76.9 top-1 zero-shot accuracy on ImageNet-1k.


from PIL import Image
import requests
from transformers import ChineseCLIPProcessor, ChineseCLIPModel
model = ChineseCLIPModel.from_pretrained(OFA-Sys/chinese-clip-vit-base-patch16)
processor = ChineseCLIPProcessor.from_pretrained(OFA-Sys/chinese-clip-vit-base-patch16)
url = https://clip-cn-beijing.oss-cn-beijing.aliyuncs.com/pokemon.jpeg
image = Image.open(requests.get(url, stream=True).raw)
texts = [, , , ]
inputs = processor(images=image, return_tensors=pt)
image_features = model.get_image_features(**inputs)
image_features = image_features / image_features.norm(p=2, dim=-1, keepdim=True)
inputs = processor(text=texts, padding=True, return_tensors=pt)
text_features = model.get_text_features(**inputs)
text_features = text_features / text_features.norm(p=2, dim=-1, keepdim=True)
inputs = processor(text=texts, images=image, return_tensors=pt, padding=True)
outputs = model(**inputs)
logits_per_image = outputs.logits_per_image
probs = logits_per_image.softmax(dim=1)

Chinese CLIP is a simple implementation of CLIP on a large-scale dataset of around 200 million Chinese image-text pairs. It uses ViT-B/16 as the image encoder and RoBERTa-wwm-base as the text encoder.


from transformers import pipeline
repo = 'Bingsu/clip-vit-base-patch32-ko'
pipe = pipeline('zero-shot-image-classification', model=repo)
url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
result = pipe(images=url, candidate_labels=[], hypothesis_template='{}')
result

Korean CLIP model trained by Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation. It is a zero-shot image classification model that can be used to classify images without any training data.


texts

from PIL import Image
import requests
from transformers import ChineseCLIPProcessor, ChineseCLIPModel
model = ChineseCLIPModel.from_pretrained(OFA-Sys/chinese-clip-vit-large-patch14-336px)
processor = ChineseCLIPProcessor.from_pretrained(OFA-Sys/chinese-clip-vit-large-patch14-336px)
url = https://clip-cn-beijing.oss-cn-beijing.aliyuncs.com/pokemon.jpeg
image = Image.open(requests.get(url, stream=True).raw)
texts = []
inputs = processor(images=image, return_tensors=pt)
image_features = model.get_image_features(**inputs)
image_features = image_features / image_features.norm(p=2, dim=-1, keepdim=True)
inputs = processor(text=texts, padding=True, return_tensors=pt)
text_features = model.get_text_features(**inputs)
text_features = text_features / text_features.norm(p=2, dim=-1, keepdim=True)
inputs = processor(text=texts, images=image, return_tensors=pt, padding=True)
outputs = model(**inputs)
logits_per_image = outputs.logits_per_image
probs = logits_per_image.softmax(dim=1)

Chinese CLIP is a simple implementation of CLIP on a large-scale dataset of around 200 million Chinese image-text pairs. It uses ViT-L/14@336px as the image encoder and RoBERTa-wwm-base as the text encoder.


Natural Language Processing Text Classification

import torch
from transformers import DistilBertTokenizer, DistilBertForSequenceClassification
tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')
model = DistilBertForSequenceClassification.from_pretrained('distilbert-base-uncased-finetuned-sst-2-english')
inputs = tokenizer('Hello, my dog is cute', return_tensors='pt')
with torch.no_grad():
    logits = model(**inputs).logits
predicted_class_id = logits.argmax().item()
model.config.id2label[predicted_class_id]

This model is a fine-tune checkpoint of DistilBERT-base-uncased, fine-tuned on SST-2. It reaches an accuracy of 91.3 on the dev set (for comparison, Bert bert-base-uncased version reaches an accuracy of 92.7). This model can be used for topic classification.


sentiment_analysis('I love this product!')

This is a fine-tuned downstream version of the bert-base-uncased model for sentiment analysis, this model is not intended for further downstream fine-tuning for any other tasks. This model is trained on a classified dataset for text classification.


from transformers import AutoModelForSequenceClassification
from transformers import TFAutoModelForSequenceClassification
from transformers import AutoTokenizer
import numpy as np
from scipy.special import softmax
import csv
import urllib.request

def preprocess(text):
 new_text = []
for t in text.split( ):
 t = '@user' if t.startswith('@') and len(t) > 1 else t
 t = 'http' if t.startswith('http') else t
 new_text.append(t)
return  .join(new_text)

task='sentiment'
MODEL = fcardiffnlp/twitter-roberta-base-{task}
tokenizer = AutoTokenizer.from_pretrained(MODEL)

labels=[]
mapping_link = fhttps://raw.githubusercontent.com/cardiffnlp/tweeteval/main/datasets/{task}/mapping.txt
with urllib.request.urlopen(mapping_link) as f:
 html = f.read().decode('utf-8').split(\n)
 csvreader = csv.reader(html, delimiter='\t')
labels = [row[1] for row in csvreader if len(row) > 1]

model = AutoModelForSequenceClassification.from_pretrained(MODEL)
model.save_pretrained(MODEL)
text = Good night 😊
text = preprocess(text)
encoded_input = tokenizer(text, return_tensors='pt')
output = model(**encoded_input)
scores = output[0][0].detach().numpy()
scores = softmax(scores)

ranking = np.argsort(scores)
ranking = ranking[::-1]
for i in range(scores.shape[0]):
 l = labels[ranking[i]]
 s = scores[ranking[i]]
 print(f{i+1}) {l} {np.round(float(s), 4)})

Twitter-roBERTa-base for Sentiment Analysis. This is a roBERTa-base model trained on ~58M tweets and finetuned for sentiment analysis with the TweetEval benchmark. This model is suitable for English.


from transformers import pipeline
model_path = cardiffnlp/twitter-xlm-roberta-base-sentiment
sentiment_task = pipeline(sentiment-analysis, model=model_path, tokenizer=model_path)
sentiment_task(T'estimo!)

This is a multilingual XLM-roBERTa-base model trained on ~198M tweets and finetuned for sentiment analysis. The sentiment fine-tuning was done on 8 languages (Ar, En, Fr, De, Hi, It, Sp, Pt) but it can be used for more languages (see paper for details).


from PIL import Image
import requests
from transformers import CLIPProcessor, CLIPModel
model = CLIPModel.from_pretrained(geolocal/StreetCLIP)
processor = CLIPProcessor.from_pretrained(geolocal/StreetCLIP)
url = https://huggingface.co/geolocal/StreetCLIP/resolve/main/sanfrancisco.jpeg
image = Image.open(requests.get(url, stream=True).raw)
choices = [San Jose, San Diego, Los Angeles, Las Vegas, San Francisco]
inputs = processor(text=choices, images=image, return_tensors=pt, padding=True)
outputs = model(**inputs)
logits_per_image = outputs.logits_per_image
probs = logits_per_image.softmax(dim=1)

StreetCLIP is a robust foundation model for open-domain image geolocalization and other geographic and climate-related tasks. Trained on an original dataset of 1.1 million street-level urban and rural geo-tagged images, it achieves state-of-the-art performance on multiple open-domain image geolocalization benchmarks in zero-shot, outperforming supervised models trained on millions of images.


from PIL import Image
import requests
from transformers import ChineseCLIPProcessor, ChineseCLIPModel
model = ChineseCLIPModel.from_pretrained(OFA-Sys/chinese-clip-vit-large-patch14)
processor = ChineseCLIPProcessor.from_pretrained(OFA-Sys/chinese-clip-vit-large-patch14)
url = https://clip-cn-beijing.oss-cn-beijing.aliyuncs.com/pokemon.jpeg
image = Image.open(requests.get(url, stream=True).raw)
texts = []
inputs = processor(images=image, return_tensors=pt)
image_features = model.get_image_features(**inputs)
image_features = image_features / image_features.norm(p=2, dim=-1, keepdim=True) # normalize
inputs = processor(text=texts, padding=True, return_tensors=pt)
text_features = model.get_text_features(**inputs)
text_features = text_features / text_features.norm(p=2, dim=-1, keepdim=True) # normalize
inputs = processor(text=texts, images=image, return_tensors=pt, padding=True)
outputs = model(**inputs)
logits_per_image = outputs.logits_per_image # this is the image-text similarity score
probs = logits_per_image.softmax(dim=1) # probs: [[0.0066, 0.0211, 0.0031, 0.9692]]

MUGE Text-to-Image Retrieval, Flickr30K-CN Retrieval, COCO-CN Retrieval, CIFAR10, CIFAR100, DTD, EuroSAT, FER, FGV, KITTI, MNIST, PASCAL VOC

Chinese-CLIP-ViT-Large-Patch14 is a large version of the Chinese CLIP model, with ViT-L/14 as the image encoder and RoBERTa-wwm-base as the text encoder. Chinese CLIP is a simple implementation of CLIP on a large-scale dataset of around 200 million Chinese image-text pairs. It is designed for zero-shot image classification tasks.


language_detection('Hello, how are you?')

This model is a fine-tuned version of xlm-roberta-base on the Language Identification dataset. It is an XLM-RoBERTa transformer model with a classification head on top, and can be used as a language detector for sequence classification tasks. It supports 20 languages including Arabic, Bulgarian, German, Greek, English, Spanish, French, Hindi, Italian, Japanese, Dutch, Polish, Portuguese, Russian, Swahili, Thai, Turkish, Urdu, Vietnamese, and Chinese.


from transformers import BertTokenizer, BertForSequenceClassification
from transformers import pipeline
finbert = BertForSequenceClassification.from_pretrained('yiyanghkust/finbert-tone',num_labels=3)
tokenizer = BertTokenizer.from_pretrained('yiyanghkust/finbert-tone')
nlp = pipeline(sentiment-analysis, model=finbert, tokenizer=tokenizer)
sentences = [there is a shortage of capital, and we need extra financing,
 growth is strong and we have plenty of liquidity,
 there are doubts about our finances,
 profits are flat]
results = nlp(sentences)
print(results)

10,000 manually annotated sentences from analyst reports

FinBERT is a BERT model pre-trained on financial communication text. It is trained on the following three financial communication corpus: Corporate Reports 10-K & 10-Q, Earnings Call Transcripts, and Analyst Reports. This released finbert-tone model is the FinBERT model fine-tuned on 10,000 manually annotated (positive, negative, neutral) sentences from analyst reports. This model achieves superior performance on financial tone analysis task.


from transformers import pipeline; classifier = pipeline('sentiment-analysis', model='ProsusAI/finbert'); classifier('your_text_here')

FinBERT is a pre-trained NLP model to analyze sentiment of financial text. It is built by further training the BERT language model in the finance domain, using a large financial corpus and thereby fine-tuning it for financial sentiment classification. Financial PhraseBank by Malo et al. (2014) is used for fine-tuning.


from transformers import pipeline
sentiment_task = pipeline(sentiment-analysis, model=model_path, tokenizer=model_path)
sentiment_task(Covid cases are increasing fast!)

This is a RoBERTa-base model trained on ~124M tweets from January 2018 to December 2021, and finetuned for sentiment analysis with the TweetEval benchmark. The model is suitable for English.


string

from transformers import pipeline
classifier = pipeline(text-classification, model=j-hartmann/emotion-english-distilroberta-base, return_all_scores=True)
classifier(I love this!)

This model classifies emotions in English text data. It predicts Ekman's 6 basic emotions, plus a neutral class: anger, disgust, fear, joy, neutral, sadness, and surprise. The model is a fine-tuned checkpoint of DistilRoBERTa-base.


Parrot is a paraphrase-based utterance augmentation framework purpose-built to accelerate training NLU models. This model is an ancillary model for Parrot paraphraser.


from transformers import pipeline
pipe = pipeline(text-classification, model=roberta-base-openai-detector)
print(pipe(Hello world! Is this content AI-generated?))

RoBERTa base OpenAI Detector is the GPT-2 output detector model, obtained by fine-tuning a RoBERTa base model with the outputs of the 1.5B-parameter GPT-2 model. The model can be used to predict if text was generated by a GPT-2 model.


result = sentiment_pipeline('I love this product!')

This a bert-base-multilingual-uncased model finetuned for sentiment analysis on product reviews in six languages: English, Dutch, German, French, Spanish and Italian. It predicts the sentiment of the review as a number of stars (between 1 and 5).


Sentiment Inferencing for stock-related comments

from transformers import RobertaForSequenceClassification, RobertaTokenizer
from transformers import pipeline
import pandas as pd
import emoji
tokenizer_loaded = RobertaTokenizer.from_pretrained('zhayunduo/roberta-base-stocktwits-finetuned')
model_loaded = RobertaForSequenceClassification.from_pretrained('zhayunduo/roberta-base-stocktwits-finetuned')
nlp = pipeline(text-classification, model=model_loaded, tokenizer=tokenizer_loaded)
sentences = pd.Series(['just buy','just sell it','entity rocket to the sky!','go down','even though it is going up, I still think it will not keep this trend in the near future'])
sentences = list(sentences)
results = nlp(sentences)
print(results)

This model is fine-tuned with roberta-base model on 3,200,000 comments from stocktwits, with the user-labeled tags 'Bullish' or 'Bearish'.


prediction = classifier('I love using transformers. The best part is wide range of support and its easy to use')

Distilbert is created with knowledge distillation during the pre-training phase which reduces the size of a BERT model by 40%, while retaining 97% of its language understanding. It's smaller, faster than Bert and any other Bert-based model. Distilbert-base-uncased finetuned on the emotion dataset using HuggingFace Trainer.


from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
model = AutoModelForSequenceClassification.from_pretrained('model_name')
tokenizer = AutoTokenizer.from_pretrained('model_name')
features = tokenizer(['How many people live in Berlin?', 'How many people live in Berlin?'], ['Berlin has a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers.', 'New York City is famous for the Metropolitan Museum of Art.'], padding=True, truncation=True, return_tensors=pt)
model.eval()
with torch.no_grad():
 scores = model(**features).logits
 print(scores)

This model was trained on the MS Marco Passage Ranking task and can be used for Information Retrieval. Given a query, encode the query with all possible passages, then sort the passages in a decreasing order.


Model trained with TASS 2020 corpus (around ~5k tweets) of several dialects of Spanish. Base model is BETO, a BERT model trained in Spanish. Uses POS, NEG, NEU labels.


from transformers import pipeline
nlp = pipeline('text-classification', model='finiteautomata/bertweet-base-sentiment-analysis')
result = nlp('I love this movie!')

Model trained with SemEval 2017 corpus (around ~40k tweets). Base model is BERTweet, a RoBERTa model trained on English tweets. Uses POS, NEG, NEU labels.


This model is a fine-tuned version of distilbert-base-uncased on the imdb dataset. It is used for sentiment analysis on movie reviews and achieves an accuracy of 0.928 on the evaluation set.


Parrot is a paraphrase-based utterance augmentation framework purpose-built to accelerate training NLU models. A paraphrase framework is more than just a paraphrasing model.


from transformers import AutoTokenizer, AutoModelForSequenceClassification

This model was trained on the MS Marco Passage Ranking task. The model can be used for Information Retrieval: Given a query, encode the query will all possible passages (e.g. retrieved with ElasticSearch). Then sort the passages in a decreasing order. See SBERT.net Retrieve & Re-rank for more details. The training code is available here: SBERT.net Training MS Marco


from transformers import AutoModelForSequenceClassification, AutoTokenizer, TextClassificationPipeline
model_path = martin-ha/toxic-comment-model
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForSequenceClassification.from_pretrained(model_path)
pipeline = TextClassificationPipeline(model=model, tokenizer=tokenizer)
print(pipeline('This is a test text.'))

This model is a fine-tuned version of the DistilBERT model to classify toxic comments.


from germansentiment import SentimentModel

Mit keinem guten Ergebniss,Das ist gar nicht mal so gut,

Total awesome!,nicht so schlecht wie erwartet,

Der Test verlief positiv.,Sie fhrt ein grnes Auto.]

holidaycheck
scare
filmstarts
germeval
PotTS
emotions
sb10k
Leipzig Wikipedia Corpus 2016
all

This model was trained for sentiment classification of German language texts. The model uses the Googles Bert architecture and was trained on 1.834 million German-language samples. The training data contains texts from various domains like Twitter, Facebook and movie, app and hotel reviews.


from transformers import pipeline
sentiment_analysis = pipeline(sentiment-analysis, model=siebert/sentiment-roberta-large-english)
print(sentiment_analysis(I love this!))

McAuley and Leskovec (2013) (Review Titles)

This model ('SiEBERT', prefix for 'Sentiment in English') is a fine-tuned checkpoint of RoBERTa-large (Liu et al. 2019). It enables reliable binary sentiment analysis for various types of English-language text. For each instance, it predicts either positive (1) or negative (0) sentiment. The model was fine-tuned and evaluated on 15 data sets from diverse text sources to enhance generalization across different types of texts (reviews, tweets, etc.). Consequently, it outperforms models trained on only one type of text (e.g., movie reviews from the popular SST-2 benchmark) when used on new data as shown below.


from transformers import pipeline
nlp = pipeline('text-classification', model='joeddav/distilbert-base-uncased-go-emotions-student')
result = nlp('I am so happy today!')

This model is distilled from the zero-shot classification pipeline on the unlabeled GoEmotions dataset. It is primarily intended as a demo of how an expensive NLI-based zero-shot model can be distilled to a more efficient student, allowing a classifier to be trained with only unlabeled data.


tokenizer = AutoTokenizer.from_pretrained(shahrukhx01/question-vs-statement-classifier)
model = AutoModelForSequenceClassification.from_pretrained(shahrukhx01/question-vs-statement-classifier)

Trained to add the feature for classifying queries between Question Query vs Statement Query using classification in Haystack


This model is a fine-tuned version of textattack/bert-base-uncased-yelp-polarity on a filtered and manually reviewed Yelp dataset containing restaurant reviews only. It is intended to perform text classification, specifically sentiment analysis, on text data obtained from restaurant reviews to determine if the particular review is positive or negative.


from transformers import AutoModelForSequenceClassification, AutoTokenizer
model = AutoModelForSequenceClassification.from_pretrained(madhurjindal/autonlp-Gibberish-Detector-492513457, use_auth_token=True)
tokenizer = AutoTokenizer.from_pretrained(madhurjindal/autonlp-Gibberish-Detector-492513457, use_auth_token=True)
inputs = tokenizer(I love AutoNLP, return_tensors=pt)
outputs = model(**inputs)

A multi-class text classification model for detecting gibberish text. Trained using AutoNLP and DistilBERT.


classifier(I see you’ve set aside this special time to humiliate yourself in public.)

DistilBERT is a transformer model that performs sentiment analysis. I fine-tuned the model on Reddit posts with the purpose of classifying not safe for work (NSFW) content, specifically text that is considered inappropriate and unprofessional. The model predicts 2 classes, which are NSFW or safe for work (SFW). The model is a fine-tuned version of DistilBERT. It was fine-tuned on 14317 Reddit posts pulled from the Reddit API.


SemEval-2018, EI-reg, Mohammad et al. (2018)

DistilRoBERTa-base is a transformer model that performs sentiment analysis. I fine-tuned the model on transcripts from the Friends show with the goal of classifying emotions from text data, specifically dialogue from Netflix shows or movies. The model predicts 6 Ekman emotions and a neutral class. These emotions include anger, disgust, fear, joy, neutrality, sadness, and surprise.


features = tokenizer(['How many people live in Berlin?', 'How many people live in Berlin?'], ['Berlin has a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers.', 'New York City is famous for the Metropolitan Museum of Art.'], padding=True, truncation=True, return_tensors='pt')

from transformers import AutoTokenizer, AutoModelForSequenceClassification

with torch.no_grad():
    scores = model(**features).logits
    print(scores)

This model was trained on the MS Marco Passage Ranking task. It can be used for Information Retrieval: Given a query, encode the query with all possible passages (e.g. retrieved with ElasticSearch). Then sort the passages in a decreasing order. The training code is available here: SBERT.net Training MS Marco.


Natural Language Processing Token Classification

from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline
tokenizer = AutoTokenizer.from_pretrained(dslim/bert-base-NER)
model = AutoModelForTokenClassification.from_pretrained(dslim/bert-base-NER)
nlp = pipeline(ner, model=model, tokenizer=tokenizer)
example = My name is Wolfgang and I live in Berlin
ner_results = nlp(example)
print(ner_results)

bert-base-NER is a fine-tuned BERT model that is ready to use for Named Entity Recognition and achieves state-of-the-art performance for the NER task. It has been trained to recognize four types of entities: location (LOC), organizations (ORG), person (PER) and Miscellaneous (MISC). Specifically, this model is a bert-base-cased model that was fine-tuned on the English version of the standard CoNLL-2003 Named Entity Recognition dataset.


from flair.data import Sentence
from flair.models import SequenceTagger
tagger = SequenceTagger.load('flair/ner-english-fast')
sentence = Sentence('George Washington went to Washington')
tagger.predict(sentence)
print(sentence)
for entity in sentence.get_spans('ner'):
    print(entity)

This is the fast 4-class NER model for English that ships with Flair. It predicts 4 tags: PER (person name), LOC (location name), ORG (organization name), and MISC (other name). The model is based on Flair embeddings and LSTM-CRF.


from transformers import pipeline, AutoTokenizer, AutoModelForTokenClassification

pipe(The patient reported no recurrence of palpitations at follow-up 6 months after the ablation.)

An English Named Entity Recognition model, trained on Maccrobat to recognize the bio-medical entities (107 entities) from a given text corpus (case reports etc.). This model was built on top of distilbert-base-uncased.


from transformers import AutoTokenizer, AutoModelForTokenClassification
tokenizer = AutoTokenizer.from_pretrained(Jean-Baptiste/camembert-ner)
model = AutoModelForTokenClassification.from_pretrained(Jean-Baptiste/camembert-ner)

from transformers import pipeline
nlp = pipeline('ner', model=model, tokenizer=tokenizer, aggregation_strategy=simple)
nlp(Apple est cre le 1er avril 1976 dans le garage de la maison d'enfance de Steve Jobs  Los Altos en Californie par Steve Jobs, Steve Wozniak et Ronald Wayne14, puis constitue sous forme de socit le 3 janvier 1977  l'origine sous le nom d'Apple Computer, mais pour ses 30 ans et pour reflter la diversification de ses produits, le mot « computer  est retir le 9 janvier 2015.)

camembert-ner is a Named Entity Recognition (NER) model fine-tuned from camemBERT on the wikiner-fr dataset. It can recognize entities such as persons, organizations, locations, and miscellaneous entities.


from deepmultilingualpunctuation import PunctuationModel
model = PunctuationModel()
text = My name is Clara and I live in Berkeley California Ist das eine Frage Frau Müller
result = model.restore_punctuation(text)
print(result)

This model predicts the punctuation of English, Italian, French and German texts. It was developed to restore the punctuation of transcribed spoken language and trained on the Europarl Dataset provided by the SEPP-NLG Shared Task. The model restores the following punctuation markers: ., ,, ?, -, :.


from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline

example = Nader Jokhadar had given Syria the lead with a well-struck header in the seventh minute.; ner_results = nlp(example); print(ner_results)

bert-base-multilingual-cased-ner-hrl is a Named Entity Recognition model for 10 high resourced languages (Arabic, German, English, Spanish, French, Italian, Latvian, Dutch, Portuguese and Chinese) based on a fine-tuned mBERT base model. It has been trained to recognize three types of entities: location (LOC), organizations (ORG), and person (PER).


from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline
tokenizer = AutoTokenizer.from_pretrained(Davlan/distilbert-base-multilingual-cased-ner-hrl)
model = AutoModelForTokenClassification.from_pretrained(Davlan/distilbert-base-multilingual-cased-ner-hrl)
nlp = pipeline(ner, model=model, tokenizer=tokenizer)
example = Nader Jokhadar had given Syria the lead with a well-struck header in the seventh minute.
ner_results = nlp(example)
print(ner_results)

distilbert-base-multilingual-cased-ner-hrl is a Named Entity Recognition model for 10 high resourced languages (Arabic, German, English, Spanish, French, Italian, Latvian, Dutch, Portuguese and Chinese) based on a fine-tuned Distiled BERT base model. It has been trained to recognize three types of entities: location (LOC), organizations (ORG), and person (PER).


nlp('My name is John and I live in New York.')

A pretrained BERT model for Named Entity Recognition (NER) on uncased text. It can be used to extract entities such as person names, locations, and organizations from text.


Stanford de-identifier was trained on a variety of radiology and biomedical documents with the goal of automatising the de-identification process while reaching satisfactory accuracy for use in production.


classifier(Hello I'm Omar and I live in Zürich.)

The XLM-RoBERTa model is a large multi-lingual language model, trained on 2.5TB of filtered CommonCrawl data. This model is XLM-RoBERTa-large fine-tuned with the conll2003 dataset in English. It can be used for token classification tasks such as Named Entity Recognition (NER) and Part-of-Speech (PoS) tagging.


My name is Wolfgang and I live in Berlin

bert-large-NER is a fine-tuned BERT model that is ready to use for Named Entity Recognition and achieves state-of-the-art performance for the NER task. It has been trained to recognize four types of entities: location (LOC), organizations (ORG), person (PER) and Miscellaneous (MISC).


from transformers import AutoModelForTokenClassification, AutoTokenizer
model = AutoModelForTokenClassification.from_pretrained(ismail-lucifer011/autotrain-name_all-904029577, use_auth_token=True)
tokenizer = AutoTokenizer.from_pretrained(ismail-lucifer011/autotrain-name_all-904029577, use_auth_token=True)
inputs = tokenizer(I love AutoTrain, return_tensors=pt)
outputs = model(**inputs)

This model is trained using AutoTrain for entity extraction. It is based on the DistilBert architecture and has a CO2 Emissions of 0.8375653425894861 grams.


from transformers import AutoModelForTokenClassification, AutoTokenizer
model = AutoModelForTokenClassification.from_pretrained(ismail-lucifer011/autotrain-company_all-903429548, use_auth_token=True)
tokenizer = AutoTokenizer.from_pretrained(ismail-lucifer011/autotrain-company_all-903429548, use_auth_token=True)
inputs = tokenizer(I love AutoTrain, return_tensors=pt)
outputs = model(**inputs)

A token classification model trained using AutoTrain for entity extraction. The model is based on the distilbert architecture and trained on the ismail-lucifer011/autotrain-data-company_all dataset. It can be used to identify and extract company names from text.


example = Today's meal: Fresh olive pok bowl topped with chia seeds. Very delicious!
ner_entity_results = pipe(example, aggregation_strategy='simple')
print(ner_entity_results)

InstaFoodRoBERTa-NER is a fine-tuned BERT model that is ready to use for Named Entity Recognition of Food entities on informal text (social media like). It has been trained to recognize a single entity: food (FOOD). Specifically, this model is a roberta-base model that was fine-tuned on a dataset consisting of 400 English Instagram posts related to food.


nlp(Apple was founded in 1976 by Steve Jobs, Steve Wozniak and Ronald Wayne to develop and sell Wozniak's Apple I personal computer)

roberta-large-ner-english is an english NER model that was fine-tuned from roberta-large on conll2003 dataset. Model was validated on emails/chat data and outperformed other models on this type of data specifically. In particular, the model seems to work better on entities that don't start with an upper case.


from flair.data import Sentence
from flair.models import SequenceTagger
tagger = SequenceTagger.load(flair/ner-english-ontonotes)
sentence = Sentence(On September 1st George Washington won 1 dollar.)
tagger.predict(sentence)
print(sentence)
for entity in sentence.get_spans('ner'):
    print(entity)

This is the 18-class NER model for English that ships with Flair. It predicts 18 tags such as cardinal value, date value, event name, building name, geo-political entity, language name, law name, location name, money name, affiliation, ordinal value, organization name, percent value, person name, product name, quantity value, time value, and name of work of art. Based on Flair embeddings and LSTM-CRF.


from transformers import AutoModelForTokenClassification, AutoTokenizer
model = AutoModelForTokenClassification.from_pretrained(ismail-lucifer011/autotrain-job_all-903929564, use_auth_token=True)
tokenizer = AutoTokenizer.from_pretrained(ismail-lucifer011/autotrain-job_all-903929564, use_auth_token=True)
inputs = tokenizer(I love AutoTrain, return_tensors=pt)
outputs = model(**inputs)

A Token Classification model trained using AutoTrain for Entity Extraction. The model is based on distilbert and achieves high accuracy, precision, recall, and F1 score.


A finetuned xlm-roberta-base model for punctuation prediction on twelve languages: English, German, French, Spanish, Bulgarian, Italian, Polish, Dutch, Czech, Portugese, Slovak, Slovenian.


This is a BERT-large-cased model fine-tuned on the CoNLL-03 dataset for token classification tasks.


from flair.data import Sentence
from flair.models import SequenceTagger
tagger = SequenceTagger.load('flair/upos-english')
sentence = Sentence('I love Berlin.')
tagger.predict(sentence)
print(sentence)
for entity in sentence.get_spans('pos'):
    print(entity)

This is the standard universal part-of-speech tagging model for English that ships with Flair. It predicts universal POS tags such as ADJ, ADP, ADV, AUX, CCONJ, DET, INTJ, NOUN, NUM, PART, PRON, PROPN, PUNCT, SCONJ, SYM, VERB, and X. The model is based on Flair embeddings and LSTM-CRF.


from transformers import (
 BertTokenizerFast,
 AutoModel,
)
tokenizer = BertTokenizerFast.from_pretrained('bert-base-chinese')
model = AutoModel.from_pretrained('ckiplab/bert-base-chinese-ws')

This project provides traditional Chinese transformers models (including ALBERT, BERT, GPT2) and NLP tools (including word segmentation, part-of-speech tagging, named entity recognition).


from transformers import (
  BertTokenizerFast,
  AutoModel,
)
tokenizer = BertTokenizerFast.from_pretrained('bert-base-chinese')
model = AutoModel.from_pretrained('ckiplab/bert-base-chinese-pos')


from flair.data import Sentence
from flair.models import SequenceTagger

# load tagger
tagger = SequenceTagger.load(flair/ner-english-large)

# make example sentence
sentence = Sentence(George Washington went to Washington)

# predict NER tags
tagger.predict(sentence)

# print sentence
print(sentence)

# print predicted NER spans
print('The following NER tags are found:')

# iterate over entities and print
for entity in sentence.get_spans('ner'):
 print(entity)

This is the large 4-class NER model for English that ships with Flair. It predicts 4 tags: PER (person name), LOC (location name), ORG (organization name), and MISC (other name). The model is based on document-level XLM-R embeddings and FLERT.


from flair.data import Sentence
from flair.models import SequenceTagger

tagger = SequenceTagger.load(flair/pos-english)

sentence = Sentence(I love Berlin.)

tagger.predict(sentence)

print(sentence)

for entity in sentence.get_spans('pos'):
 print(entity)

This is the standard part-of-speech tagging model for English that ships with Flair. It predicts fine-grained POS tags based on Flair embeddings and LSTM-CRF.


from flair.data import Sentence
from flair.models import SequenceTagger

# load tagger
tagger = SequenceTagger.load('flair/ner-english')

# make example sentence
sentence = Sentence('George Washington went to Washington')

# predict NER tags
tagger.predict(sentence)

# print sentence
print(sentence)

# print predicted NER spans
print('The following NER tags are found:')

# iterate over entities and print
for entity in sentence.get_spans('ner'):
    print(entity)

This is the standard 4-class NER model for English that ships with Flair. It predicts 4 tags: PER (person name), LOC (location name), ORG (organization name), and MISC (other name). The model is based on Flair embeddings and LSTM-CRF.


from flair.data import Sentence
from flair.models import SequenceTagger
tagger = SequenceTagger.load(flair/ner-english-ontonotes-fast)
sentence = Sentence(On September 1st George Washington won 1 dollar.)
tagger.predict(sentence)
print(sentence)
for entity in sentence.get_spans('ner'):
  print(entity)

This is the fast version of the 18-class NER model for English that ships with Flair. It predicts 18 tags such as cardinal value, date value, event name, building name, geo-political entity, language name, law name, location name, money name, affiliation, ordinal value, organization name, percent value, person name, product name, quantity value, time value, and name of work of art. The model is based on Flair embeddings and LSTM-CRF.


A BERT model fine-tuned for Part-of-Speech (POS) tagging in English text.


from flair.data import Sentence
from flair.models import SequenceTagger

# load tagger
tagger = SequenceTagger.load('flair/ner-german')

# make example sentence
sentence = Sentence('George Washington ging nach Washington')

# predict NER tags
tagger.predict(sentence)

# print sentence
print(sentence)

# print predicted NER spans
print('The following NER tags are found:')

# iterate over entities and print
for entity in sentence.get_spans('ner'):
    print(entity)

This is the standard 4-class NER model for German that ships with Flair. It predicts 4 tags: PER (person name), LOC (location name), ORG (organization name), and MISC (other name). The model is based on Flair embeddings and LSTM-CRF.


Natural Language Processing Table Question Answering

from transformers import TapasTokenizer, TapasForQuestionAnswering

model = TapasForQuestionAnswering.from_pretrained('google/tapas-base-finetuned-wtq')
tokenizer = TapasTokenizer.from_pretrained('google/tapas-base-finetuned-wtq')

TAPAS base model fine-tuned on WikiTable Questions (WTQ). This model is pretrained on a large corpus of English data from Wikipedia in a self-supervised fashion, and then fine-tuned on SQA, WikiSQL, and finally WTQ. It can be used for answering questions related to a table.


TAPEX (Table Pre-training via Execution) is a conceptually simple and empirically powerful pre-training approach to empower existing models with table reasoning skills. TAPEX realizes table pre-training by learning a neural SQL executor over a synthetic corpus, which is obtained by automatically synthesizing executable SQL queries. TAPEX is based on the BART architecture, the transformer encoder-encoder (seq2seq) model with a bidirectional (BERT-like) encoder and an autoregressive (GPT-like) decoder.


from transformers import pipeline

qa_pipeline = pipeline('table-question-answering', model='google/tapas-large-finetuned-wtq')

result = qa_pipeline(table=table, query=query)

TAPAS large model fine-tuned on WikiTable Questions (WTQ). This model was pre-trained on MLM and an additional step which the authors call intermediate pre-training, and then fine-tuned in a chain on SQA, WikiSQL and finally WTQ. It uses relative position embeddings (i.e. resetting the position index at every cell of the table).


from transformers import pipeline

# Initialize the pipeline
table_qa_pipeline = pipeline('table-question-answering', model='google/tapas-medium-finetuned-sqa')

# Provide the table and query
result = table_qa_pipeline(table=table, query='What is the total revenue?')

TAPAS medium model fine-tuned on Sequential Question Answering (SQA). This model is pretrained on a large corpus of English data from Wikipedia and uses relative position embeddings. It can be used for answering questions related to a table in a conversational set-up.


from transformers import TapasForQuestionAnswering, TapasTokenizer

model = TapasForQuestionAnswering.from_pretrained('google/tapas-small-finetuned-wtq')
tokenizer = TapasTokenizer.from_pretrained('google/tapas-small-finetuned-wtq')

inputs = tokenizer(table=table, queries=query, return_tensors='pt')
outputs = model(**inputs)

predicted_answer_coordinates, predicted_aggregation_indices = tokenizer.convert_logits_to_predictions(inputs, outputs.logits.detach(), outputs.logits_aggregation.detach())

TAPAS small model fine-tuned on WikiTable Questions (WTQ). This model was pre-trained on MLM and an additional step which the authors call intermediate pre-training, and then fine-tuned in a chain on SQA, WikiSQL and finally WTQ. It uses relative position embeddings (i.e. resetting the position index at every cell of the table).


tokenizer = AutoTokenizer.from_pretrained(Babelscape/wikineural-multilingual-ner)
model = AutoModelForTokenClassification.from_pretrained(Babelscape/wikineural-multilingual-ner)
nlp = pipeline(ner, model=model, tokenizer=tokenizer)
example = My name is Wolfgang and I live in Berlin
ner_results = nlp(example)
print(ner_results)

span-based F1-score up to 6 points over previous state-of-the-art systems for data creation

A multilingual Named Entity Recognition (NER) model fine-tuned on the WikiNEuRal dataset, supporting 9 languages (de, en, es, fr, it, nl, pl, pt, ru). It is based on the mBERT architecture and trained on all 9 languages jointly. The model can be used with the Hugging Face Transformers pipeline for NER tasks.


from flair.data import Sentence
from flair.models import SequenceTagger

# load tagger
tagger = SequenceTagger.load('flair/ner-english-ontonotes-large')

# make example sentence
sentence = Sentence('On September 1st George won 1 dollar while watching Game of Thrones.')

# predict NER tags
tagger.predict(sentence)

# print sentence
print(sentence)

# print predicted NER spans
print('The following NER tags are found:')

# iterate over entities and print
for entity in sentence.get_spans('ner'):
    print(entity)

English NER in Flair (Ontonotes large model). This is the large 18-class NER model for English that ships with Flair. It predicts 18 tags such as cardinal value, date value, event name, building name, geo-political entity, language name, law name, location name, money name, affiliation, ordinal value, organization name, percent value, person name, product name, quantity value, time value, and name of work of art. The model is based on document-level XLM-R embeddings and FLERT.


A tiny TAPAS model for table question answering tasks.


This model can be loaded on the Inference API on-demand.

A tiny TAPAS model trained on the WikiTableQuestions dataset for table question answering tasks.


from transformers import pipeline

# Initialize the pipeline
tapas_pipeline = pipeline('table-question-answering', model='google/tapas-base-finetuned-sqa')

# Define the table and question
question = 'How many goals did player A score?'
table = [['Player', 'Goals'], ['Player A', 5], ['Player B', 3]]

# Get the answer
result = tapas_pipeline(question=question, table=table)
print(result)

TAPAS base model fine-tuned on Sequential Question Answering (SQA). It is a BERT-like transformers model pretrained on a large corpus of English data from Wikipedia and fine-tuned on SQA. It can be used for answering questions related to a table in a conversational set-up.


TAPAS is a BERT-like transformers model pretrained on a large corpus of English data from Wikipedia in a self-supervised fashion. It was pretrained with two objectives: Masked language modeling (MLM) and Intermediate pre-training. Fine-tuning is done by adding a cell selection head and aggregation head on top of the pre-trained model, and then jointly train these randomly initialized classification heads with the base model on SQA and WikiSQL.


TAPAS small model fine-tuned on Sequential Question Answering (SQA). It uses relative position embeddings (i.e. resetting the position index at every cell of the table).


Question: str

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
import pandas as pd
tokenizer = AutoTokenizer.from_pretrained(neulab/omnitab-large-finetuned-wtq)
model = AutoModelForSeq2SeqLM.from_pretrained(neulab/omnitab-large-finetuned-wtq)
data = {
 year: [1896, 1900, 1904, 2004, 2008, 2012],
 city: [athens, paris, st. louis, athens, beijing, london]
}
table = pd.DataFrame.from_dict(data)
query = In which year did beijing host the Olympic Games?
encoding = tokenizer(table=table, query=query, return_tensors=pt)
outputs = model.generate(**encoding)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))

OmniTab is a table-based QA model proposed in OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering. The original Github repository is https://github.com/jzbjyb/OmniTab.


Question: str

from transformers import TapexTokenizer, BartForConditionalGeneration
import pandas as pd
tokenizer = TapexTokenizer.from_pretrained(microsoft/tapex-base)
model = BartForConditionalGeneration.from_pretrained(microsoft/tapex-base)
data = {
 year: [1896, 1900, 1904, 2004, 2008, 2012],
 city: [athens, paris, st. louis, athens, beijing, london]
}
table = pd.DataFrame.from_dict(data)
query = select year where city = beijing
encoding = tokenizer(table=table, query=query, return_tensors=pt)
outputs = model.generate(**encoding)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))

TAPEX (Table Pre-training via Execution) is a conceptually simple and empirically powerful pre-training approach to empower existing models with table reasoning skills. TAPEX realizes table pre-training by learning a neural SQL executor over a synthetic corpus, which is obtained by automatically synthesizing executable SQL queries.


Question: query

from transformers import TapexTokenizer, BartForConditionalGeneration
import pandas as pd
tokenizer = TapexTokenizer.from_pretrained(microsoft/tapex-base-finetuned-wtq)
model = BartForConditionalGeneration.from_pretrained(microsoft/tapex-base-finetuned-wtq)
data = {
 year: [1896, 1900, 1904, 2004, 2008, 2012],
 city: [athens, paris, st. louis, athens, beijing, london]
}
table = pd.DataFrame.from_dict(data)
query = In which year did beijing host the Olympic Games?
encoding = tokenizer(table=table, query=query, return_tensors=pt)
outputs = model.generate(**encoding)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))


Question: str

from transformers import TapexTokenizer, BartForConditionalGeneration
import pandas as pd
tokenizer = TapexTokenizer.from_pretrained(microsoft/tapex-large-finetuned-wtq)
model = BartForConditionalGeneration.from_pretrained(microsoft/tapex-large-finetuned-wtq)
data = {
 year: [1896, 1900, 1904, 2004, 2008, 2012],
 city: [athens, paris, st. louis, athens, beijing, london]
}
table = pd.DataFrame.from_dict(data)
query = In which year did beijing host the Olympic Games?
encoding = tokenizer(table=table, query=query, return_tensors=pt)
outputs = model.generate(**encoding)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))

TAPEX (Table Pre-training via Execution) is a conceptually simple and empirically powerful pre-training approach to empower existing models with table reasoning skills. TAPEX realizes table pre-training by learning a neural SQL executor over a synthetic corpus, which is obtained by automatically synthesizing executable SQL queries. TAPEX is based on the BART architecture, the transformer encoder-decoder (seq2seq) model with a bidirectional (BERT-like) encoder and an autoregressive (GPT-like) decoder. This model is the tapex-base model fine-tuned on the WikiTableQuestions dataset.


from transformers import pipeline; table_qa = pipeline('table-question-answering', model='dsba-lab/koreapas-finetuned-korwikitq'); table_qa(table=table, query='질문')

A Korean Table Question Answering model finetuned on the korwikitq dataset.


Question: query

from transformers import TapexTokenizer, BartForConditionalGeneration
import pandas as pd
tokenizer = TapexTokenizer.from_pretrained(microsoft/tapex-large-finetuned-wikisql)
model = BartForConditionalGeneration.from_pretrained(microsoft/tapex-large-finetuned-wikisql)
data = {
 year: [1896, 1900, 1904, 2004, 2008, 2012],
 city: [athens, paris, st. louis, athens, beijing, london]
}
table = pd.DataFrame.from_dict(data)
query = In which year did beijing host the Olympic Games?
encoding = tokenizer(table=table, query=query, return_tensors=pt)
outputs = model.generate(**encoding)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))

TAPEX (Table Pre-training via Execution) is a conceptually simple and empirically powerful pre-training approach to empower existing models with table reasoning skills. TAPEX realizes table pre-training by learning a neural SQL executor over a synthetic corpus, which is obtained by automatically synthesizing executable SQL queries. TAPEX is based on the BART architecture, the transformer encoder-encoder (seq2seq) model with a bidirectional (BERT-like) encoder and an autoregressive (GPT-like) decoder. This model is the tapex-base model fine-tuned on the WikiSQL dataset.


Question: str

from transformers import TapexTokenizer, BartForConditionalGeneration
import pandas as pd
tokenizer = TapexTokenizer.from_pretrained(microsoft/tapex-large-sql-execution)
model = BartForConditionalGeneration.from_pretrained(microsoft/tapex-large-sql-execution)
data = {
 year: [1896, 1900, 1904, 2004, 2008, 2012],
 city: [athens, paris, st. louis, athens, beijing, london]
}
table = pd.DataFrame.from_dict(data)
query = select year where city = beijing
encoding = tokenizer(table=table, query=query, return_tensors=pt)
outputs = model.generate(**encoding)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))


from transformers import pipeline

qa_pipeline = pipeline('table-question-answering', model='google/tapas-large-finetuned-wikisql-supervised')

result = qa_pipeline(question='What is the capital of France?', table=table)
print(result)

TAPAS is a BERT-like transformers model pretrained on a large corpus of English data from Wikipedia in a self-supervised fashion. It can be used for answering questions related to a table.


TAPAS large model fine-tuned on Sequential Question Answering (SQA). This model was pre-trained on MLM and an additional step which the authors call intermediate pre-training, and then fine-tuned on SQA. It uses relative position embeddings (i.e. resetting the position index at every cell of the table).


from transformers import TapasTokenizer, TapasForQuestionAnswering
tokenizer = TapasTokenizer.from_pretrained('google/tapas-small-finetuned-wikisql-supervised')
model = TapasForQuestionAnswering.from_pretrained('google/tapas-small-finetuned-wikisql-supervised')

TAPAS is a BERT-like transformers model pretrained on a large corpus of English data from Wikipedia in a self-supervised fashion. This model is fine-tuned on WikiSQL and can be used for answering questions related to a table.


SQA (Sequential Question Answering by Microsoft)

WTQ (Wiki Table Questions by Stanford University)

TAPAS, the model learns an inner representation of the English language used in tables and associated texts, which can then be used to extract features useful for downstream tasks such as answering questions about a table, or determining whether a sentence is entailed or refuted by the contents of a table. It is a BERT-based model specifically designed (and pre-trained) for answering questions about tabular data. TAPAS uses relative position embeddings and has 7 token types that encode tabular structure. It is pre-trained on the masked language modeling (MLM) objective on a large dataset comprising millions of tables from English Wikipedia and corresponding texts.


tokenizer = AutoTokenizer.from_pretrained('google/tapas-mini-finetuned-wtq'); model = AutoModelForTableQuestionAnswering.from_pretrained('google/tapas-mini-finetuned-wtq')

from transformers import pipeline

nlp = pipeline('table-question-answering', model='google/tapas-mini-finetuned-wtq', tokenizer='google/tapas-mini-finetuned-wtq')

TAPAS mini model fine-tuned on WikiTable Questions (WTQ). It is pretrained on a large corpus of English data from Wikipedia and can be used for answering questions related to a table.


Question: str

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
import pandas as pd
tokenizer = AutoTokenizer.from_pretrained(neulab/omnitab-large)
model = AutoModelForSeq2SeqLM.from_pretrained(neulab/omnitab-large)
data = {
 year: [1896, 1900, 1904, 2004, 2008, 2012],
 city: [athens, paris, st. louis, athens, beijing, london]
}
table = pd.DataFrame.from_dict(data)
query = In which year did beijing host the Olympic Games?
encoding = tokenizer(table=table, query=query, return_tensors=pt)
outputs = model.generate(**encoding)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))

OmniTab is a table-based QA model proposed in OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering. neulab/omnitab-large (based on BART architecture) is initialized with microsoft/tapex-large and continuously pretrained on natural and synthetic data.


tokenizer = TapasTokenizer.from_pretrained('lysandre/tapas-temporary-repo')
model = TapasForQuestionAnswering.from_pretrained('lysandre/tapas-temporary-repo')
inputs = tokenizer(table=table, queries=question, return_tensors='pt')
outputs = model(**inputs)
predicted_answer_coordinates, predicted_aggregation_indices = tokenizer.convert_logits_to_predictions(inputs, outputs.logits.detach(), outputs.logits_aggregation.detach())

TAPAS base model fine-tuned on Sequential Question Answering (SQA). This model is pretrained on a large corpus of English data from Wikipedia in a self-supervised fashion and can be used for answering questions related to a table in a conversational set-up.


Question: str

from transformers import TapexTokenizer, BartForConditionalGeneration
import pandas as pd
tokenizer = TapexTokenizer.from_pretrained(microsoft/tapex-base-finetuned-wikisql)
model = BartForConditionalGeneration.from_pretrained(microsoft/tapex-base-finetuned-wikisql)
data = {
 year: [1896, 1900, 1904, 2004, 2008, 2012],
 city: [athens, paris, st. louis, athens, beijing, london]
}
table = pd.DataFrame.from_dict(data)
query = In which year did beijing host the Olympic Games?
encoding = tokenizer(table=table, query=query, return_tensors=pt)
outputs = model.generate(**encoding)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))


TAPAS mini model fine-tuned on Sequential Question Answering (SQA)


Question: str

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
import pandas as pd
tokenizer = AutoTokenizer.from_pretrained(neulab/omnitab-large-1024shot)
model = AutoModelForSeq2SeqLM.from_pretrained(neulab/omnitab-large-1024shot)
data = {
 year: [1896, 1900, 1904, 2004, 2008, 2012],
 city: [athens, paris, st. louis, athens, beijing, london]
}
table = pd.DataFrame.from_dict(data)
query = In which year did beijing host the Olympic Games?
encoding = tokenizer(table=table, query=query, return_tensors=pt)
outputs = model.generate(**encoding)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))

OmniTab is a table-based QA model proposed in OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering. neulab/omnitab-large-1024shot (based on BART architecture) is initialized with microsoft/tapex-large and continuously pretrained on natural and synthetic data (SQL2NL model trained in the 1024-shot setting).


from transformers import pipeline

# Initialize the pipeline
table_qa = pipeline('table-question-answering', model='google/tapas-medium-finetuned-wtq')

# Define the table and the query
table = {...}
query = '...'

# Get the answer
answer = table_qa(table=table, query=query)

TAPAS medium model fine-tuned on WikiTable Questions (WTQ). This model is pretrained on a large corpus of English data from Wikipedia and is used for answering questions related to a table.


Question: string

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
import pandas as pd
tokenizer = AutoTokenizer.from_pretrained(neulab/omnitab-large-1024shot-finetuned-wtq-1024shot)
model = AutoModelForSeq2SeqLM.from_pretrained(neulab/omnitab-large-1024shot-finetuned-wtq-1024shot)
data = {
 year: [1896, 1900, 1904, 2004, 2008, 2012],
 city: [athens, paris, st. louis, athens, beijing, london]
}
table = pd.DataFrame.from_dict(data)
query = In which year did beijing host the Olympic Games?
encoding = tokenizer(table=table, query=query, return_tensors=pt)
outputs = model.generate(**encoding)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))

OmniTab is a table-based QA model proposed in OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering. The original Github repository is https://github.com/jzbjyb/OmniTab. neulab/omnitab-large-1024shot-finetuned-wtq-1024shot (based on BART architecture) is initialized with neulab/omnitab-large-1024shot and fine-tuned on WikiTableQuestions in the 1024-shot setting.


Natural Language Processing Question Answering

from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
model_name = deepset/roberta-base-squad2
nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
QA_input = {
 'question': 'Why is model conversion important?',
 'context': 'The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.'
}
res = nlp(QA_input)

This is the roberta-base model, fine-tuned using the SQuAD2.0 dataset for the task of Question Answering. It's been trained on question-answer pairs, including unanswerable questions.


Question: query

from transformers import AutoModelForTableQuestionAnswering, AutoTokenizer, pipeline

# Load model & tokenizer
tapas_model = AutoModelForTableQuestionAnswering.from_pretrained('navteca/tapas-large-finetuned-wtq')
tapas_tokenizer = AutoTokenizer.from_pretrained('navteca/tapas-large-finetuned-wtq')

# Get predictions
nlp = pipeline('table-question-answering', model=tapas_model, tokenizer=tapas_tokenizer)
result = nlp({'table': {'Repository': ['Transformers', 'Datasets', 'Tokenizers'], 'Stars': ['36542', '4512', '3934'], 'Contributors': ['651', '77', '34'], 'Programming language': ['Python', 'Python', 'Rust, Python and NodeJS']}, 'query': 'How many stars does the transformers repository have?'})
print(result)

TAPAS large model fine-tuned on WikiTable Questions (WTQ). It is a BERT-like transformers model pretrained on a large corpus of English data from Wikipedia in a self-supervised fashion. It can be used for answering questions related to a table.


from transformers import pipeline
qa_pipeline = pipeline('question-answering', model='sultan/BioM-ELECTRA-Large-SQuAD2')
result = qa_pipeline({'context': 'your_context', 'question': 'your_question'})

BioM-ELECTRA-Large-SQuAD2 is a fine-tuned version of BioM-ELECTRA-Large, which was pre-trained on PubMed Abstracts, on the SQuAD2.0 dataset. Fine-tuning the biomedical language model on the SQuAD dataset helps improve the score on the BioASQ challenge. This model is suitable for working with BioASQ or biomedical QA tasks.


from transformers import pipeline
question_answerer = pipeline(question-answering, model='distilbert-base-uncased-distilled-squad')
context = r
... Extractive Question Answering is the task of extracting an answer from a text given a question. An example of a
... question answering dataset is the SQuAD dataset, which is entirely based on that task. If you would like to fine-tune
... a model on a SQuAD task, you may leverage the examples/pytorch/question-answering/run_squad.py script.
...
result = question_answerer(question=What is a good example of a question answering dataset?, context=context)
print(
... fAnswer: '{result['answer']}', score: {round(result['score'], 4)}, start: {result['start']}, end: {result['end']}
...)

DistilBERT base uncased distilled SQuAD is a fine-tuned version of DistilBERT-base-uncased for the task of question answering. It has 40% less parameters than bert-base-uncased, runs 60% faster while preserving over 95% of BERT's performances as measured on the GLUE language understanding benchmark.


Context: string

Question: string

qa_pipeline({
 'context': Manuel Romero has been working hardly in the repository hugginface/transformers lately,
 'question': Who has been working hard for hugginface/transformers lately?
})

This model is a BERT (base-multilingual-cased) fine-tuned for multilingual Question Answering on 11 different languages using the XQuAD dataset and additional data augmentation techniques.


from transformers import pipeline
qa_pipeline = pipeline('question-answering', model='bert-large-uncased-whole-word-masking-finetuned-squad')
result = qa_pipeline({'question': 'What is the capital of France?', 'context': 'Paris is the capital of France.'})
print(result)

BERT large model (uncased) whole word masking finetuned on SQuAD. The model was pretrained on BookCorpus and English Wikipedia. It was trained with two objectives: Masked language modeling (MLM) and Next sentence prediction (NSP). This model should be used as a question-answering model.


QA_input = {
 'question': 'Why is model conversion important?',
 'context': 'The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.'
}
res = nlp(QA_input)

This is a bert-large model, fine-tuned using the SQuAD2.0 dataset for the task of question answering. It is designed for extractive question answering and supports English language.


from transformers import pipeline
question_answerer = pipeline(question-answering, model='distilbert-base-cased-distilled-squad')
context = r
... Extractive Question Answering is the task of extracting an answer from a text given a question. An example of a
... question answering dataset is the SQuAD dataset, which is entirely based on that task. If you would like to fine-tune
... a model on a SQuAD task, you may leverage the examples/pytorch/question-answering/run_squad.py script.
...
result = question_answerer(question=What is a good example of a question answering dataset?, context=context)
print(
... fAnswer: '{result['answer']}', score: {round(result['score'], 4)}, start: {result['start']}, end: {result['end']}
...)

DistilBERT base cased distilled SQuAD is a fine-tuned checkpoint of DistilBERT-base-cased, trained using knowledge distillation on SQuAD v1.1 dataset. It has 40% less parameters than bert-base-uncased, runs 60% faster while preserving over 95% of BERT's performances as measured on the GLUE language understanding benchmark. This model can be used for question answering.


from transformers import AutoTokenizer, AutoModelForQuestionAnswering

This model is trained for the task of Question Answering on Legal Documents using the CUAD dataset. It is based on the RoBERTa architecture and can be used to extract answers from legal contracts and documents.


A Korean Question Answering model based on Electra and trained on the KorQuAD dataset.


from transformers import pipeline; nlp = pipeline('question-answering', model='deepset/roberta-large-squad2'); nlp({'question': 'What is the capital of Germany?', 'context': 'Berlin is the capital of Germany.'})

A pre-trained RoBERTa model for question answering tasks, specifically trained on the SQuAD v2 dataset. It can be used to answer questions based on a given context.


Question: Why is model conversion important?

Context: The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.

This model is a Roberta-based model fine-tuned on SQuAD-style CORD-19 annotations for the task of extractive question answering in the context of COVID-19. It can be used with the Hugging Face Transformers library for question answering tasks.


import torch
from transformers import AutoTokenizer, AutoModelForQuestionAnswering
tokenizer = AutoTokenizer.from_pretrained('valhalla/longformer-base-4096-finetuned-squadv1')
model = AutoModelForQuestionAnswering.from_pretrained('valhalla/longformer-base-4096-finetuned-squadv1')
text = 'Huggingface has democratized NLP. Huge thanks to Huggingface for this.'
question = 'What has Huggingface done ?'
encoding = tokenizer(question, text, return_tensors='pt')
input_ids = encoding['input_ids']
attention_mask = encoding['attention_mask']
start_scores, end_scores = model(input_ids, attention_mask=attention_mask)
all_tokens = tokenizer.convert_ids_to_tokens(input_ids[0].tolist())
answer_tokens = all_tokens[torch.argmax(start_scores) :torch.argmax(end_scores)+1]
answer = tokenizer.decode(tokenizer.convert_tokens_to_ids(answer_tokens))

This is longformer-base-4096 model fine-tuned on SQuAD v1 dataset for question answering task. Longformer model created by Iz Beltagy, Matthew E. Peters, Arman Coha from AllenAI. As the paper explains it, Longformer is a BERT-like model for long documents. The pre-trained model can handle sequences with up to 4096 tokens.


Question: Why is model conversion important?

from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
model_name = deepset/tinyroberta-squad2
nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
QA_input = {
 'question': 'Why is model conversion important?',
 'context': 'The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.'
}
res = nlp(QA_input)

This is the distilled version of the deepset/roberta-base-squad2 model. This model has a comparable prediction quality and runs at twice the speed of the base model.


from transformers import pipeline
qa_pipeline = pipeline('question-answering', model='philschmid/distilbert-onnx')
qa_pipeline({'context': 'This is a context', 'question': 'What is this?'})

This model is a fine-tune checkpoint of DistilBERT-base-cased, fine-tuned using (a second step of) knowledge distillation on SQuAD v1.1.


qa_pipeline({'context': 'This model can be loaded on the Inference API on-demand.', 'question': 'Where can this model be loaded?'})

This is a BERT base cased model trained on SQuAD v2


qa_pipeline({'question': 'What is the capital of France?', 'context': 'France is a country in Europe. Its capital is Paris.'})

ELECTRA_large_discriminator language model fine-tuned on SQuAD2.0 for question answering tasks.


qa({'context': 'This is an example context.', 'question': 'What is this example about?'})

This model is a distilled version of deepset/bert-large-uncased-whole-word-masking-squad2, trained on the SQuAD 2.0 dataset for question answering tasks. It is based on the BERT-medium architecture and uses the Hugging Face Transformers library.


from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
model_name = deepset/minilm-uncased-squad2
nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
QA_input = {
 'question': 'Why is model conversion important?',
 'context': 'The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.'
}
res = nlp(QA_input)

from transformers import AutoModelForQuestionAnswering, AutoTokenizer
model_name = deepset/minilm-uncased-squad2
model = AutoModelForQuestionAnswering.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

MiniLM-L12-H384-uncased is a language model fine-tuned for extractive question answering on the SQuAD 2.0 dataset. It is based on the microsoft/MiniLM-L12-H384-uncased model and can be used with the Hugging Face Transformers library.


Context: string

Question: string

from transformers import pipeline
qa_pipeline = pipeline('question-answering', model='deepset/roberta-base-squad2-distilled')
result = qa_pipeline({'context': 'This is a context.', 'question': 'What is this?'})
print(result)

This model is a distilled version of deepset/roberta-large-squad2, trained on SQuAD 2.0 dataset for question answering tasks. It is based on the Roberta architecture and has been fine-tuned using Haystack's distillation feature.


model_name = 'thatdramebaazguy/roberta-base-squad'
qa_pipeline = pipeline(model=model_name, tokenizer=model_name, revision='v1.0', task='question-answering')

This is Roberta Base trained to do the SQuAD Task. This makes a QA model capable of answering questions.


from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline

nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)

Question: Why is model conversion important?

This is the deberta-v3-base model, fine-tuned using the SQuAD2.0 dataset. It's been trained on question-answer pairs, including unanswerable questions, for the task of Question Answering.


from transformers import pipeline
qa_pipeline = pipeline(
 question-answering,
 model=csarron/bert-base-uncased-squad-v1,
 tokenizer=csarron/bert-base-uncased-squad-v1
)
predictions = qa_pipeline({
 'context': The game was played on February 7, 2016 at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California.,
 'question': What day was the game played on?
})
print(predictions)

BERT-base uncased model fine-tuned on SQuAD v1. This model is case-insensitive and does not make a difference between english and English.


A VisualBERT model for Visual Question Answering.


from transformers import pipeline
qa_pipeline = pipeline('question-answering', model='luhua/chinese_pretrain_mrc_roberta_wwm_ext_large')
result = qa_pipeline({'context': 'your_context_here', 'question': 'your_question_here'})

A Chinese MRC roberta_wwm_ext_large model trained on a large amount of Chinese MRC data. This model has significantly improved performance on reading comprehension and classification tasks. It has helped multiple users achieve top 5 results in the Dureader-2021 competition.


from transformers import pipeline
qa_pipeline = pipeline('question-answering', model='bert-large-cased-whole-word-masking-finetuned-squad')
result = qa_pipeline({'context': 'This is a context example.', 'question': 'What is this example for?'})
print(result)

BERT large model (cased) whole word masking finetuned on SQuAD. This model is cased and trained with a new technique: Whole Word Masking. After pre-training, this model was fine-tuned on the SQuAD dataset.


qa_pipeline({'context': 'This model can be loaded on the Inference API on-demand.', 'question': 'Where can the model be loaded?'})

This model is a fine-tuned version of cambridgeltl/SapBERT-from-PubMedBERT-fulltext on the squad_v2 dataset.


nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
QA_input = {
 'question': 'Why is model conversion important?',
 'context': 'The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.'
}
res = nlp(QA_input)

model = AutoModelForQuestionAnswering.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

This is the deberta-v3-large model, fine-tuned using the SQuAD2.0 dataset. It's been trained on question-answer pairs, including unanswerable questions, for the task of Question Answering.


from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline

QA_input = {
 'question': 'Why is model conversion important?',
 'context': 'The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.'
}
res = nlp(QA_input)

Multilingual XLM-RoBERTa large model for extractive question answering on various languages. Trained on SQuAD 2.0 dataset and evaluated on SQuAD dev set, German MLQA, and German XQuAD.


nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
QA_input = {
 'question': 'Why is model conversion important?',
 'context': 'The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.'
}
res = nlp(QA_input)

from transformers import AutoModelForQuestionAnswering, AutoTokenizer

model = AutoModelForQuestionAnswering.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

This is an ONNX conversion of the deepset/roberta-base-squad2 model for extractive question answering. It is trained on the SQuAD 2.0 dataset and is compatible with the Transformers library.


distilbart-mnli is the distilled version of bart-large-mnli created using the No Teacher Distillation technique proposed for BART summarisation by Huggingface. It is designed for zero-shot classification tasks.


sequence = L'équipe de France joue aujourd'hui au Parc des Princes
candidate_labels = [sport,politique,science]
hypothesis_template = Ce texte parle de {}.
classifier(sequence, candidate_labels, hypothesis_template=hypothesis_template)

Camembert-base model fine-tuned on french part of XNLI dataset. One of the few Zero-Shot classification models working on French.


sequence_to_classify = Algún día iré a ver el mundo
candidate_labels = ['viaje', 'cocina', 'danza']
classifier(sequence_to_classify, candidate_labels)

XLM-RoBERTa-large model finetunned over several NLI datasets, ready to use for zero-shot classification.


tokenizer = AutoTokenizer.from_pretrained(typeform/distilbert-base-uncased-mnli)
model = AutoModelForSequenceClassification.from_pretrained(typeform/distilbert-base-uncased-mnli)

This is the uncased DistilBERT model fine-tuned on Multi-Genre Natural Language Inference (MNLI) dataset for the zero-shot classification task.


sequence_to_classify = '¿A quién vas a votar en 2020?'
candidate_labels = ['Europa', 'salud pública', 'política']
hypothesis_template = 'Este ejemplo es {}.'
classifier(sequence_to_classify, candidate_labels, hypothesis_template=hypothesis_template)

This model takes xlm-roberta-large and fine-tunes it on a combination of NLI data in 15 languages. It is intended to be used for zero-shot text classification, such as with the Hugging Face ZeroShotClassificationPipeline.


from transformers import AutoModelForSequenceClassification, AutoTokenizer, pipeline

from transformers import pipeline
classifier = pipeline('zero-shot-classification', model='facebook/bart-large-mnli')
sequence_to_classify = 'one day I will see the world'
candidate_labels = ['travel', 'cooking', 'dancing']
classifier(sequence_to_classify, candidate_labels)

from transformers import AutoModelForSequenceClassification, AutoTokenizer
nli_model = AutoModelForSequenceClassification.from_pretrained('facebook/bart-large-mnli')
tokenizer = AutoTokenizer.from_pretrained('facebook/bart-large-mnli')
premise = sequence
hypothesis = f'This example is {label}.'
x = tokenizer.encode(premise, hypothesis, return_tensors='pt', truncation_strategy='only_first')
logits = nli_model(x.to(device))[0]
entail_contradiction_logits = logits[:,[0,2]]
probs = entail_contradiction_logits.softmax(dim=1)
prob_label_is_true = probs[:,1]

This is the checkpoint for bart-large after being trained on the MultiNLI (MNLI) dataset. The model can be used for zero-shot text classification by posing the sequence to be classified as the NLI premise and constructing a hypothesis from each candidate label. The probabilities for entailment and contradiction are then converted to label probabilities.


distilbart-mnli is the distilled version of bart-large-mnli created using the No Teacher Distillation technique proposed for BART summarisation by Huggingface. It is a simple and effective technique with very little performance drop.


DeBERTa-v3-base fine-tuned with multi-task learning on 520 tasks of the tasksource collection. This checkpoint has strong zero-shot validation performance on many tasks, and can be used for zero-shot NLI pipeline (similar to bart-mnli but better).


from transformers import pipeline
classifier = pipeline(zero-shot-classification, model=Recognai/bert-base-spanish-wwm-cased-xnli)
classifier(
El autor se perfila, a los 50 años de su muerte, como uno de los grandes de su siglo,
candidate_labels=[cultura, sociedad, economia, salud, deportes],
hypothesis_template=Este ejemplo es {}.
)

This model is a fine-tuned version of the spanish BERT model with the Spanish portion of the XNLI dataset. You can have a look at the training script for details of the training.


Cross-Encoder for Natural Language Inference

('A man is eating pizza', 'A man eats something')

from transformers import pipeline
classifier = pipeline(zero-shot-classification, model='cross-encoder/nli-distilroberta-base')
sent = Apple just announced the newest iPhone X
candidate_labels = [technology, sports, politics]
res = classifier(sent, candidate_labels)
print(res)

See SBERT.net - Pretrained Cross-Encoder for evaluation results

This model was trained using SentenceTransformers Cross-Encoder class on the SNLI and MultiNLI datasets. For a given sentence pair, it will output three scores corresponding to the labels: contradiction, entailment, neutral.


from sentence_transformers import CrossEncoder
model = CrossEncoder('cross-encoder/nli-deberta-v3-small')
scores = model.predict([('A man is eating pizza', 'A man eats something'), ('A black race car starts up in front of a crowd of people.', 'A man is driving down a lonely road.')])

Cross-Encoder for Natural Language Inference based on microsoft/deberta-v3-small, trained on the SNLI and MultiNLI datasets. For a given sentence pair, it will output three scores corresponding to the labels: contradiction, entailment, neutral.


from sentence_transformers import CrossEncoder
model = CrossEncoder('cross-encoder/nli-deberta-v3-base')
scores = model.predict([('A man is eating pizza', 'A man eats something'), ('A black race car starts up in front of a crowd of people.', 'A man is driving down a lonely road.')])

This model is based on microsoft/deberta-v3-base and was trained on the SNLI and MultiNLI datasets. For a given sentence pair, it will output three scores corresponding to the labels: contradiction, entailment, neutral.


from transformers import pipeline
classifier = pipeline(zero-shot-classification, model='cross-encoder/nli-roberta-base')
sent = Apple just announced the newest iPhone X
candidate_labels = [technology, sports, politics]
res = classifier(sent, candidate_labels)
print(res)

See SBERT.net - Pretrained Cross-Encoder

Cross-Encoder for Natural Language Inference trained on the SNLI and MultiNLI datasets. Outputs three scores corresponding to the labels: contradiction, entailment, neutral.


from transformers import pipeline
classifier = pipeline(zero-shot-classification, model=MoritzLaurer/mDeBERTa-v3-base-mnli-xnli)
sequence_to_classify = Angela Merkel ist eine Politikerin in Deutschland und Vorsitzende der CDU
candidate_labels = [politics, economy, entertainment, environment]
output = classifier(sequence_to_classify, candidate_labels, multi_label=False)
print(output)

This multilingual model can perform natural language inference (NLI) on 100 languages and is therefore also suitable for multilingual zero-shot classification. The underlying model was pre-trained by Microsoft on the CC100 multilingual dataset. It was then fine-tuned on the XNLI dataset, which contains hypothesis-premise pairs from 15 languages, as well as the English MNLI dataset.


from transformers import pipeline
classifier = pipeline(zero-shot-classification, model=MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli)
sequence_to_classify = Angela Merkel is a politician in Germany and leader of the CDU
candidate_labels = [politics, economy, entertainment, environment]
output = classifier(sequence_to_classify, candidate_labels, multi_label=False)
print(output)

This model was fine-tuned on the MultiNLI, Fever-NLI, Adversarial-NLI (ANLI), LingNLI and WANLI datasets, which comprise 885 242 NLI hypothesis-premise pairs. This model is the best performing NLI model on the Hugging Face Hub as of 06.06.22 and can be used for zero-shot classification. It significantly outperforms all other large models on the ANLI benchmark.


from transformers import AutoTokenizer, AutoModelForSequenceClassification
tokenizer = AutoTokenizer.from_pretrained(typeform/mobilebert-uncased-mnli)
model = AutoModelForSequenceClassification.from_pretrained(typeform/mobilebert-uncased-mnli)

This model is the Multi-Genre Natural Language Inference (MNLI) fine-turned version of the uncased MobileBERT model. It can be used for the task of zero-shot classification.


from sentence_transformers import CrossEncoder
model = CrossEncoder('cross-encoder/nli-MiniLM2-L6-H768')
scores = model.predict([('A man is eating pizza', 'A man eats something'), ('A black race car starts up in front of a crowd of people.', 'A man is driving down a lonely road.')])


distilbart-mnli is the distilled version of bart-large-mnli created using the No Teacher Distillation technique proposed for BART summarisation by Huggingface. It is used for zero-shot text classification tasks.


from transformers import pipeline
nlp = pipeline('zero-shot-classification', model='typeform/squeezebert-mnli')
result = nlp('The quick brown fox jumps over the lazy dog', candidate_labels=['sports', 'language', 'animals'])

SqueezeBERT is a transformer model designed for efficient inference on edge devices. This specific model, typeform/squeezebert-mnli, is fine-tuned on the MultiNLI dataset for zero-shot classification tasks.


Premise: string

Hypothesis: string

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
device = torch.device(cuda) if torch.cuda.is_available() else torch.device(cpu)
model_name = MoritzLaurer/DeBERTa-v3-xsmall-mnli-fever-anli-ling-binary
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
premise = I first thought that I liked the movie, but upon second thought it was actually disappointing.
hypothesis = The movie was good.
input = tokenizer(premise, hypothesis, truncation=True, return_tensors=pt)
output = model(input[input_ids].to(device))
prediction = torch.softmax(output[logits][0], -1).tolist()
label_names = [entailment, not_entailment]
prediction = {name: round(float(pred) * 100, 1) for pred, name in zip(prediction, label_names)}
print(prediction)

This model was trained on 782 357 hypothesis-premise pairs from 4 NLI datasets: MultiNLI, Fever-NLI, LingNLI and ANLI. The base model is DeBERTa-v3-xsmall from Microsoft. The v3 variant of DeBERTa substantially outperforms previous versions of the model by including a different pre-training objective.


from transformers import pipeline
nlp = pipeline('zero-shot-classification', model='valhalla/distilbart-mnli-12-6')
result = nlp('The movie was great!', ['positive', 'negative'])
print(result)

distilbart-mnli is the distilled version of bart-large-mnli created using the No Teacher Distillation technique proposed for BART summarisation by Huggingface. It is designed for zero-shot classification tasks.


politics economy entertainment environment

from transformers import pipeline
classifier = pipeline(zero-shot-classification, model=MoritzLaurer/mDeBERTa-v3-base-mnli-xnli)
sequence_to_classify = Angela Merkel ist eine Politikerin in Deutschland und Vorsitzende der CDU
candidate_labels = [politics, economy, entertainment, environment]
output = classifier(sequence_to_classify, candidate_labels, multi_label=False)
print(output)

This multilingual model can perform natural language inference (NLI) on 100 languages and is therefore also suitable for multilingual zero-shot classification. The underlying mDeBERTa-v3-base model was pre-trained by Microsoft on the CC100 multilingual dataset with 100 languages. The model was then fine-tuned on the XNLI dataset and on the multilingual-NLI-26lang-2mil7 dataset. Both datasets contain more than 2.7 million hypothesis-premise pairs in 27 languages spoken by more than 4 billion people.


from transformers import pipeline
classifier = pipeline('zero-shot-classification', model='cross-encoder/nli-deberta-v3-xsmall')
sent = 'Apple just announced the newest iPhone X'
candidate_labels = ['technology', 'sports', 'politics']
res = classifier(sent, candidate_labels)
print(res)

This model is a Cross-Encoder for Natural Language Inference, trained on the SNLI and MultiNLI datasets. It can be used for zero-shot classification tasks.


from transformers import pipeline
classifier = pipeline(zero-shot-classification, model=Sahajtomar/German_Zeroshot)
sequence = Letzte Woche gab es einen Selbstmord in einer nahe gelegenen kolonie
candidate_labels = [Verbrechen,Tragödie,Stehlen]
hypothesis_template = In deisem geht es um {}. ## Since monolingual model,its sensitive to hypothesis template. This can be experimented
classifier(sequence, candidate_labels, hypothesis_template=hypothesis_template)

This model has GBERT Large as base model and fine-tuned it on xnli de dataset. The default hypothesis template is in English: This text is {}. While using this model, change it to In deisem geht es um {}. or something different. While inferencing through huggingface api may give poor results as it uses by default english template. Since model is monolingual and not multilingual, hypothesis template needs to be changed accordingly.


DeBERTa improves the BERT and RoBERTa models using disentangled attention and enhanced mask decoder. It outperforms BERT and RoBERTa on the majority of NLU tasks with 80GB training data. This is the DeBERTa large model fine-tuned with MNLI task.


from transformers import pipeline
classifier = pipeline(zero-shot-classification, model=MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli)
sequence_to_classify = Angela Merkel is a politician in Germany and leader of the CDU
candidate_labels = [politics, economy, entertainment, environment]
output = classifier(sequence_to_classify, candidate_labels, multi_label=False)
print(output)

This model was trained on the MultiNLI, Fever-NLI and Adversarial-NLI (ANLI) datasets, which comprise 763 913 NLI hypothesis-premise pairs. This base model outperforms almost all large models on the ANLI benchmark. The base model is DeBERTa-v3-base from Microsoft. The v3 variant of DeBERTa substantially outperforms previous versions of the model by including a different pre-training objective, see annex 11 of the original DeBERTa paper.


from transformers import pipeline
zershot_pipeline = pipeline(zero-shot-classification, model=svalabs/gbert-large-zeroshot-nli)
sequence = Ich habe ein Problem mit meinem Iphone das so schnell wie möglich gelöst werden muss
labels = [Computer, Handy, Tablet, dringend, nicht dringend]
hypothesis_template = In diesem Satz geht es um das Thema {}.

A German zeroshot classification model based on the German BERT large model from deepset.ai and finetuned for natural language inference using machine-translated nli sentence pairs from mnli, anli, and snli datasets.


from transformers import pipeline
translation = pipeline('translation_en_to_es', model='Helsinki-NLP/opus-mt-en-es')
translated_text = translation('Hello, how are you?')

This model is a translation model from English to Spanish using the Hugging Face Transformers library. It is based on the Marian framework and trained on the OPUS dataset. The model achieves a BLEU score of 54.9 on the Tatoeba test set.