torch
json
urllib
pytorchvideo
torchvision
torchaudio
torchtext
torcharrow
TorchData
TorchRec
TorchServe
PyTorch on XLA Devices

model = torch.hub.load('facebookresearch/pytorchvideo', 'slow_r50', pretrained=True)

The 3D ResNet model is a Resnet-style video classification network pretrained on the Kinetics 400 dataset. It is based on the architecture from the paper 'SlowFast Networks for Video Recognition' by Christoph Feichtenhofer et al.


Traffic Object Detection, Drivable Area Segmentation, Lane Detection

import torch
model = torch.hub.load('datvuthanh/hybridnets', 'hybridnets', pretrained=True)
img = torch.randn(1,3,640,384)
features, regression, classification, anchors, segmentation = model(img)

HybridNets is an end2end perception network for multi-tasks. Our work focused on traffic object detection, drivable area segmentation and lane detection. HybridNets can run real-time on embedded systems, and obtains SOTA Object Detection, Lane Detection on BDD100K Dataset.


Object Detection, Drivable Area Segmentation, Lane Detection

pip install -qr https://github.com/hustvl/YOLOP/blob/main/requirements.txt

import torch
model = torch.hub.load('hustvl/yolop', 'yolop', pretrained=True)
img = torch.randn(1,3,640,640)
det_out, da_seg_out,ll_seg_out = model(img)

YOLOP is an efficient multi-task network that can jointly handle three crucial tasks in autonomous driving: object detection, drivable area segmentation and lane detection. And it is also the first to reach real-time on embedded devices while maintaining state-of-the-art level performance on the BDD100K dataset.


Computing relative depth from a single image

url, filename = ('https://github.com/pytorch/hub/raw/master/images/dog.jpg', 'dog.jpg')

midas = torch.hub.load('intel-isl/MiDaS', 'DPT_Large')

device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')

midas_transforms = torch.hub.load('intel-isl/MiDaS', 'transforms')

if model_type == 'DPT_Large' or model_type == 'DPT_Hybrid':

img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

MiDaS computes relative inverse depth from a single image. The repository provides multiple models that cover different use cases ranging from a small, high-speed model to a very large model that provide the highest accuracy. The models have been trained on 10 distinct datasets using multi-objective optimization to ensure high quality on a wide range of inputs.


midas = torch.hub.load('intel-isl/MiDaS', 'DPT_Hybrid')


midas = torch.hub.load('intel-isl/MiDaS', 'MiDaS_small')


separator = torch.hub.load('sigsep/open-unmix-pytorch', 'umxhq')

Open-Unmix provides ready-to-use models that allow users to separate pop music into four stems: vocals, drums, bass and the remaining other instruments. The models were pre-trained on the freely available MUSDB18 dataset.


torch.hub.load(repo_or_dir='nicolalandro/ntsnet-cub200', model='ntsnet', pretrained=True, **{'topN': 6, 'device':'cpu', 'num_classes': 200})

from torchvision import transforms
import torch
import urllib
from PIL import Image

transform_test = transforms.Compose([
 transforms.Resize((600, 600), Image.BILINEAR),
 transforms.CenterCrop((448, 448)),
 transforms.ToTensor(),
 transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
])

model = torch.hub.load('nicolalandro/ntsnet-cub200', 'ntsnet', pretrained=True, **{'topN': 6, 'device':'cpu', 'num_classes': 200})
model.eval()

url = 'https://raw.githubusercontent.com/nicolalandro/ntsnet-cub200/master/images/nts-net.png'
img = Image.open(urllib.request.urlopen(url))
scaled_img = transform_test(img)
torch_images = scaled_img.unsqueeze(0)

with torch.no_grad():
 top_n_coordinates, concat_out, raw_logits, concat_logits, part_logits, top_n_index, top_n_prob = model(torch_images)

_, predict = torch.max(concat_logits, 1)
pred_id = predict.item()
print('bird class:', model.bird_classes[pred_id])

This is an nts-net pretrained with CUB200 2011 dataset, which is a fine-grained dataset of birds species.


pip install -q torchaudio omegaconf soundfile

model, decoder, utils = torch.hub.load(repo_or_dir='snakers4/silero-models', model='silero_stt', language='en', device=device)

(read_batch, split_into_batches, read_audio, prepare_model_input) = utils

batches = split_into_batches(test_files, batch_size=10)

input = prepare_model_input(read_batch(batches[0]), device=device)

See the wiki for quality and performance benchmarks

Silero Speech-To-Text models provide enterprise grade STT in a compact form-factor for several commonly spoken languages. The models are robust to a variety of dialects, codecs, domains, noises, and lower sampling rates. They consume a normalized audio in the form of samples and output frames with token probabilities. A decoder utility is provided for simplicity.


import torch
language = 'en'
speaker = 'lj_16khz'
device = torch.device('cpu')
model, symbols, sample_rate, example_text, apply_tts = torch.hub.load(repo_or_dir='snakers4/silero-models', model='silero_tts', language=language, speaker=speaker)
model = model.to(device)
audio = apply_tts(texts=[example_text], model=model, sample_rate=sample_rate, symbols=symbols, device=device)

High throughput on slow hardware. Decent performance on one CPU thread

Silero Text-To-Speech models provide enterprise grade TTS in a compact form-factor for several commonly spoken languages. They offer one-line usage, naturally sounding speech, no GPU or training required, minimalism and lack of dependencies, a library of voices in many languages, support for 16kHz and 8kHz out of the box.


import torch
torch.set_num_threads(1)
from IPython.display import Audio
from pprint import pprint

model, utils = torch.hub.load(repo_or_dir='snakers4/silero-vad', model='silero_vad', force_reload=True)

(get_speech_timestamps, _, read_audio, _) = utils

wav = read_audio('en_example.wav', sampling_rate=sampling_rate)

speech_timestamps = get_speech_timestamps(wav, model, sampling_rate=sampling_rate)

Silero VAD is a pre-trained enterprise-grade Voice Activity Detector (VAD) that aims to provide a high-quality and modern alternative to the WebRTC Voice Activity Detector. The model is optimized for performance on 1 CPU thread and is quantized.


model = torch.hub.load('pytorch/vision:v0.10.0', 'deeplabv3_resnet50', pretrained=True)

DeepLabV3 models with ResNet-50, ResNet-101 and MobileNet-V3 backbones for semantic segmentation. The pre-trained models have been trained on a subset of COCO train2017, on the 20 categories that are present in the Pascal VOC dataset.


model = torch.hub.load('pytorch/vision:v0.10.0', 'deeplabv3_resnet101', pretrained=True)


model = torch.hub.load('pytorch/vision:v0.10.0', 'deeplabv3_mobilenet_v3_large', pretrained=True)


model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True)

YOLOv5 is a family of compound-scaled object detection models trained on the COCO dataset, and includes simple functionality for Test Time Augmentation (TTA), model ensembling, hyperparameter evolution, and export to ONNX, CoreML and TFLite.


torch.hub.load(repo_or_dir='facebookresearch/pytorch_GAN_zoo:hub', model='DCGAN', pretrained=True, useGPU=use_gpu)

import torch
import matplotlib.pyplot as plt
import torchvision

use_gpu = True if torch.cuda.is_available() else False

model = torch.hub.load('facebookresearch/pytorch_GAN_zoo:hub', 'DCGAN', pretrained=True, useGPU=use_gpu)

noise, _ = model.buildNoiseData(num_images)

with torch.no_grad(): generated_images = model.test(noise)

DCGAN is a model designed in 2015 by Radford et. al. in the paper Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. It is a GAN architecture both very simple and efficient for low resolution image generation (up to 64x64).


ResNeXt-101 32x8d WSL model

model = torch.hub.load('facebookresearch/WSL-Images', 'resnext101_32x8d_wsl')

The provided ResNeXt models are pre-trained in weakly-supervised fashion on 940 million public images with 1.5K hashtags matching with 1000 ImageNet1K synsets, followed by fine-tuning on ImageNet1K dataset. The models significantly improve the training accuracy on ImageNet compared to training from scratch. They achieve state-of-the-art accuracy of 85.4% on ImageNet with the ResNext-101 32x48d model.


ResNeXt-101 32x16d WSL model

model = torch.hub.load('facebookresearch/WSL-Images', 'resnext101_32x16d_wsl')


ResNeXt-101 32x32d WSL model

model = torch.hub.load('facebookresearch/WSL-Images', 'resnext101_32x32d_wsl')


ResNeXt-101 32x48d WSL model

model = torch.hub.load('facebookresearch/WSL-Images', 'resnext101_32x48d_wsl')


model = torch.hub.load('facebookresearch/pytorch_GAN_zoo:hub', 'PGAN', model_name='celebAHQ-512', pretrained=True, useGPU=use_gpu)

grid = torchvision.utils.make_grid(generated_images.clamp(min=-1, max=1), scale_each=True, normalize=True)

Progressive Growing of GANs (PGAN) is a method for generating high-resolution images using generative adversarial networks. The model is trained progressively, starting with low-resolution images and gradually increasing the resolution until the desired output is achieved. This implementation is based on the paper by Tero Karras et al., 'Progressive Growing of GANs for Improved Quality, Stability, and Variation'.


torch.hub.load(repo_or_dir='mateuszbuda/brain-segmentation-pytorch', model='unet', in_channels=3, out_channels=1, init_features=32, pretrained=True)

model = torch.hub.load('mateuszbuda/brain-segmentation-pytorch', 'unet', in_channels=3, out_channels=1, init_features=32, pretrained=True)

m, s = np.mean(input_image, axis=(0, 1)), np.std(input_image, axis=(0, 1))

preprocess = transforms.Compose([transforms.ToTensor(),transforms.Normalize(mean=m, std=s)])

U-Net with batch normalization for biomedical image segmentation with pretrained weights for abnormality segmentation in brain MRI. The model comprises four levels of blocks containing two convolutional layers with batch normalization and ReLU activation function, and one max pooling layer in the encoding part and up-convolutional layers instead in the decoding part. The number of convolutional filters in each block is 32, 64, 128, and 256. The bottleneck layer has 512 convolutional filters. From the encoding layers, skip connections are used to the corresponding layers in the decoding part. Input image is a 3-channel brain MRI slice from pre-contrast, FLAIR, and post-contrast sequences, respectively. Output is a one-channel probability map of abnormality regions with the same size as the input image.


import torchvision.transforms as transforms

device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')

resnet50 = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_resnet50', pretrained=True)

utils = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_convnets_processing_utils')

batch = torch.cat([utils.prepare_input_from_uri(uri) for uri in uris]).to(device)

output = torch.nn.functional.softmax(resnet50(batch), dim=1)

results = utils.pick_n_best(predictions=output, n=5)

img = Image.open(requests.get(uri, stream=True).raw)

The ResNet50 v1.5 model is a modified version of the original ResNet50 v1 model. The difference between v1 and v1.5 is that, in the bottleneck blocks which requires downsampling, v1 has stride = 2 in the first 1x1 convolution, whereas v1.5 has stride = 2 in the 3x3 convolution. This difference makes ResNet50 v1.5 slightly more accurate (~0.5% top1) than v1, but comes with a small performance drawback (~5% imgs/sec). The model is initialized as described in Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. This model is trained with mixed precision using Tensor Cores on Volta, Turing, and the NVIDIA Ampere GPU architectures.


device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")

resneXt = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_resneXt')

[utils.prepare_input_from_uri(uri) for uri in uris]

output = torch.nn.functional.softmax(resneXt(batch), dim=1)

ResNeXt101-32x4d is a model introduced in the Aggregated Residual Transformations for Deep Neural Networks paper. It is based on regular ResNet model, substituting 3x3 convolutions inside the bottleneck block for 3x3 grouped convolutions. This model is trained with mixed precision using Tensor Cores on Volta, Turing, and the NVIDIA Ampere GPU architectures. It can be deployed for inference on the NVIDIA Triton Inference Server using TorchScript, ONNX Runtime or TensorRT as an execution backend.


import torch
from PIL import Image
import torchvision.transforms as transforms
import numpy as np
import json
import requests
import matplotlib.pyplot as plt
import warnings
warnings.filterwarnings('ignore')
%matplotlib inline
device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')
print(f'Using {device} for inference')
resneXt = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_se_resnext101_32x4d')
utils = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_convnets_processing_utils')
resneXt.eval().to(device)
uris = ['http://images.cocodataset.org/test-stuff2017/000000024309.jpg','http://images.cocodataset.org/test-stuff2017/000000028117.jpg','http://images.cocodataset.org/test-stuff2017/000000006149.jpg','http://images.cocodataset.org/test-stuff2017/000000004954.jpg']
batch = torch.cat([utils.prepare_input_from_uri(uri) for uri in uris]).to(device)
with torch.no_grad():
 output = torch.nn.functional.softmax(resneXt(batch), dim=1)
results = utils.pick_n_best(predictions=output, n=5)
for uri, result in zip(uris, results):
 img = Image.open(requests.get(uri, stream=True).raw)
 img.thumbnail((256,256), Image.ANTIALIAS)
 plt.imshow(img)
 plt.show()
 print(result)

The SE-ResNeXt101-32x4d is a ResNeXt101-32x4d model with added Squeeze-and-Excitation module. This model is trained with mixed precision using Tensor Cores on Volta, Turing, and the NVIDIA Ampere GPU architectures, which allows researchers to get results 3x faster than training without Tensor Cores while experiencing the benefits of mixed precision training.


model = torch.hub.load('pytorch/vision:v0.10.0', 'alexnet', pretrained=True)

AlexNet is a deep convolutional neural network that achieved a top-5 error of 15.3% in the 2012 ImageNet Large Scale Visual Recognition Challenge. The main contribution of the original paper was the depth of the model, which was computationally expensive but made feasible through the use of GPUs during training. The pretrained AlexNet model in PyTorch can be used for image classification tasks.


numpy scipy librosa unidecode inflect libsndfile1

waveglow = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_waveglow', model_math='fp32')

tacotron2 = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_tacotron2', model_math='fp32')

text = "hello world, I missed you so much"

utils = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_tts_utils')

sequences, lengths = utils.prepare_input_sequence([text])

mel, _, _ = tacotron2.infer(sequences, lengths)

The Tacotron 2 and WaveGlow model form a text-to-speech system that enables users to synthesize natural-sounding speech from raw transcripts without any additional prosody information. The Tacotron 2 model produces mel spectrograms from input text using encoder-decoder architecture. WaveGlow is a flow-based model that consumes the mel spectrograms to generate speech.


roberta = torch.hub.load('pytorch/fairseq', 'roberta.large')

all_layers = roberta.extract_features(tokens, return_all_hiddens=True)

roberta = torch.hub.load('pytorch/fairseq', 'roberta.large.mnli')

tokens = roberta.encode('Roberta is a heavily optimized version of BERT.', 'Roberta is not very optimized.')

prediction = roberta.predict('mnli', tokens).argmax().item()

tokens = roberta.encode('Roberta is a heavily optimized version of BERT.', 'Roberta is based on BERT.')

logprobs = roberta.predict('new_task', tokens)

RoBERTa is a robustly optimized version of BERT, a revolutionary self-supervised pretraining technique that learns to predict intentionally hidden (masked) sections of text. RoBERTa builds on BERT's language masking strategy and modifies key hyperparameters, including removing BERT's next-sentence pretraining objective, and training with much larger mini-batches and learning rates. RoBERTa was also trained on an order of magnitude more data than BERT, for a longer amount of time, allowing it to generalize even better to downstream tasks.


Densenet-121 model

model = torch.hub.load('pytorch/vision:v0.10.0', 'densenet121', pretrained=True)

Dense Convolutional Network (DenseNet) connects each layer to every other layer in a feed-forward fashion. It alleviates the vanishing-gradient problem, strengthens feature propagation, encourages feature reuse, and substantially reduces the number of parameters.


Densenet-169 model

model = torch.hub.load('pytorch/vision:v0.10.0', 'densenet169', pretrained=True)


Densenet-201 model

model = torch.hub.load('pytorch/vision:v0.10.0', 'densenet201', pretrained=True)


Densenet-161 model

model = torch.hub.load('pytorch/vision:v0.10.0', 'densenet161', pretrained=True)


pretrained

If True, returns a model pre-trained on COCO train2017

model = torch.hub.load('pytorch/vision:v0.10.0', 'fcn_resnet50', pretrained=True)

FCN-ResNet is a Fully-Convolutional Network model using a ResNet-50 or a ResNet-101 backbone. The pre-trained models have been trained on a subset of COCO train2017, on the 20 categories that are present in the Pascal VOC dataset.


model = torch.hub.load('pytorch/vision:v0.10.0', 'fcn_resnet101', pretrained=True)


HarDNet-39DS model

model = torch.hub.load('PingoLH/Pytorch-HarDNet', 'hardnet39ds', pretrained=True)

Harmonic DenseNet (HarDNet) is a low memory traffic CNN model, which is fast and efficient. The basic concept is to minimize both computational cost and memory access cost at the same time, such that the HarDNet models are 35% faster than ResNet running on GPU comparing to models with the same accuracy (except the two DS models that were designed for comparing with MobileNet).


HarDNet-68DS model

model = torch.hub.load('PingoLH/Pytorch-HarDNet', 'hardnet68ds', pretrained=True)


HarDNet-68 model

model = torch.hub.load('PingoLH/Pytorch-HarDNet', 'hardnet68', pretrained=True)

Harmonic DenseNet (HarDNet) is a low memory trafficCNN model, which is fast and efficient. The basic concept is to minimize both computational cost and memory access cost at the same time, such that the HarDNet models are 35% faster than ResNet running on GPU comparing to models with the same accuracy (except the two DS models that were designed for comparing with MobileNet).


HarDNet-85 model

model = torch.hub.load('PingoLH/Pytorch-HarDNet', 'hardnet85', pretrained=True)


Efficient networks by generating more features from cheap operations

model = torch.hub.load('huawei-noah/ghostnet', 'ghostnet_1x', pretrained=True)

probabilities = torch.nn.functional.softmax(output[0], dim=0)

The GhostNet architecture is based on an Ghost module structure which generates more features from cheap operations. Based on a set of intrinsic feature maps, a series of cheap operations are applied to generate many ghost feature maps that could fully reveal information underlying intrinsic features. Experiments conducted on benchmarks demonstrate the superiority of GhostNet in terms of speed and accuracy tradeoff.


import torch
import urllib
from PIL import Image
from torchvision import transforms

model = torch.hub.load('pytorch/vision:v0.10.0', 'googlenet', pretrained=True)

preprocess = transforms.Compose([transforms.Resize(256), transforms.CenterCrop(224), transforms.ToTensor(), transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])])

if torch.cuda.is_available():
input_batch = input_batch.to('cuda')
model.to('cuda')
with torch.no_grad():
output = model(input_batch)

probabilities = torch.nn.functional.softmax(output[0], dim=0)

top5_prob, top5_catid = torch.topk(probabilities, 5)

GoogLeNet is based on a deep convolutional neural network architecture codenamed 'Inception', which was responsible for setting the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC 2014).


ResNet-50-IBN-a model

model = torch.hub.load('XingangPan/IBN-Net', 'resnet50_ibn_a', pretrained=True)

IBN-Net is a CNN model with domain/appearance invariance. Motivated by style transfer works, IBN-Net carefully unifies instance normalization and batch normalization in a single deep network. It provides a simple way to increase both modeling and generalization capacities without adding model complexity. IBN-Net is especially suitable for cross domain or person/vehicle re-identification tasks.


ResNet-101-IBN-a model

model = torch.hub.load('XingangPan/IBN-Net', 'resnet101_ibn_a', pretrained=True)


ResNeXt-101-IBN-a model

model = torch.hub.load('XingangPan/IBN-Net', 'resnext101_ibn_a', pretrained=True)


SE-ResNet-101-IBN-a model

model = torch.hub.load('XingangPan/IBN-Net', 'se_resnet101_ibn_a', pretrained=True)


model = torch.hub.load('pytorch/vision:v0.10.0', 'inception_v3', pretrained=True)

Inception v3, also called GoogleNetv3, is a famous Convolutional Neural Network trained on the ImageNet dataset from 2015. It is based on the exploration of ways to scale up networks to utilize the added computation as efficiently as possible by using suitably factorized convolutions and aggressive regularization. The model achieves a top-1 error of 22.55% and a top-5 error of 6.44% on the ImageNet dataset.


model = torch.hub.load('zhanghang1989/ResNeSt', 'resnest50', pretrained=True)

ResNeSt models are from the ResNeSt: Split-Attention Networks paper. They are a new ResNet variant that enables attention across feature-map groups. By stacking Split-Attention blocks ResNet-style, ResNeSt models outperform other networks with similar model complexities, and also help downstream tasks including object detection, instance segmentation, and semantic segmentation.


model = torch.hub.load('zhanghang1989/ResNeSt', 'resnest101', pretrained=True)


model = torch.hub.load('zhanghang1989/ResNeSt', 'resnest200', pretrained=True)


model = torch.hub.load('zhanghang1989/ResNeSt', 'resnest269', pretrained=True)


ProxylessNAS optimized for CPU

model = torch.hub.load('mit-han-lab/ProxylessNAS', target_platform, pretrained=True)

ProxylessNAS models are from the ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware paper. They specialize CNN architectures for different hardware platforms, offering free yet significant performance boost on all three platforms (CPU, GPU, and Mobile) with similar accuracy.


ProxylessNAS optimized for GPU


ProxylessNAS optimized for Mobile


model = torch.hub.load('pytorch/vision', 'mobilenet_v2', pretrained=True)

The MobileNet v2 architecture is based on an inverted residual structure where the input and output of the residual block are thin bottleneck layers opposite to traditional residual models which use expanded representations in the input. MobileNet v2 uses lightweight depthwise convolutions to filter features in the intermediate expansion layer. Additionally, non-linearities in the narrow layers were removed in order to maintain representational power.


If True, returns a model pre-trained on ImageNet

model = torch.hub.load('pytorch/vision:v0.10.0', 'resnet18', pretrained=True)

ResNet models are deep residual networks pre-trained on ImageNet. They were proposed in the paper 'Deep Residual Learning for Image Recognition'. Available model variants include ResNet18, ResNet34, ResNet50, ResNet101, and ResNet152.


model = torch.hub.load('pytorch/vision:v0.10.0', 'resnet34', pretrained=True)


model = torch.hub.load('pytorch/vision:v0.10.0', 'resnet50', pretrained=True)


model = torch.hub.load('pytorch/vision:v0.10.0', 'resnet101', pretrained=True)


model = torch.hub.load('pytorch/vision:v0.10.0', 'resnet152', pretrained=True)


model = torch.hub.load('pytorch/vision:v0.10.0', 'resnext50_32x4d', pretrained=True)

probabilities = torch.nn.functional.softmax(output[0], dim=0)

ResNext is a next-generation ResNet architecture for image classification. It is more efficient and accurate than the original ResNet. This implementation includes two versions of the model, resnext50_32x4d and resnext101_32x8d, with 50 and 101 layers respectively.


model = torch.hub.load('pytorch/vision:v0.10.0', 'resnext101_32x4d', pretrained=True)


SNNMLP Tiny model

model = torch.hub.load('huawei-noah/Efficient-AI-Backbones', 'snnmlp_t', pretrained=True)

SNNMLP incorporates the mechanism of LIF neurons into the MLP models, to achieve better accuracy without extra FLOPs. We propose a full-precision LIF operation to communicate between patches, including horizontal LIF and vertical LIF in different directions. We also propose to use group LIF to extract better local features. With LIF modules, our SNNMLP model achieves 81.9%, 83.3% and 83.6% top-1 accuracy on ImageNet dataset with only 4.4G, 8.5G and 15.2G FLOPs, respectively.


SNNMLP Small model

model = torch.hub.load('huawei-noah/Efficient-AI-Backbones', 'snnmlp_s', pretrained=True)


SNNMLP Base model

model = torch.hub.load('huawei-noah/Efficient-AI-Backbones', 'snnmlp_b', pretrained=True)


model = torch.hub.load('pytorch/vision:v0.10.0', 'squeezenet1_0', pretrained=True)

SqueezeNet is an image classification model that achieves AlexNet-level accuracy with 50x fewer parameters. It has two versions: squeezenet1_0 and squeezenet1_1, with squeezenet1_1 having 2.4x less computation and slightly fewer parameters than squeezenet1_0, without sacrificing accuracy.


model = torch.hub.load('pytorch/vision:v0.10.0', 'squeezenet1_1', pretrained=True)


import torch
from PIL import Image
from torchvision import transforms
import urllib

model = torch.hub.load('pytorch/vision:v0.10.0', 'shufflenet_v2_x1_0', pretrained=True)

url, filename = ('https://github.com/pytorch/hub/raw/master/images/dog.jpg', 'dog.jpg')

if torch.cuda.is_available():
input_batch = input_batch.to('cuda')
model.to('cuda')
with torch.no_grad():
output = model(input_batch)

probabilities = torch.nn.functional.softmax(output[0], dim=0)

top5_prob, top5_catid = torch.topk(probabilities, 5)

ShuffleNet V2 is an efficient ConvNet optimized for speed and memory, pre-trained on Imagenet. It is designed based on practical guidelines for efficient network design, including speed and accuracy tradeoff.


VGG11 model

model = torch.hub.load('pytorch/vision:v0.10.0', 'vgg11', pretrained=True)

vgg-nets are award-winning ConvNets from the 2014 Imagenet ILSVRC challenge. They are used for large-scale image recognition tasks. The available models are vgg11, vgg11_bn, vgg13, vgg13_bn, vgg16, vgg16_bn, vgg19, and vgg19_bn.


VGG11 model with batch normalization

model = torch.hub.load('pytorch/vision:v0.10.0', 'vgg11_bn', pretrained=True)


VGG13 model

model = torch.hub.load('pytorch/vision:v0.10.0', 'vgg13', pretrained=True)

vgg-nets are award-winning ConvNets from the 2014 Imagenet ILSVRC challenge. They are used for large-scale image recognition tasks. The available models are vgg11, vgg11_bn, vgg13, vgg13_bn,vgg16, vgg16_bn, vgg19, and vgg19_bn.


VGG13 model with batch normalization

model = torch.hub.load('pytorch/vision:v0.10.0', 'vgg13_bn', pretrained=True)


VGG16 model

model = torch.hub.load('pytorch/vision:v0.10.0', 'vgg16', pretrained=True)


VGG16 model with batch normalization

model = torch.hub.load('pytorch/vision:v0.10.0', 'vgg16_bn', pretrained=True)


VGG19 model

model = torch.hub.load('pytorch/vision:v0.10.0', 'vgg19', pretrained=True)


VGG19 model with batch normalization

model = torch.hub.load('pytorch/vision:v0.10.0', 'vgg19_bn', pretrained=True)


model = torch.hub.load('pytorch/vision:v0.10.0', 'wide_resnet50_2', pretrained=True)

Wide Residual networks simply have increased number of channels compared to ResNet. Otherwise the architecture is the same. Deeper ImageNet models with bottleneck block have increased number of channels in the inner 3x3 convolution. The wide_resnet50_2 and wide_resnet101_2 models were trained in FP16 with mixed precision training using SGD with warm restarts. Checkpoints have weights in half precision (except batch norm) for smaller size, and can be used in FP32 models too.


model = torch.hub.load('pytorch/vision:v0.10.0', 'wide_resnet101_2', pretrained=True)


model = torch.hub.load('facebookresearch/pytorchvideo', 'slowfast_r50', pretrained=True)

Slow Fast model architectures are based on the paper 'SlowFast Networks for Video Recognition' by Christoph Feichtenhofer et al. They are pretrained on the Kinetics 400 dataset using the 8x8 setting. This model is capable of classifying video clips into different action categories. It is provided by the FAIR PyTorchVideo library.


model = torch.hub.load('facebookresearch/pytorchvideo', 'x3d_s', pretrained=True)

X3D model architectures are based on the paper 'X3D: Expanding Architectures for Efficient Video Recognition' by Christoph Feichtenhofer. They are pretrained on the Kinetics 400 dataset. This model is capable of classifying video clips into different action categories. It is provided by the FAIR PyTorchVideo library.


torch.hub.load(repo_or_dir='NVIDIA/DeepLearningExamples:torchhub', model='nvidia_gpunet', pretrained=True, model_type=model_type, model_math=precision)

gpunet = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_gpunet', pretrained=True, model_type=model_type, model_math=precision)

device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')

GPUNet demonstrates state-of-the-art inference performance up to 2x faster than EfficientNet-X and FBNet-V3.

GPUNet is a family of Convolutional Neural Networks designed by NVIDIA using novel Neural Architecture Search (NAS) methods. They are optimized for NVIDIA GPU and TensorRT performance. GPUNet models are pretrained on the IMAGENET dataset and are capable of classifying images into different categories. The models are provided by the NVIDIA Deep Learning Examples library.


super_net = torch.hub.load('mit-han-lab/once-for-all', super_net_name, pretrained=True).eval()

OFA networks outperform state-of-the-art NAS methods (up to 4.0% ImageNet top1 accuracy improvement over MobileNetV3, or same accuracy but 1.5x faster than MobileNetV3, 2.6x faster than EfficientNet w.r.t measured latency) while reducing many orders of magnitude GPU hours and CO2 emission.

Once-for-all (OFA) networks are a family of neural networks designed by MIT Han Lab. They decouple training and search, achieving efficient inference across various edge devices and resource constraints. OFA networks are pretrained on the IMAGENET dataset and are capable of classifying images into different categories.


model = torch.hub.load('facebookresearch/semi-supervised-ImageNet1K-models', 'resnet18_swsl', pretrained=True)

Semi-supervised and semi-weakly supervised ImageNet models achieve state-of-the-art accuracy of 81.2% on ImageNet for the widely used/adopted ResNet-50 model architecture.

Semi-supervised and semi-weakly supervised ImageNet Models are introduced in the 'Billion scale semi-supervised learning for image classification' paper. These models are pretrained on a subset of unlabeled YFCC100M public image dataset and fine-tuned with the ImageNet1K training dataset. They are capable of classifying images into different categories and are provided by the Facebook Research library.


tqdm boto3 requests regex sentencepiece sacremoses

import torch
tokenizer = torch.hub.load('huggingface/pytorch-transformers', 'tokenizer', 'bert-base-cased')

text_1 = "Jim Henson was a puppeteer"
text_2 = "Who was Jim Henson ?"

indexed_tokens = tokenizer.encode(text_1, text_2, add_special_tokens=True)

model = torch.hub.load('huggingface/pytorch-transformers', 'model', 'bert-base-cased')

with torch.no_grad():
 encoded_layers, _ = model(tokens_tensor, token_type_ids=segments_tensors)

PyTorch-Transformers is a library of state-of-the-art pre-trained models for Natural Language Processing (NLP) including BERT, GPT, GPT-2, Transformer-XL, XLNet, XLM, RoBERTa, and DistilBERT. The library provides functionality for tokenization, configuration, and various model architectures for different tasks such as causal language modeling, sequence classification, question answering, and masked language modeling.


model_name tokenizer bpe beam sampling sampling_topk

bitarray fastBPE hydra-core omegaconf regex requests sacremoses subword_nmt

import torch

en2fr = torch.hub.load('pytorch/fairseq', 'transformer.wmt14.en-fr', tokenizer='moses', bpe='subword_nmt')

en2fr.cuda()

fr = en2fr.translate('Hello world!', beam=5)
assert fr == 'Bonjour à tous !'

Transformer (NMT) is a powerful sequence-to-sequence modeling architecture that produces state-of-the-art neural machine translation systems. It is based on the paper 'Attention Is All You Need' and has been improved using techniques such as large-scale semi-supervised training, back-translation, and noisy-channel reranking. It supports English-French and English-German translation as well as round-trip translation for paraphrasing.


import torch

efficientnet = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_efficientnet_b0', pretrained=True)
utils = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_convnets_processing_utils')

efficientnet.eval().to(device)

batch = torch.cat([utils.prepare_input_from_uri(uri) for uri in uris]).to(device)

with torch.no_grad():
 output = torch.nn.functional.softmax(efficientnet(batch), dim=1)

results = utils.pick_n_best(predictions=output, n=5)

EfficientNet is a family of image classification models that achieve state-of-the-art accuracy while being smaller and faster. The models are trained with mixed precision using Tensor Cores on the NVIDIA Volta and Ampere GPU architectures. The EfficientNet models include EfficientNet-B0, EfficientNet-B4, EfficientNet-WideSE-B0, and EfficientNet-WideSE-B4. The WideSE models use wider Squeeze-and-Excitation layers than the original EfficientNet models, resulting in slightly better accuracy.


import torch

ssd_model = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_ssd')
utils = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_ssd_processing_utils')

ssd_model.to('cuda')
ssd_model.eval()

inputs = [utils.prepare_input(uri) for uri in uris]
tensor = utils.prepare_tensor(inputs)

with torch.no_grad():
 detections_batch = ssd_model(tensor)

results_per_input = utils.decode_results(detections_batch)
best_results_per_input = [utils.pick_best(results, 0.40) for results in results_per_input]

The SSD (Single Shot MultiBox Detector) model is an object detection model based on the paper 'SSD: Single Shot MultiBox Detector'. It uses a deep neural network for detecting objects in images. This implementation replaces the obsolete VGG model backbone with the more modern ResNet-50 model. The SSD model is trained on the COCO dataset and can be used to detect objects in images with high accuracy and efficiency.


tacotron2 = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_tacotron2', model_math='fp16')

text = 'Hello world, I missed you so much.'

utils = torch.hub.load('NVIDIA/DeepLearningExamples:torchhub', 'nvidia_tts_utils')

sequences, lengths = utils.prepare_input_sequence([text])

mel, _, _ = tacotron2.infer(sequences, lengths)

The Tacotron 2 model generates mel spectrograms from input text using an encoder-decoder architecture, and it is designed for generating natural-sounding speech from raw transcripts without any additional prosody information. This implementation uses Dropout instead of Zoneout to regularize the LSTM layers. The WaveGlow model (also available via torch.hub) is a flow-based model that consumes the mel spectrograms to generate speech.


import torch
from PIL import Image
from torchvision import transforms

model = torch.hub.load('szq0214/MEAL-V2','meal_v2', 'mealv2_resnest50_cutmix', pretrained=True)
model.eval()

input_image = Image.open('dog.jpg')
preprocess = transforms.Compose([
 transforms.Resize(256),
 transforms.CenterCrop(224),
 transforms.ToTensor(),
 transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
input_tensor = preprocess(input_image)
input_batch = input_tensor.unsqueeze(0)

if torch.cuda.is_available():
 input_batch = input_batch.to('cuda')
 model.to('cuda')

with torch.no_grad():
 output = model(input_batch)
probabilities = torch.nn.functional.softmax(output[0], dim=0)
print(probabilities)

MEAL V2 models are from the MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tricks paper. The method is based on ensemble knowledge distillation via discriminators, and it achieves state-of-the-art results without using common tricks such as architecture modification, outside training data, autoaug/randaug, cosine learning rate, mixup/cutmix training, or label smoothing.


torch.hub.load(repo_or_dir='szq0214/MEAL-V2', 'meal_v2', model='mealv2_resnest50', pretrained=True)


torch.hub.load(repo_or_dir='szq0214/MEAL-V2', 'meal_v2', model='mealv2_resnest50_cutmix', pretrained=True)


torch.hub.load(repo_or_dir='szq0214/MEAL-V2', 'meal_v2', model='mealv2_resnest50_380x380', pretrained=True)


torch.hub.load(repo_or_dir='szq0214/MEAL-V2', 'meal_v2', model='mealv2_mobilenetv3_small_075', pretrained=True)


torch.hub.load(repo_or_dir='szq0214/MEAL-V2', 'meal_v2', model='mealv2_mobilenetv3_small_100', pretrained=True)


torch.hub.load(repo_or_dir='szq0214/MEAL-V2', 'meal_v2', model='mealv2_mobilenet_v3_large_100', pretrained=True)


torch.hub.load(repo_or_dir='szq0214/MEAL-V2', 'meal_v2', model='mealv2_efficientnet_b0', pretrained=True)