embedl_deploy.tensorrt package#

Subpackages:

Module contents:

TensorRT backend — curated pattern lists and convenience API.

Quick start:

import torch
from torchvision.models import resnet50
from embedl_deploy import transform
from embedl_deploy.tensorrt import TENSORRT_PATTERNS

model = resnet50(weights=None).eval()
args = (torch.randn(1, 3, 224, 224),)
deployed = transform(model, args, patterns=TENSORRT_PATTERNS).model

Pattern lists#

TENSORRT_PATTERNS

Conversions (structural transforms) and fusions (Conv→BN→ReLU, etc.). This is the recommended list for most users. Recomposition is handled automatically by transform() during tracing — you do not need to include recomposition patterns.

TENSORRT_RECOMPOSITION_PATTERNS

Recomposition-only patterns. Lift aten-level ops back into nn.Module nodes (for torch.export output). Kept for backwards compatibility; recomposition is now applied automatically during tracing.

TENSORRT_CONVERSION_PATTERNS

Structural conversions applied before fusion (e.g. Flatten→Linear → Conv1×1→Flatten). Already included in TENSORRT_PATTERNS.

TENSORRT_FUSION_PATTERNS

Fusion-only patterns (Conv→BN→ReLU, Stem, residual, etc.). Already included in TENSORRT_PATTERNS.