embedl_deploy.tensorrt package#

Subpackages:

Module contents:

TensorRT backend — curated pattern lists and convenience API.

Quick start:

import torch
from torchvision.models import resnet50
from embedl_deploy import transform
from embedl_deploy.tensorrt import TENSORRT_PATTERNS

model = resnet50(weights=None).eval()
deployed = transform(model, patterns=TENSORRT_PATTERNS).model

Pattern lists#

TENSORRT_PATTERNS

Complete transformation pipeline: recomposition (lifting aten-level ops to nn.Module nodes), conversions (structural transforms), and fusions (Conv→BN→ReLU, etc.). This is the recommended list for most users and handles torch.export output correctly.

TENSORRT_RECOMPOSITION_PATTERNS

Recomposition-only patterns. Lift aten-level ops back into nn.Module nodes (for torch.export output). Already included in TENSORRT_PATTERNS.

TENSORRT_CONVERSION_PATTERNS

Structural conversions applied before fusion (e.g. Flatten→Linear → Conv1×1→Flatten). Already included in TENSORRT_PATTERNS.

TENSORRT_FUSION_PATTERNS

Fusion-only patterns (Conv→BN→ReLU, Stem, residual, etc.). Already included in TENSORRT_PATTERNS.

TENSORRT_QUANTIZED_PATTERNS

Q/DQ stub insertion, propagation, deduplication, and surround.