User Guide

This guide covers the full Embedl Deploy optimization pipeline — from graph conversions through operator fusions to INT8 quantization — with working examples on ResNet50, ConvNeXt, and Vision Transformer (ViT).

• Optimization Pipeline
  • Pipeline stages
  • One-shot API
  • Plan-based API
  • Pattern priority
  • Pattern groups
  • Verifying numerical equivalence
  • ONNX export and compilation
• Graph Conversions
  • Built-in TensorRT conversions
  • When conversions matter
  • Running conversions only
• Operator Fusions
  • Convolution fusions
  • Linear fusions
  • Attention fusions
  • Pooling fusions
  • Fusion summary by architecture
  • Running fusions only
• Quantization
  • Quantization pipeline
  • Step 1: Transform and fuse
  • Step 2: Insert QDQ stubs
  • Step 3: Calibrate
  • QDQ point placement
  • Why pattern-aware QDQ matters
  • Full example: ResNet50 INT8 PTQ
  • Quantization-Aware Training (QAT)
• Custom Patterns
  • Why customize?
  • Writing a custom pattern
  • Building a custom pattern list
  • Using the custom pattern list
  • Impact on benchmarks
• Benchmark Results
  • Test setup
  • ResNet50
  • ConvNeXt
  • Vision Transformer (ViT-B/16)
  • Summary
  • Reproducing these results