7. EdgeAI Model Workflow

This guide covers the workflow for audio deep-learning (DL) models on the AM62Dx platform, using Processor SDK Linux EdgeAI AM62Dx 12.1. It targets embedded developers. You can run, compile, evaluate, and benchmark audio models on the AM62D. Run pre-compiled artifacts directly on the target. Or use the 64-bit x86 compile-and-evaluate workflow.

Four reference models serve as worked examples throughout the guide. The workflow is not limited to them: the same compile-evaluate-inspect flow is a bring-your-own-model (BYOM) flow that accepts any supported ONNX model. The guide will add more reference models over time.

The remaining sections cover:

  • EdgeAI Model Zoo - out-of-box demos on the AM62D target (inference scripts and Jupyter notebooks) using pre-trained models and pre-compiled artifacts.

  • Model Compile and Evaluate - the x86 BYOM workflow built on edgeai-tidlrunner: compile a model for the AM62D and evaluate accuracy on a data set.

  • Model Inspect - analyze the compiled subgraphs and per-layer behavior with Model Inspector.

  • Performance benchmark - latency and memory method with per-model tables.

7.1. Development workflow

The end-to-end path is the same for a reference model or your own model:

  1. Obtain an ONNX model - a reference graph shipped with the EdgeAI ModelZoo, or your own exported ONNX.

  2. Compile it for the C7™ Neural Processing Unit (NPU) on an x86 host (Model Compile and Evaluate).

  3. Evaluate accuracy or audio-quality metrics on a data set.

  4. Inspect the compiled subgraphs and per-layer analysis (Model Inspect).

  5. Run on the AM62D target using the compiled artifact (EdgeAI Model Zoo).

  6. Benchmark latency and memory (Performance).

7.2. Reference models

Four reference models cover two task families. Each is a worked example of the workflow, delivered as an ONNX graph. Three of the four run on the C7™ NPU through the TVM runtime (TVM-RT).

7.2.1. Audio analysis and classification

VGGish11 (model ID ac-0101)

Audio-clip classification. Input is a log-mel patch of shape (1, 1, 64, 126); output is (1, 10) class logits plus a (1, 512) embedding. Runs at 8-bit, fully offloaded to the C7™ NPU via TVM-RT (a single TIDL subgraph).

YAMNet (model ID ac-0201)

Audio-event classification. Input is a log-mel patch of shape (1, 1, 96, 64); output is (1, 521) class scores. Runs at 8-bit, fully offloaded to the C7™ NPU via TVM-RT (a single TIDL subgraph).

Both classification models are evaluated on the UrbanSound8K data set.

7.2.2. Speech enhancement

GCRN (model ID ase-0201)

Convolutional-recurrent speech enhancement (noise suppression) over a fixed 4-second window. Input and output are complex spectrograms of shape (1, 2, 401, 161). Runs at 16-bit and is offloaded to the C7™ NPU via TVM-RT (tidl_offload: true).

Both speech-enhancement models are evaluated on the VoiceBank-DEMAND-16k data set.

7.3. TVM + TIDL framework

The AM62D audio models compile and deploy with TI TVM. This is TI’s fork of the Apache TVM deep-learning compiler. The target accelerator is the C7™ NPU - TI’s Neural Processing Unit. It combines the C7x DSP with the Matrix Multiplication Accelerator (MMA). Older documents call this combination C7x+MMA.

TI TVM adds TIDL (TI Deep Learning) as an accelerator backend. TIDL offloads supported layers to the MMA. TVM generates C7x DSP code for any layers that are left. The MMA and the C7x DSP together form the C7™ NPU. The result is one deployable artifact. The entire model runs on the C7™ NPU.

Compilation runs on an x86 host in three stages:

  1. Model import - TVM imports the ONNX graph.

  2. Partitioning - the TIDL import API splits the graph into TIDL-supported subgraphs and the rest. The TIDL network compiler builds the subgraphs. TVM generates C7x DSP code for the rest.

  3. Artifact generation - TVM packages both parts into one TVM-RT deployable artifact (deploy_lib.so + deploy_graph.json + deploy_param.params).

At run time, the AM62D TVM runtime loads this artifact. It then sends the offloaded subgraphs to TIDL-RT on the C7™ NPU.

../../../_images/tvm_tidl_flow.svg

Fig. 7.1 TVM + TIDL compile-and-deploy flow for the AM62D audio models. Green blocks are handled by TIDL; white blocks are handled by TVM.