Processor SDK Linux Edge AI Software Developer’s Guide

Welcome to the AM62D EdgeAI Software Developer’s Guide

• 1. Overview
• 2. Hardware Setup and Configuration
  • 2.1. Hardware setup
    • 2.1.1. AM62D EVM
    • 2.1.2. Boot Mode Configuration
    • 2.1.3. Audio Connectors
  • 2.2. Software setup
    • 2.2.1. Preparing SD Card Image
    • 2.2.2. Power ON and Boot
    • 2.2.3. UART Serial Console
    • 2.2.4. Connect Remotely
    • 2.2.5. Verifying the System
      • 2.2.5.1. Check DSP state (should report ‘running’):
      • 2.2.5.2. List RPMsg devices:
      • 2.2.5.3. List DMA heap devices:
• 3. Out-of-Box Web GUI Demo
  • 3.1. OOB Web GUI Home Screen
  • 3.2. Speech Enhancement Demo
  • 3.3. Audio Classification Demo
  • 3.4. Custom Audio GStreamer Pipeline
    • 3.4.1. Speech Enhancement Pipeline with GCRN
    • 3.4.2. Audio Classification with YAMNet
    • 3.4.3. Audio Classification with VGGish
• 4. Model Artifacts and Setup
  • 4.1. Pre-Packaged Models
  • 4.2. Model Artifacts
  • 4.3. Label Files
  • 4.4. Input Audio
  • 4.5. Pre-Packaged Libraries
  • 4.6. Model Daemon
  • 4.7. DSP Firmware Check
• 5. C++ Inference Application
  • 5.1. Introduction
  • 5.2. Supported Pipelines
  • 5.3. Prerequisites
  • 5.4. Building on AM62D EVM
  • 5.5. Building on Host
    • 5.5.1. Basic Syntax
  • 5.6. GCRN based Speech Enhancement
    • 5.6.1. Speech Enhancement Pipeline using GCRN
  • 5.7. YAMNet & VGGish based Audio Classification
    • 5.7.1. Audio Classification pipeline using YAMNet
    • 5.7.2. Audio Classification Pipeline using VGGish
• 6. GStreamer Audio Pipelines
  • 6.1. Introduction
  • 6.2. Supported Pipelines
  • 6.3. Prerequisites
    • 6.3.1. Native Compilation & Installation on AM62D EVM
    • 6.3.2. Verifying the Installation
  • 6.4. GCRN based Speech Enhancement
    • 6.4.1. Speech Enhancement Pipeline using GCRN
  • 6.5. YAMNet & VGGish based Audio Classification
    • 6.5.1. Audio Classification Pipeline using YAMNet
      • 6.5.1.1. YAMNet Command Output
    • 6.5.2. Audio Classification Pipeline using VGGish
      • 6.5.2.1. VGGish Command Output
  • 6.6. Troubleshooting
• 7. EdgeAI Model Workflow
  • 7.1. Development workflow
  • 7.2. Reference models
    • 7.2.1. Audio analysis and classification
    • 7.2.2. Speech enhancement
  • 7.3. TVM + TIDL framework
• 8. EdgeAI Model Zoo
  • 8.1. What ships
  • 8.2. On-target setup
  • 8.3. Inference script demos
    • 8.3.1. YAMNet (sound classification)
    • 8.3.2. VGGish11 (sound classification)
    • 8.3.3. GCRN (speech enhancement)
  • 8.4. Jupyter notebook demos
  • 8.5. Compiled artifacts
• 9. Model Compile and Evaluate
  • 9.1. edgeai-tidlrunner
  • 9.2. Setup
    • 9.2.1. Data Sets
  • 9.3. Compile and evaluate
    • 9.3.1. VGGish11 (TVM-RT, 8-bit)
    • 9.3.2. YAMNet (TVM-RT, 8-bit)
    • 9.3.3. GCRN (TVM-RT, 16-bit)
• 10. Model Inspect
  • 10.1. Model Inspector
    • 10.1.1. GCRN showcase - an offloaded DSP subgraph
• 11. Integrating a New Model
  • 11.1. Overview
  • 11.2. Compile the Model
    • 11.2.1. Workflow Overview
    • 11.2.2. Compilation Artifacts
    • 11.2.3. Key Compilation Options
  • 11.3. DSP Firmware Changes
    • 11.3.1. Files to modify or create
    • 11.3.2. Key functions to implement in stft_signal_chain_<model>.cpp
    • 11.3.3. Two model types are supported
  • 11.4. Linux Application
    • 11.4.1. Using a New Model with an Existing Pipeline Type
    • 11.4.2. Adding a New Pipeline Type in C++
    • 11.4.3. Debugging and Troubleshooting
  • 11.5. GStreamer
    • 11.5.1. Bring-Your-Own Model
      • 11.5.1.1. Requirements
      • 11.5.1.2. Example: A New Model
    • 11.5.2. Supporting New Model Architectures in the Plugins
      • 11.5.2.1. Adding a New GStreamer Property to tidspkernel
      • 11.5.2.2. Adding a New Property to titvm
      • 11.5.2.3. Automatic Model Detection from Path
      • 11.5.2.4. Building the Modified Plugins
        • 11.5.2.4.1. Native Build on the Board
• 12. Performance Benchmark
  • 12.1. Methodology
  • 12.2. Per-model results
  • 12.3. Scope & Limitations

For technical support please post your questions at https://e2e.ti.com.