diff --git a/examples/ultralytics-yolo/README.md b/examples/ultralytics-yolo/README.md index 37f2e91c4a..6ed18ddb50 100644 --- a/examples/ultralytics-yolo/README.md +++ b/examples/ultralytics-yolo/README.md @@ -60,10 +60,10 @@ run the following examples. to run inferences on images, videos, or webcam streams. For a full list of options `python annotate.py -h`. -To run pruned-quantized YOLOv3 on a local webcam run: +To run pruned-quantized YOLOv5s on a local webcam run: ```bash python annotate.py \ - zoo:cv/detection/yolo_v3-spp/pytorch/ultralytics/coco/pruned_quant-aggressive_94 \ + zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned_quant-aggressive_94 \ --source 0 \ --quantized-inputs \ --image-shape 416 416 \ @@ -79,7 +79,7 @@ corresponding webcam is available, an exception will be raised. `benchmark.py` is a script for benchmarking sparsified and quantized YOLO performance with DeepSparse. For a full list of options run `python benchmark.py -h`. -To run a benchmark run: +To run a YOLOv3 pruned-quantized benchmark run: ```bash python benchmark.py \ zoo:cv/detection/yolo_v3-spp/pytorch/ultralytics/coco/pruned_quant-aggressive_94 \ @@ -87,6 +87,14 @@ python benchmark.py \ --quantized-inputs ``` +To run a YOLOv5s pruned-quantized benchmark run: +```bash +python benchmark.py \ + zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned_quant-aggressive_94 \ + --batch-size 1 \ + --quantized-inputs +``` + Note for quantized performance, your CPU must support VNNI instructions. Review `/proc/cpuinfo` for the flag `avx512_vnni` to verify chipset compatibility. @@ -106,13 +114,20 @@ The client can make requests into the server returning object detection results First, start up the host `server.py` with your model of choice, SparseZoo stubs are also supported. -Example command: +Example YOLOv3 Pruned Quantized command: ```bash python server.py \ zoo:cv/detection/yolo_v3-spp/pytorch/ultralytics/coco/pruned_quant-aggressive_94 \ --quantized-inputs ``` +Example YOLOv5s Pruned Quantized command: +```bash +python server.py \ + zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned_quant-aggressive_94 \ + --quantized-inputs +``` + You can leave that running as a detached process or in a spare terminal. This starts a Flask app with the DeepSparse Engine as the inference backend, accessible at `http://0.0.0.0:5543` by default. diff --git a/examples/ultralytics-yolo/annotate.py b/examples/ultralytics-yolo/annotate.py index 0b47eb0e04..f558e4a85b 100644 --- a/examples/ultralytics-yolo/annotate.py +++ b/examples/ultralytics-yolo/annotate.py @@ -92,9 +92,9 @@ --image-shape 416 416 ########## -Example command for running video annotations with pruned quantized YOLOv3: +Example command for running video annotations with pruned YOLOv5l: python annotate.py \ - zoo:cv/detection/yolo_v3-spp/pytorch/ultralytics/coco/pruned-aggressive_97 \ + zoo:cv/detection/yolov5-l/pytorch/ultralytics/coco/pruned-aggressive_98 \ --source my_video.mp4 \ --image-shape 416 416 diff --git a/examples/ultralytics-yolo/benchmark.py b/examples/ultralytics-yolo/benchmark.py index 5524bfced6..eb814e2aa3 100644 --- a/examples/ultralytics-yolo/benchmark.py +++ b/examples/ultralytics-yolo/benchmark.py @@ -83,9 +83,9 @@ YOLOv3/YOLOv5 anchors ########## -Example command for running a benchmark on a pruned quantized YOLOv3: +Example command for running a benchmark on a pruned quantized YOLOv5s: python benchmark.py \ - zoo:cv/detection/yolo_v3-spp/pytorch/ultralytics/coco/pruned_quant-aggressive_94 \ + zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned_quant-aggressive_94 \ --batch-size 32 \ --quantized-inputs @@ -99,9 +99,9 @@ --half-precision ########## -Example for benchmarking on a local YOLOv3 ONNX with onnxruntime: +Example for benchmarking on a local YOLOv5l ONNX with onnxruntime: python benchmark.py \ - /PATH/TO/yolov3-spp.onnx \ + /PATH/TO/yolov5l.onnx \ --engine onnxruntime \ --batch-size 32 \ @@ -112,7 +112,19 @@ * Pruned YOLOv3 (87% sparse) - "zoo:cv/detection/yolo_v3-spp/pytorch/ultralytics/coco/pruned-aggressive-97" * Pruned-Quantized YOLOv3 (83% sparse, CPU must support VNNI) - - "zoo:cv/detection/yolo_v3-spp/pytorch/ultralytics/coco/pruned_quant-aggressive-94" + "zoo:cv/detection/yolo_v3-spp/pytorch/ultralytics/coco/pruned_quant-aggressive-94" +* Baseline dense YOLOv5l - + "zoo:cv/detection/yolov5-l/pytorch/ultralytics/coco/base-none" +* Pruned YOLOv5l (86.3% sparse) - + "zoo:cv/detection/yolov5-l/pytorch/ultralytics/coco/pruned-aggressive_98" +* Pruned-Quantized YOLOv5l (79.6% sparse, CPU must support VNNI) - + "zoo:cv/detection/yolov5-l/pytorch/ultralytics/coco/pruned_quant-aggressive_95" +* Baseline dense YOLOv5s - + "zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/base-none" +* Pruned YOLOv5s (75.6% sparse) - + "zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned-aggressive_96" +* Pruned-Quantized YOLOv5s (68.2% sparse, CPU must support VNNI) - + "zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned_quant-aggressive_94" """ diff --git a/examples/ultralytics-yolo/server.py b/examples/ultralytics-yolo/server.py index 11041ee9a0..eceae0d025 100644 --- a/examples/ultralytics-yolo/server.py +++ b/examples/ultralytics-yolo/server.py @@ -47,9 +47,14 @@ YOLOv3/YOLOv5 anchors ########## -Example command for running: +Example command for running from a local YOLOv3 file: python server.py \ - ~/models/yolo-v3-pruned_quant.onnx + ~/models/yolo-v3-pruned.onnx + +########## +Example command for running from a pruned YOLOv5s file: +python server.py \ + zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned-aggressive_96 """ import argparse