Skip to content

Commit

Permalink
恢复语音demo,调整cmsis的目录结构
Browse files Browse the repository at this point in the history
  • Loading branch information
QingChuanWS committed Jan 29, 2021
1 parent 075d266 commit 81d41d9
Show file tree
Hide file tree
Showing 120 changed files with 3,118 additions and 26,436 deletions.
65 changes: 0 additions & 65 deletions .vscode/settings.json

This file was deleted.

35 changes: 20 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ Tensorflow Lite Micro软件包(简称TFLu)是针对RT-Thread实时操作系统
| 名称 | 说明 |
| ---- | :--- |
| docs | 文档目录 |
| examples | Tensorflow Lite Micro示例 |
| third_party | Tensorflow Lite Micro依赖的第三方库 |
| tensorflow | Google Tensorflow Lite Micro推理框架 |
| examples | Tensorflow Lite Micro demo |
| third_party | Tensorflow Lite Micro 依赖的第三方库 |
| tensorflow | Tensorflow Lite Micro 推理框架 |

### 1.2 许可证

Expand Down Expand Up @@ -63,38 +63,43 @@ RT-Thread online packages
其中, Select Offical Example 中有两个选项:

```
(X) Enable Tensorflow Lite Micro audio example
( ) No Tensorflow Lite Micro example
( ) Enable Tensorflow Lite Micro audio example
(X) No Tensorflow Lite Micro example
```

注 : audio example 是执行官方携带的语音示例, No example 则是不集成 example 文件, 只使用 Tensorflow Lite Micro 标准框架.
注 : audio example 是执行官方携带的语音示例, No example 则是不集成 example 文件, 只使用 Tensorflow Lite Micro 标准框架. (注:默认不开启语音实例)

- 如果选择了语音示例, 请将example文件夹下的audio_main.cc文件拷贝到工程的Application目录中, 然后编译, 烧录/下载查看效果了

Select Tensorflow Lite Operations Type 中有两个选项:

```
(X) Using Tensorflow Lite reference operations
( ) Using Tensorflow Lite CMSIS NN operations
( ) Using Tensorflow Lite reference operations
(X) Using Tensorflow Lite CMSIS NN operations
```

注 : reference operation 是应用 TFLMicro 的通用算子(算子与平台隔离,可移植性好), CMSIS NN operations 是应用CMSIS库对具有ARM内核的平台进行算子的加速. **有关注意事项请参照第四部分!!**
- reference operation 对应的是 TFLMicro 的通用算子库,其中的各类算子实现与硬件平台隔离,具备良好的可移植性,但是运行速度较慢。

- Tensorflow Lite Micro整个框架功能较为复杂, 各类API比较多, 请先参考文档中[introduction.md](introduction.md), 然后通过 [user-guide.md](user-guide.md) 来了解基本的深度学习端测部署流程. 在有了以上基础之后, 就可以尝试开发自己端测部署任务了.
- CMSIS NN operations 对应的是 TFLMicro 基于CMSIS-NN 库进行二次开发的算子库,对ARM Cortex M4,M7 有大幅加速. **使用的有关注意事项请参照第四部分!!**

* API 手册可以访问这个[链接](docs/api.md), 其中提供了目前支持算子的情况
* 更多文档位于 [`/docs`](/docs) 下,使用前 **务必查看**
**注意:**

- Tensorflow Lite Micro 框架中的模块较多,使用之前请先参考 [introduction.md ](introduction.md), 通过 [user-guide.md](user-guide.md) 来了解基本使用流程流程。

* API 手册可以访问[此链接](docs/api.md), 其中提供了目前支持算子的情况
* 更多文档位于 [`docs`](/docs) 下,使用前 **务必查看**

## 4、注意事项

- 关于`Using Tensorflow Lite CMSIS NN operations `选项:
- 目前CMSIS对于算子的优化主要针对ARM Cortex M4以上的内核 ( 其搭载了DSP, FPU等硬件加速部件) 进行计算优化, 不推荐M4以下的MCU应用该选项.
- 目前CMSIS的算子优化目前只支持M系列的MCU, A系列, R系列不推荐应用该选项.
- CMSIS-NN 对于算子的优化主要针对 ARM Cortex M4 以上的 ARM 内核 ( 因为其搭载了DSP, FPU等硬件加速部件) 进行计算优化, M4 以下的 MCU 不建议使用该软件包。
- CMSIS-NN 的优化目前只提供了M系列的MCU;ARM Cortex A系列, R系列不应应用该选项.
- 目前CMSIS NN算子还处在测试阶段, 可能存在问题.
- 本软件包在运行时会占用16KB RAM空间, 同时自带的语音识别案例在运行时总共占用22KB内存, **请注意通过menuconfig来修改主函数栈的大小以及内存管理算法 !**

## 5、联系方式 & 感谢

* 维护:QingChuanWS
* 主页:https://github.com/QingChuanWS
* 欢迎各位RT-Threader试用本软件包并提出自己的建议, 我会认真听取并不断改进本软件包. 你的支持就是我最大的动力
* 欢迎各位developer使用该组件并提出自己的建议, 我会认真听取并不断改进. 您的支持就是我最大的动力

33 changes: 33 additions & 0 deletions examples/audio_main.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
/* Copyright 2019 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/

#include <rtthread.h>
#include <rtdevice.h>
#include <board.h>
#include "tflite/micro/examples/micro_speech/main_functions.h"

// This is the default main used on systems that have the standard C entry
// point. Other devices (for example FreeRTOS or ESP32) that have different
// requirements for entry code (like an app_main function) should specialize
// this main.cc file in a target-specific subfolder.
int main(int argc, char* argv[]) {
setup();
rt_kprintf("model load successfully!!\n");
while (true) {
loop();
}

return 0;
}
2 changes: 1 addition & 1 deletion tensorflow/lite/experimental/SConscript
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ cwd = GetCurrentDir()
objs = []
list = os.listdir(cwd)

if GetDepend('PKG_USING_TENSORFLOWLITEMICRO'):
if GetDepend('PKG_USING_TENSORFLOWLITEMICRO_AUDIO_EXAMPLE'):
for d in list:
path = os.path.join(cwd, d)
if os.path.isfile(os.path.join(path, 'SConscript')):
Expand Down
87 changes: 67 additions & 20 deletions tensorflow/lite/micro/SConscript
Original file line number Diff line number Diff line change
@@ -1,27 +1,74 @@
from building import *
import os

if GetDepend(['PKG_USING_TENSORFLOWLITEMICRO_REFERENCE']):
src = Glob('kernels/*.cc') + Glob('*.c') + Glob('*.cc') + Glob('benchmarks/*.cc') \
+ Glob('examples/person_detection_experimental/*.cc') \
+ Glob('memory_planner/*.cc') + Glob('testing/*.cc') + Glob('kernels/reference/*.cc') \
+ Glob('tools/make/downloads/person_model_int8/*.cc')
src = Split('''
kernels/activations.cc
kernels/arg_min_max.cc
kernels/ceil.cc
kernels/circular_buffer.cc
kernels/comparisons.cc
kernels/concatenation.cc
kernels/dequantize.cc
kernels/elementwise.cc
kernels/ethosu.cc
kernels/floor.cc
kernels/hard_swish.cc
kernels/kernel_runner.cc
kernels/kernel_util.cc
kernels/l2norm.cc
kernels/logical.cc
kernels/logistic.cc
kernels/maximum_minimum.cc
kernels/neg.cc
kernels/pack.cc
kernels/pad.cc
kernels/prelu.cc
kernels/quantize.cc
kernels/reduce.cc
kernels/reshape.cc
kernels/resize_nearest_neighbor.cc
kernels/round.cc
kernels/split.cc
kernels/strided_slice.cc
kernels/sub.cc
kernels/svdf.cc
kernels/tanh.cc
kernels/unpack.cc
''') + Glob('*.c') + Glob('*.cc') + Glob('benchmarks/*.cc') \
+ Glob('memory_planner/*.cc') + Glob('testing/*.cc')

if GetDepend(['PKG_USING_TENSORFLOWLITEMICRO_CMSISNN']):
src = Glob('kernels/*.cc') + Glob('*.c') + Glob('*.cc') + Glob('benchmarks/*.cc') \
+ Glob('examples/person_detection_experimental/*.cc') \
+ Glob('memory_planner/*.cc') + Glob('testing/*.cc') + Glob('kernels/cmsis-nn/*.cc') \
+ Glob('tools/make/downloads/person_model_int8/*.cc')
if GetDepend('PKG_USING_TENSORFLOWLITEMICRO_AUDIO_EXAMPLE'):
src += Glob('examples/micro_speech/*.cc') + Glob('examples/micro_speech/micro_features/*.cc')

src += Glob('tools/make/downloads/cmsis/CMSIS/NN/Source/ActivationFunctions/*.c')
src += Glob('tools/make/downloads/cmsis/CMSIS/NN/Source/BasicMathFunctions/*.c')
src += Glob('tools/make/downloads/cmsis/CMSIS/NN/Source/ConcatenationFunctions/*.c')
src += Glob('tools/make/downloads/cmsis/CMSIS/NN/Source/ConvolutionFunctions/*.c')
src += Glob('tools/make/downloads/cmsis/CMSIS/NN/Source/FullyConnectedFunctions/*.c')
src += Glob('tools/make/downloads/cmsis/CMSIS/NN/Source/NNSupportFunctions/*.c')
src += Glob('tools/make/downloads/cmsis/CMSIS/NN/Source/PoolingFunctions/*.c')
src += Glob('tools/make/downloads/cmsis/CMSIS/NN/Source/ReshapeFunctions/*.c')
src += Glob('tools/make/downloads/cmsis/CMSIS/NN/Source/SoftmaxFunctions/*.c')
if GetDepend('PKG_USING_TENSORFLOWLITEMICRO_REFERENCE'):
src += Split('''
kernels/add.cc
kernels/conv.cc
kernels/depthwise_conv.cc
kernels/fully_connected.cc
kernels/mul.cc
kernels/pooling.cc
kernels/softmax.cc
''')
if GetDepend('PKG_USING_TENSORFLOWLITEMICRO_CMSISNN'):
src += Split('''
kernels/cmsis-nn/add.cc
kernels/cmsis-nn/conv.cc
kernels/cmsis-nn/depthwise_conv.cc
kernels/cmsis-nn/fully_connected.cc
kernels/cmsis-nn/mul.cc
kernels/cmsis-nn/pooling.cc
kernels/cmsis-nn/softmax.cc
''')
src += Glob('tools/cmsis/CMSIS/NN/Source/ActivationFunctions/*.c')
src += Glob('tools/cmsis/CMSIS/NN/Source/BasicMathFunctions/*.c')
src += Glob('tools/cmsis/CMSIS/NN/Source/ConcatenationFunctions/*.c')
src += Glob('tools/cmsis/CMSIS/NN/Source/ConvolutionFunctions/*.c')
src += Glob('tools/cmsis/CMSIS/NN/Source/FullyConnectedFunctions/*.c')
src += Glob('tools/cmsis/CMSIS/NN/Source/NNSupportFunctions/*.c')
src += Glob('tools/cmsis/CMSIS/NN/Source/PoolingFunctions/*.c')
src += Glob('tools/cmsis/CMSIS/NN/Source/ReshapeFunctions/*.c')
src += Glob('tools/cmsis/CMSIS/NN/Source/SoftmaxFunctions/*.c')

#.
root = str(Dir('#'))
Expand All @@ -40,7 +87,7 @@ kissfft = os.path.join(tflm_pkg, "third_party/kissfft")
#./third_party/ruy
ruy = os.path.join(tflm_pkg, "third_party/ruy")
#./tensorflow/lite/micro/tools/make/downloads
cmsis = os.path.join(tflm_pkg, "tensorflow/lite/micro/tools/make/downloads")
cmsis = os.path.join(tflm_pkg, "tensorflow/lite/micro/tools/")


CPPPATH = [tflm_pkg, flatbuffer, gemmlowp, kissfft, ruy, cmsis]
Expand Down
Loading

0 comments on commit 81d41d9

Please sign in to comment.