diff --git a/.gitignore b/.gitignore index c4531785..77a60eb9 100644 --- a/.gitignore +++ b/.gitignore @@ -3,3 +3,5 @@ .vscode/ *audio_frontend* *google* +*__pycache__* +venv diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5001196e..a18ce1b0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -292,8 +292,7 @@ Below are some tips that might be useful and improve the development experience. ## Python notes -* [TensorFlow guide](https://www.tensorflow.org/community/contribute/code_style#python_style) - for Python development +* [TFLM Python guide](docs/python.md) # Continuous Integration System * Some [additional documentation](docs/continuous_integration.md) on the TFLM CI. diff --git a/README.md b/README.md index 0b3c820a..ba5d9b7f 100644 --- a/README.md +++ b/README.md @@ -93,6 +93,7 @@ The following resources may also be useful: * [Optimized Kernel Implementations](tensorflow/lite/micro/docs/optimized_kernel_implementations.md) * [New Platform Support](tensorflow/lite/micro/docs/new_platform_support.md) * [Software Emulation with Renode](tensorflow/lite/micro/docs/renode.md) + * [Python Dev Guide](docs/python.md) # RFCs diff --git a/WORKSPACE b/WORKSPACE index 5a7b695a..c78bb94b 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -13,7 +13,7 @@ # limitations under the License. # ============================================================================== -workspace(name = "tflm_bazel") +workspace(name = "tflite_micro") load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") @@ -29,7 +29,7 @@ load("@rules_python//python:pip.bzl", "pip_install") # third-party packages specified in the requirements.txt file. pip_install( name = "tflm_pip_deps", - requirements = "//third_party:bazel_requirements.txt", + requirements = "//third_party:requirements.txt", ) load("@//tensorflow:workspace.bzl", "workspace") diff --git a/ci/sync_from_upstream_tf.sh b/ci/sync_from_upstream_tf.sh index 3159a059..902c8ca3 100755 --- a/ci/sync_from_upstream_tf.sh +++ b/ci/sync_from_upstream_tf.sh @@ -50,7 +50,7 @@ done # repositories (TF and tflite-micro) which needs the import statements to be # modified. PY_FILES=$(find tensorflow/lite/tools tensorflow/lite/python -name "*.py") -sed -i 's/from tensorflow\.lite/from tflm_bazel\.tensorflow\.lite/' ${PY_FILES} +sed -i 's/from tensorflow\.lite/from tflite_micro\.tensorflow\.lite/' ${PY_FILES} # Since the TFLM code was deleted from the tensorflow repository, the # microfrontend is no longer sync'd from upstream and instead maintaned as a diff --git a/docs/python.md b/docs/python.md new file mode 100644 index 00000000..559141a5 --- /dev/null +++ b/docs/python.md @@ -0,0 +1,60 @@ + + * [Using Bazel](#using-bazel) + * [Manual Setup Illustration](#manual-setup-illustration) + + + + + +Writing and using Python scripts from the TFLM repository is currently in the +prototyping stage. As such, the instructions below are somewhat sparse and +subject to change. + + +* [TensorFlow Python style guide](https://www.tensorflow.org/community/contribute/code_style#python_style) + + +# Using Bazel + +We use Bazel as our default build system for Python and the continuous +integration infrastrucutre only runs the Python unit tests via Bazel. + +When using Bazel + Python, all the environment setup is handled as part of the +build. + +Some example commands: +```sh +bazel test tensorflow/lite/tools:flatbuffer_utils_test +bazel build tensorflow/lite/tools:visualize + +bazel-bin/tensorflow/lite/tools/visualize tensorflow/lite/micro/models/person_detect.tflite tensorflow/lite/micro/models/person_detect.tflite.html +``` + +# Manual Setup Illustration + +For advanced users that would like to use the Python code in the TFLM repository +independent of bazel, here is one approach. + +Please note that this setup is unsupported and will need users to debug various +issues on their own. It is described here for illustrative purposes only. + +```sh +# The cloned tflite-micro folder needs to be renamed to tflite_micro +mv tflite-micro tflite_micro +python -m venv tflite_micro/venv +echo "export PYTHONPATH=\${PYTHONPATH}:${PWD}" >> tflite_micro/venv/bin/activate +cd tflite_micro +source venv/bin/activate +pip install --upgrade pip +pip install -r third_party/requirements.txt + +# (Optional) +pip install ipython +``` + +Run some tests and binaries: +```sh +python tensorflow/lite/tools/flatbuffer_utils_test.py +python tensorflow/lite/tools/visualize.py tensorflow/lite/micro/models/person_detect.tflite tensorflow/lite/micro/models/person_detect.tflite.html +``` + diff --git a/tensorflow/lite/micro/examples/person_detection/utils/raw_to_bitmap_test.py b/tensorflow/lite/micro/examples/person_detection/utils/raw_to_bitmap_test.py index e46d045d..659f629d 100644 --- a/tensorflow/lite/micro/examples/person_detection/utils/raw_to_bitmap_test.py +++ b/tensorflow/lite/micro/examples/person_detection/utils/raw_to_bitmap_test.py @@ -22,8 +22,8 @@ import io import numpy as np -from tflm_bazel.tensorflow.lite.micro.examples.person_detection.utils.raw_to_bitmap import parse_file -from tflm_bazel.tensorflow.lite.micro.examples.person_detection.utils.raw_to_bitmap import reshape_bitmaps +from tflite_micro.tensorflow.lite.micro.examples.person_detection.utils.raw_to_bitmap import parse_file +from tflite_micro.tensorflow.lite.micro.examples.person_detection.utils.raw_to_bitmap import reshape_bitmaps from tensorflow.python.platform import googletest _RGB_RAW = u""" diff --git a/tensorflow/lite/tools/flatbuffer_utils.py b/tensorflow/lite/tools/flatbuffer_utils.py index fe4dfe1b..af40c869 100644 --- a/tensorflow/lite/tools/flatbuffer_utils.py +++ b/tensorflow/lite/tools/flatbuffer_utils.py @@ -26,7 +26,7 @@ import random import re import flatbuffers -from tflm_bazel.tensorflow.lite.python import schema_py_generated as schema_fb +from tflite_micro.tensorflow.lite.python import schema_py_generated as schema_fb from tensorflow.python.platform import gfile _TFLITE_FILE_IDENTIFIER = b'TFL3' diff --git a/tensorflow/lite/tools/flatbuffer_utils_test.py b/tensorflow/lite/tools/flatbuffer_utils_test.py index db4bdbb4..3d0cf91f 100644 --- a/tensorflow/lite/tools/flatbuffer_utils_test.py +++ b/tensorflow/lite/tools/flatbuffer_utils_test.py @@ -17,8 +17,8 @@ import copy import os import subprocess -from tflm_bazel.tensorflow.lite.tools import flatbuffer_utils -from tflm_bazel.tensorflow.lite.tools import test_utils +from tflite_micro.tensorflow.lite.tools import flatbuffer_utils +from tflite_micro.tensorflow.lite.tools import test_utils from tensorflow.python.framework import test_util from tensorflow.python.platform import test diff --git a/tensorflow/lite/tools/randomize_weights.py b/tensorflow/lite/tools/randomize_weights.py index 17f4d218..a712295c 100644 --- a/tensorflow/lite/tools/randomize_weights.py +++ b/tensorflow/lite/tools/randomize_weights.py @@ -17,7 +17,7 @@ r"""Randomize all weights in a tflite file.""" from absl import app from absl import flags -from tflm_bazel.tensorflow.lite.tools import flatbuffer_utils +from tflite_micro.tensorflow.lite.tools import flatbuffer_utils FLAGS = flags.FLAGS diff --git a/tensorflow/lite/tools/strip_strings.py b/tensorflow/lite/tools/strip_strings.py index ccbf989d..ff769db8 100644 --- a/tensorflow/lite/tools/strip_strings.py +++ b/tensorflow/lite/tools/strip_strings.py @@ -17,7 +17,7 @@ r"""Strips all nonessential strings from a TFLite file.""" from absl import app from absl import flags -from tflm_bazel.tensorflow.lite.tools import flatbuffer_utils +from tflite_micro.tensorflow.lite.tools import flatbuffer_utils FLAGS = flags.FLAGS diff --git a/tensorflow/lite/tools/test_utils.py b/tensorflow/lite/tools/test_utils.py index ce98e2e3..47cb3f8a 100644 --- a/tensorflow/lite/tools/test_utils.py +++ b/tensorflow/lite/tools/test_utils.py @@ -18,7 +18,7 @@ All functions that can be commonly used by various tests. """ import flatbuffers -from tflm_bazel.tensorflow.lite.python import schema_py_generated as schema_fb +from tflite_micro.tensorflow.lite.python import schema_py_generated as schema_fb TFLITE_SCHEMA_VERSION = 3 diff --git a/tensorflow/lite/tools/visualize.py b/tensorflow/lite/tools/visualize.py index 31c8f790..206b16d7 100644 --- a/tensorflow/lite/tools/visualize.py +++ b/tensorflow/lite/tools/visualize.py @@ -30,7 +30,7 @@ import numpy as np if not os.path.splitext(__file__)[0].endswith( os.path.join("tflite_runtime", "visualize")): # This file is part of tensorflow package. - from tflm_bazel.tensorflow.lite.python import schema_py_generated as schema_fb + from tflite_micro.tensorflow.lite.python import schema_py_generated as schema_fb else: # This file is part of tflite_runtime package. from tflite_runtime import schema_py_generated as schema_fb diff --git a/tensorflow/lite/tools/visualize_test.py b/tensorflow/lite/tools/visualize_test.py index 3af4e191..468008f9 100644 --- a/tensorflow/lite/tools/visualize_test.py +++ b/tensorflow/lite/tools/visualize_test.py @@ -16,8 +16,8 @@ import os import re -from tflm_bazel.tensorflow.lite.tools import test_utils -from tflm_bazel.tensorflow.lite.tools import visualize +from tflite_micro.tensorflow.lite.tools import test_utils +from tflite_micro.tensorflow.lite.tools import visualize from tensorflow.python.framework import test_util from tensorflow.python.platform import test diff --git a/third_party/bazel_requirements.txt b/third_party/requirements.txt similarity index 100% rename from third_party/bazel_requirements.txt rename to third_party/requirements.txt