New issue creation and updating mechanism for action error trapping.
On error creates a new issue with the tag ci:bot_issue if one doesn't exist. Places information about the script failing and run in the body.
If an issue with the script name in the body and the ci:bot_issue tag exists and is open, will update with a comment about the current failure.
Error trapping implemented for testing in generate_integration_tests.yml
BUG=https://issuetracker.google.com/issues/229828377
* Sync from upstream TF.
* Generate C++ bindings for schema as part of the import.
Co-authored-by: Advait Jain <advaitjain@users.noreply.github.com>
Co-authored-by: Advait Jain <advaitjain@google.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Using flatbuffers-1.12.0 to generate Python bindings from schema.fbs
This is a bit of a short-term workaround that is necessary to keep the
TF to TFLM sync functioning. Ideally we would be able to use a newer
version of flatbuffers with TFLM and generate the Python and C++
bindings directly from schema.fbs.
BUG=235888271
* Use a different version from upstream TF.
* fix the flatbuffers.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* First pass working with float input/output
* Numpy buffer input works. Got a sine graph for hello world output.
Copyright notice
* Remove static references
* Output tensors work
* Output tensors work
* bazel test works
bazel test tensorflow/lite/micro/tools:interpreter_test
--test_output=all
* Successfully built extension with Python setuptools
cd tensorflow/lite/micro/tools/interpreter_pypi
python -m build
pip install dist/example_tflm_interpreter_psho-0.0.3-cp39-cp39-linux_x86_64.whl --force-reinstall
python tests/test-interpreter.py
TestPyPi:
pip uninstall example-tflm-interpreter-psho
python -m twine upload --repository testpypi dist/example-tflm-interpreter-psho-0.0.3.tar.gz
pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple example-tflm-interpreter-psho==0.0.3
python tests/test-interpreter.py
* Refactored some
* Cleaned for production and tested with conv model
* Fixed formatting
* Fixed numpy header issue for Docker. Need to fix double registration issue that's somehow only present in Docker.
* Test fix with changes in ci.yml
* Added sudo
* Added nomsan and noasan tags to python test
* Added -layering-check, used py_library instead of py_binary to generate conv model, added micro namespace
* Removed extra debug code
* Added new tests
* Changed reinterpret_steal to cast
* Addressed PR comments
* size log refactor (#1156)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Addressed PR and review comments
* adding documentation to log_binary* workflows. (#1161)
* Automated binary size log update (#1160)
* Create an allocator to manage persistent arena (#1174)
* Create an allocator to manage persistent arena
This PR corresponds to internal cl/452380512.
Create an allocator to manage persistent arena.
This is another step towards the feature of enabling the client to
have two separate memory arenas.
BUG=https://b/226971240
* Manually patch due to Makefile difference
* Fix virtual environment when building Vela for Arm Ethos-U (#1177)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Removed print statement
Co-authored-by: jwithers <jpwithers@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: TFLM-bot <tflm-github-bot@google.com>
Co-authored-by: deqiangc <86809673+deqiangc@users.noreply.github.com>
Co-authored-by: Måns Nilsson <mans.nilsson@arm.com>
tflite_micro does not directly use schema.fbs to generate header file,
but it provides good documentation value and is required by some
projects.
BUG=https://b/226974950
This PR:
1. Update the tflite_files.txt with two additional files to be
pulled in
2. Add build rules to include context_util, builtin_op_data.h
and builtin_ops.h
3. Sync from upstream manually
BUG=observe errors in automatically
sync from upstream
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
common.c is renamed to common.cc in upstream
github.com/tensorflow/tensorflow/commit/848e84c5a3acda31847457cbd52742436e024984
Hence, change the name to sync it properly
BUG=observe error in sync from tf upstream
* Add tensor_utils header files from TensorFlow Lite
* Removed tensor_utils_common.h and added new file lstm_cell.h
tensor_utils_common.h is absorbed into portable_tensor_utils.h
in http://cl/428845263 and lstm_cell.h is separated from
reference_ops.h in http://cl/428639888 and refactored in
http://cl/428808334 for TFLM.
When upgrading the docker container, we see new format
fix/check on existing python file. That is because we did not
fix yapf version. This PR sets the yapf version to the
latest 0.32.0, which is what the latest container is using
BUG=https://issuetracker.google.com/216670232
* adding pandas and matplotlib
* updating container tag to latest
* fixing formatting issues that have been slipping through older ci setup
Co-authored-by: deqiangc <86809673+deqiangc@users.noreply.github.com>
Without this change, the sync workflow fails with:
```
ERROR: The project you're trying to build requires Bazel 4.2.2 (specified in /tmp/tensorflow/.bazelversion), but it wasn't found in /usr/local/lib/bazel/bin.
```
BUG=fix broken sync from upstream TF
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Allow using Python scripts without bazel.
Manually verified and documented the steps that I can follow to run
Python code without Bazel.
BUG=http://b/204109200
NO_CHECK_TFLITE_FILES=manually ran the sync script.
* Address review comments.
Sync additional Python flatbuffer utils from Tensorflow.
* strip_strings
* visualize
Manually confirmed that I can strip strings and visualize from the TFLM repo:
```
bazel build tensorflow/lite/tools:strip_strings
bazel build tensorflow/lite/tools:visualize
bazel-bin/tensorflow/lite/tools/strip_strings --input_tflite_file tensorflow/lite/micro/models/person_detect.tflite --output_tflite_file tensorflow/lite/micro/models/person_detect.strip_strings.tflite
bazel-bin/tensorflow/lite/tools/visualize tensorflow/lite/micro/models/person_detect.strip_strings.tflite tensorflow/lite/micro/models/person_detect.strip_strings.tflite.html
```
Additionally, this changes switches the bazel build to use tensorflow-cpu to avoid CUDA not found warnings.
BUG=http://b/204109200
This is the first step towards being able to manipulate flatbuffers from
Python from the TFLM github repository.
Manually confirmed that the following command works:
```
bazel test tensorflow/lite/tools:flatbuffer_utils_test
```
Note that the code that is sync'd from TF needs to have an updated
import path and we have changed the sync script to perform this
transformation.
These python tests will also be run as part of the bazel CI.
BUG=http://b/204109200
* Update to the latest buildifier release.
For the previous buildifier version, the exit code would always be 0
(regardless of whether files needed formatting or not).
Manually confirmed that with version 4.2.3 the test_code_style.sh script
correctly errors out when a BUILD file has formatting errors (and that
it doesn't with the previous version).
Also,
* Updated the docker container and pointing to the new container.
* Fix the formatting errors in .bzl file that crept in because this check was not working in OSS.
BUG=http://b/203803273
* Exclude the downloads directory.
* Updated docs based on review comments.
This is a first step towards enabling flatbuffer manipulation from
Python.
As part of this change:
* we generate the python flatbuffer bindings from the tensorflow
repository and copy the schema_py_generated into the TFLM repo.
* sync common python flatbuffer manipulation code from upstream TF.
Manually tested that the sync script runs as expected:
```
./ci/sync_from_upstream_tf.sh
```
BUG=http://b/204109200
* Bump renode and robot versions
Signed-off-by: Jakub Piecuch <jpiecuch@antmicro.com>
* Refactor test_with_renode.sh
Pass the name of the UART device as a parameter, since RISC-V uses
a different name.
Also pass the full list of binaries to test instead of a directory,
so that we can filter some tests out from within make, which is easier
compared to doing it in bash.
Signed-off-by: Jakub Piecuch <jpiecuch@antmicro.com>
* Use TfLiteRound instead of std::round in hard_swish_test
Signed-off-by: Jakub Piecuch <jpiecuch@antmicro.com>
* Don't pass UART name as an argument to test_with_renode.sh
Passing it as an argument works when running tests via `make test`, but
it breaks when we try to run a single test via `make test_foo`, because
the recipe for `make test_foo` doesn't pass the `${UART_NAME}` argument
to the test script.
Signed-off-by: Jakub Piecuch <jpiecuch@antmicro.com>
* Add a hexagon build presubmit
Also update Hexagon Docker file
* Add a todo for to enable unit test for hexagon
Co-authored-by: Advait Jain <advaitjain@users.noreply.github.com>
* Make size check workflow error out if memory footprint increase
between head of main and this PR.
This error can be disabled by putting MEM_CHANGE_OK= tag.
In either case, a memory footprint comparison report will be produced
as part of this workflow action.
The plan is for the code reviewer to audit the report when the
PR increase memory footprint and only
allow the PR that increase memory footprint to be merged with
MEM_CHANGE_OK=what_is_the_reason_for_this_increase_and_why_this_is_ok
The PR shall be up to date for the comparison to be accurate.
So before applying MEM_CHANGE_OK=, please make the PR branch up to date.
BUG=http://b/196637015
* Fix an if condition
* Create a test used for binary size test.
Also create a script to generate size report of a such binary.
Modify the size_comp.py to be more generic (not limited to xtensa).
what is tensorflow/lite/micro/tools/ci_build/binary_size_test will
be eventually removed once we verify the feature.
The next PR will update the yml file for the size_check.yml.
BUG=http://b/196637015
* Restore lines that are deleted by mistakes
* Update comments
With this change:
* we are set up to use yapf as part of the docker container
* pigweed patch is updated to use the google Python style (pep8 with indent of 2 spaces)
* some documentation updates.
The code still needs to be formatted, and then the Python formatting check will be turned on as part of ci.
BUG=http://b/194404216
* Have a TFLM-specific version of lite/kernels/op_macros.h
* After this change, we can remove the TFLM-specific code from upstream
Tensorflow.
* Both the TfLite and TFLM implementations of op_macros.h will be
simplified.
BUG=http://b/187728891
NO_CHECK_TFLITE_FILE=lite/kernels/kernel_util.cc will be modified in upstream as well
* Fix target-specific makefile build.
* modify a tflite file in the tflite-micro repository.
* fix the path
* Update workflow logic to complete workflow without error with NO_CHECK_TFLITE_FILES
* revert change to the tflite files.
* fail PRs that overwrite a set of files from upstream
* Change some names, modify Python script logic etc.
* small fixes.
* Use more restrictive TFLM-bot token.
Co-authored-by: Advait Jain <advaitjain@users.noreply.github.com>
Co-authored-by: Advait Jain <advaitjain@google.com>
Added batch_matmul.h and tensor_utils_common.h that are needed for the BATCH_MATMUL operator.
Co-authored-by: Advait Jain <advaitjain@users.noreply.github.com>
Add the header containing the reference implementation of op
SPACE_TO_DEPTH to the list of files which are synced from the
upstream TensorFlow repository. This reference is needed for the
forthcoming implementation of SPACE_TO_DEPTH here in micro.
* The TFLM Makefile globs all the shared TFL/TFLM code.
* This allows us to move the explicit list of sources and headers to the
sync script since that is where we determine what code needs to be
sync'd from upstream TF.
With this change, we are ready to have the tflite_micro repository be
the source of truth for all TFLM code and sync only the shared TFL/TFLM
code from the tensorflow repo.
Bug: http://b/182914089
