First steps towards getting CI going (#3)

Manually confirmed that the steps in CONTRIBUTING.md work: ``` docker build -t tflm-test -f ci/Dockerfile.micro . docker run --rm tflm-test ```
2026-09-01 17:57:27 +00:00 · 2021-04-09 12:54:36 -07:00 · 2021-04-09 12:54:36 -07:00 · d1a6c7900c
commit d1a6c7900c
parent a37043ff7a
5 changed files with 132 additions and 46 deletions
--- a/tensorflow/.clang-format
+++ b/tensorflow/.clang-format
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@ -1,3 +1,31 @@
+<!--
+Semi-automated TOC generation with instructions from
+https://github.com/ekalinin/github-markdown-toc#auto-insert-and-update-toc
+-->
+<!--ts-->
+   * [How to Contribute](#how-to-contribute)
+      * [Contributor License Agreement](#contributor-license-agreement)
+      * [Community Guidelines](#community-guidelines)
+   * [Code Contribution Guidelines](#code-contribution-guidelines)
+      * [General Pull Request Guidelines](#general-pull-request-guidelines)
+      * [Guidelines for Specific Contribution Categories](#guidelines-for-specific-contribution-categories)
+         * [Bug Fixes](#bug-fixes)
+         * [Reference Kernel Implementations](#reference-kernel-implementations)
+         * [Optimized Kernel Implementations](#optimized-kernel-implementations)
+         * [New Target / Platform / IDE / Examples](#new-target--platform--ide--examples)
+         * [New Features](#new-features)
+   * [Development Workflow Notes](#development-workflow-notes)
+      * [Initial Setup](#initial-setup)
+      * [Before submitting your PR](#before-submitting-your-pr)
+      * [During the PR review](#during-the-pr-review)
+      * [Reviewer notes](#reviewer-notes)
+      * [Python notes](#python-notes)
+   * [Continuous Integration System](#continuous-integration-system)
+
+<!-- Added by: advaitjain, at: Fri 09 Apr 2021 12:51:55 PM PDT -->
+
+<!--te-->
+
 # How to Contribute

 We'd love to accept your patches and contributions to this project. There are
@ -324,50 +352,9 @@ that can be expanded and improved as necessary.

 # Continuous Integration System

-*   As a contributor, please make sure that the TfLite Micro build is green.
-    You can click on the details link to see what the errors are:
-
-[![TfLite Micro Build](docs/images/tflm_continuous_integration_1.png)](https://storage.googleapis.com/tensorflow-kokoro-build-badges/tflite-micro.html)
-
-*   Tests that are run as part of the CI are with the
-    [micro/tools/ci_build/test_all.sh](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/micro/tools/ci_build/test_all.sh)
-    script when run with the `GITHUB_PRESUBMIT` command line parameter:
-    ```
-    tensorflow/lite/micro/tools/ci_build/test_all.sh GITHUB_PRESUBMIT
-    ```
-
-*   If an error is not reproducible on your development machine, you can
-    recreate the docker container that is used on the CI servers.
-
-      * First, create a build a TFLM docker image with:
-        ```
-        tensorflow/tools/ci_build/ci_build.sh micro bash
-        ```
-        The second parameter to the ci_build.sh script is not important. It can
-        be any command.
-
-      * Next, mount the tensorflow repo on your machine to the docker container.
-        Please be careful (or make a separate clone of tensorflow) since any
-        changes docker container will also be reflected in the directory in the
-        host machine.
-        ```
-        docker run -v `pwd`:/tensorflow -it tf_ci.micro bash
-        # cd tensorflow
-        ```
-
-      * If you would prefer to not mount your local folder on the docker image,
-        you can also simply download the branch:
-        ```
-        docker run -it tf_ci.micro bash
-        # wget https://github.com/<github-username>/tensorflow/archive/<git-branch>.zip
-        # unzip <git-branch>.zip
-        # cd tensorflow-<git-branch>
-        ```
-
-      * Within the docker container, you can now run the TFLM test script, or
-        any other command that you would like to test. For example, the following
-        commands will run all of the TFLM checks:
-        ```
-        # tensorflow/lite/micro/tools/ci_build/test_all.sh GITHUB_PRESUBMIT
-        ```
+  * All the tests can be run from within a docker container:
+   ```
+   docker build -t tflm-test -f ci/Dockerfile.micro .
+   docker run --rm tflm-test
+   ```

--- a/ci/Dockerfile.micro
+++ b/ci/Dockerfile.micro
@ -0,0 +1,33 @@
+# Use a prebuilt Python image instead of base Ubuntu to speed up the build process,
+# since it has all the build dependencies we need for Micro and downloads much faster
+# than the install process.
+FROM python:3.9.0-buster
+
+RUN echo deb http://apt.llvm.org/buster/ llvm-toolchain-buster-12 main > /etc/apt/sources.list.d/llvm.list
+RUN wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add -
+
+RUN apt-get update
+
+RUN apt-get install -y zip xxd sudo
+
+RUN apt-get install -y clang-12 clang-format-12
+# Set clang-12 and clang-format-12 as the default to ensure that the pigweed
+# formatting scripts use the desired version.
+RUN ln -s /usr/bin/clang-12 /usr/bin/clang
+RUN ln -s /usr/bin/clang++-12 /usr/bin/clang++
+RUN ln -s /usr/bin/clang-format-12 /usr/bin/clang-format
+
+RUN pip install six
+# Install Renode test dependencies
+RUN pip install pyyaml requests psutil robotframework==3.1
+
+# Install bazel and buildifier so that the bazel presubmit checks can be run
+# from the micro docker container and are consistent with the rest of the CI.
+COPY ci/*.sh /install/
+RUN /install/install_bazel.sh
+RUN /install/install_buildifier.sh
+
+COPY . /opt/tflm/
+WORKDIR /opt/tflm
+
+CMD ["/opt/tflm/tensorflow/lite/micro/tools/ci_build/test_all_new.sh", "GITHUB_PRESUBMIT"]
--- a/ci/install_bazel.sh
+++ b/ci/install_bazel.sh
@ -0,0 +1,40 @@
+#!/usr/bin/env bash
+# Copyright 2021 The TensorFlow Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ==============================================================================
+
+# Select bazel version.
+BAZEL_VERSION="3.7.2"
+
+set +e
+local_bazel_ver=$(bazel version 2>&1 | grep -i label | awk '{print $3}')
+
+if [[ "$local_bazel_ver" == "$BAZEL_VERSION" ]]; then
+  exit 0
+fi
+
+set -e
+
+# Install bazel.
+mkdir -p /bazel
+cd /bazel
+if [[ ! -f "bazel-$BAZEL_VERSION-installer-linux-x86_64.sh" ]]; then
+  curl -fSsL -O https://github.com/bazelbuild/bazel/releases/download/$BAZEL_VERSION/bazel-$BAZEL_VERSION-installer-linux-x86_64.sh
+fi
+chmod +x /bazel/bazel-*.sh
+/bazel/bazel-$BAZEL_VERSION-installer-linux-x86_64.sh
+rm -f /bazel/bazel-$BAZEL_VERSION-installer-linux-x86_64.sh
+
+# Enable bazel auto completion.
+echo "source /usr/local/lib/bazel/bin/bazel-complete.bash" >> ~/.bashrc
--- a/ci/install_buildifier.sh
+++ b/ci/install_buildifier.sh
@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+# Copyright 2021 The TensorFlow Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ==============================================================================
+
+set -e
+# Download buildifier.
+wget https://github.com/bazelbuild/buildtools/releases/download/0.4.5/buildifier
+chmod +x buildifier
+sudo mv buildifier /usr/local/bin/.
+
+# Download buildozer.
+wget https://github.com/bazelbuild/buildtools/releases/download/0.4.5/buildozer
+chmod +x buildozer
+sudo mv buildozer /usr/local/bin/.