Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(json): Add requires_any field and use QIO by default to match IDE #10472

Merged
merged 5 commits into from
Oct 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ indent_size = 2
indent_style = space

[*.{bash,sh}]
indent_size = 2
indent_size = 4
indent_style = space

[*.{c,cc,cp,cpp,cxx,h,hh,hpp,hxx,ii,inl,ino,ixx,pde,tpl,tpp,txx}]
Expand Down
42 changes: 38 additions & 4 deletions .github/scripts/install-platformio-esp32.sh
Original file line number Diff line number Diff line change
Expand Up @@ -96,9 +96,9 @@ function count_sketches(){ # count_sketches <examples-path>
continue
fi

# Check if the sketch requires any configuration options
# Check if the sketch requires any configuration options (AND)
requirements=$(jq -r '.requires[]? // empty' $sketchdir/ci.json)
if [[ "$requirements" != "null" ]] || [[ "$requirements" != "" ]]; then
if [[ "$requirements" != "null" && "$requirements" != "" ]]; then
for requirement in $requirements; do
requirement=$(echo $requirement | xargs)
found_line=$(grep -E "^$requirement" "$SDKCONFIG_DIR/esp32/sdkconfig")
Expand All @@ -107,6 +107,23 @@ function count_sketches(){ # count_sketches <examples-path>
fi
done
fi

# Check if the sketch requires any configuration options (OR)
requirements_or=$(jq -r '.requires_any[]? // empty' $sketchdir/ci.json)
if [[ "$requirements_or" != "null" && "$requirements_or" != "" ]]; then
found=false
for requirement in $requirements_or; do
requirement=$(echo $requirement | xargs)
found_line=$(grep -E "^$requirement" "$SDKCONFIG_DIR/esp32/sdkconfig")
if [[ "$found_line" != "" ]]; then
found=true
break
fi
done
if [[ "$found" == "false" ]]; then
continue
fi
fi
fi

echo $sketch >> sketches.txt
Expand Down Expand Up @@ -187,9 +204,9 @@ function build_pio_sketches(){ # build_pio_sketches <board> <options> <examples-
continue
fi

# Check if the sketch requires any configuration options
# Check if the sketch requires any configuration options (AND)
requirements=$(jq -r '.requires[]? // empty' $sketchdir/ci.json)
if [[ "$requirements" != "null" ]] || [[ "$requirements" != "" ]]; then
if [[ "$requirements" != "null" && "$requirements" != "" ]]; then
for requirement in $requirements; do
requirement=$(echo $requirement | xargs)
found_line=$(grep -E "^$requirement" "$SDKCONFIG_DIR/esp32/sdkconfig")
Expand All @@ -198,6 +215,23 @@ function build_pio_sketches(){ # build_pio_sketches <board> <options> <examples-
fi
done
fi

# Check if the sketch requires any configuration options (OR)
requirements_or=$(jq -r '.requires_any[]? // empty' $sketchdir/ci.json)
if [[ "$requirements_or" != "null" && "$requirements_or" != "" ]]; then
found=false
for requirement in $requirements_or; do
requirement=$(echo $requirement | xargs)
found_line=$(grep -E "^$requirement" "$SDKCONFIG_DIR/esp32/sdkconfig")
if [[ "$found_line" != "" ]]; then
found=true
break
fi
done
if [[ "$found" == "false" ]]; then
continue
fi
fi
fi

sketchnum=$(($sketchnum + 1))
Expand Down
79 changes: 61 additions & 18 deletions .github/scripts/sketch_utils.sh
Original file line number Diff line number Diff line change
Expand Up @@ -98,34 +98,42 @@ function build_sketch(){ # build_sketch <ide_path> <user_path> <path-to-ino> [ex

# Default FQBN options if none were passed in the command line.

esp32_opts="PSRAM=enabled,FlashMode=dio${fqbn_append:+,$fqbn_append}"
esp32s2_opts="PSRAM=enabled,FlashMode=dio${fqbn_append:+,$fqbn_append}"
esp32s3_opts="PSRAM=opi,USBMode=default,FlashMode=dio${fqbn_append:+,$fqbn_append}"
esp32c3_opts="FlashMode=dio${fqbn_append:+,$fqbn_append}"
esp32c6_opts="FlashMode=dio${fqbn_append:+,$fqbn_append}"
esp32h2_opts="FlashMode=dio${fqbn_append:+,$fqbn_append}"
esp32_opts="PSRAM=enabled${fqbn_append:+,$fqbn_append}"
esp32s2_opts="PSRAM=enabled${fqbn_append:+,$fqbn_append}"
esp32s3_opts="PSRAM=opi,USBMode=default${fqbn_append:+,$fqbn_append}"
esp32c3_opts="$fqbn_append"
esp32c6_opts="$fqbn_append"
esp32h2_opts="$fqbn_append"

# Select the common part of the FQBN based on the target. The rest will be
# appended depending on the passed options.

opt=""

case "$target" in
"esp32")
fqbn="espressif:esp32:esp32:${options:-$esp32_opts}"
[ -n "${options:-$esp32_opts}" ] && opt=":${options:-$esp32_opts}"
fqbn="espressif:esp32:esp32$opt"
;;
"esp32s2")
fqbn="espressif:esp32:esp32s2:${options:-$esp32s2_opts}"
[ -n "${options:-$esp32s2_opts}" ] && opt=":${options:-$esp32s2_opts}"
fqbn="espressif:esp32:esp32s2$opt"
;;
"esp32c3")
fqbn="espressif:esp32:esp32c3:${options:-$esp32c3_opts}"
[ -n "${options:-$esp32c3_opts}" ] && opt=":${options:-$esp32c3_opts}"
fqbn="espressif:esp32:esp32c3$opt"
;;
"esp32s3")
fqbn="espressif:esp32:esp32s3:${options:-$esp32s3_opts}"
[ -n "${options:-$esp32s3_opts}" ] && opt=":${options:-$esp32s3_opts}"
fqbn="espressif:esp32:esp32s3$opt"
;;
"esp32c6")
fqbn="espressif:esp32:esp32c6:${options:-$esp32c6_opts}"
[ -n "${options:-$esp32c6_opts}" ] && opt=":${options:-$esp32c6_opts}"
fqbn="espressif:esp32:esp32c6$opt"
;;
"esp32h2")
fqbn="espressif:esp32:esp32h2:${options:-$esp32h2_opts}"
[ -n "${options:-$esp32h2_opts}" ] && opt=":${options:-$esp32h2_opts}"
fqbn="espressif:esp32:esp32h2$opt"
;;
esac

Expand Down Expand Up @@ -163,9 +171,9 @@ function build_sketch(){ # build_sketch <ide_path> <user_path> <path-to-ino> [ex
exit 0
fi

# Check if the sketch requires any configuration options
# Check if the sketch requires any configuration options (AND)
requirements=$(jq -r '.requires[]? // empty' $sketchdir/ci.json)
if [[ "$requirements" != "null" ]] || [[ "$requirements" != "" ]]; then
if [[ "$requirements" != "null" && "$requirements" != "" ]]; then
for requirement in $requirements; do
requirement=$(echo $requirement | xargs)
found_line=$(grep -E "^$requirement" "$SDKCONFIG_DIR/$target/sdkconfig")
Expand All @@ -175,6 +183,24 @@ function build_sketch(){ # build_sketch <ide_path> <user_path> <path-to-ino> [ex
fi
done
fi

# Check if the sketch excludes any configuration options (OR)
requirements_or=$(jq -r '.requires_any[]? // empty' $sketchdir/ci.json)
if [[ "$requirements_or" != "null" && "$requirements_or" != "" ]]; then
found=false
for requirement in $requirements_or; do
requirement=$(echo $requirement | xargs)
found_line=$(grep -E "^$requirement" "$SDKCONFIG_DIR/$target/sdkconfig")
if [[ "$found_line" != "" ]]; then
found=true
break
fi
done
if [[ "$found" == "false" ]]; then
echo "Target $target meets none of the requirements in requires_any for $sketchname. Skipping."
exit 0
fi
fi
fi

ARDUINO_CACHE_DIR="$HOME/.arduino/cache.tmp"
Expand Down Expand Up @@ -213,9 +239,9 @@ function build_sketch(){ # build_sketch <ide_path> <user_path> <path-to-ino> [ex
--build-cache-path "$ARDUINO_CACHE_DIR" \
--build-path "$build_dir" \
$xtra_opts "${sketchdir}" \
> $output_file
2>&1 | tee $output_file

exit_status=$?
exit_status=${PIPESTATUS[0]}
if [ $exit_status -ne 0 ]; then
echo "ERROR: Compilation failed with error code $exit_status"
exit $exit_status
Expand Down Expand Up @@ -322,9 +348,9 @@ function count_sketches(){ # count_sketches <path> [target] [file] [ignore-requi
fi

if [ "$ignore_requirements" != "1" ]; then
# Check if the sketch requires any configuration options
# Check if the sketch requires any configuration options (AND)
requirements=$(jq -r '.requires[]? // empty' $sketchdir/ci.json)
if [[ "$requirements" != "null" ]] || [[ "$requirements" != "" ]]; then
if [[ "$requirements" != "null" && "$requirements" != "" ]]; then
for requirement in $requirements; do
requirement=$(echo $requirement | xargs)
found_line=$(grep -E "^$requirement" $SDKCONFIG_DIR/$target/sdkconfig)
Expand All @@ -333,6 +359,23 @@ function count_sketches(){ # count_sketches <path> [target] [file] [ignore-requi
fi
done
fi

# Check if the sketch excludes any configuration options (OR)
requirements_or=$(jq -r '.requires_any[]? // empty' $sketchdir/ci.json)
if [[ "$requirements_or" != "null" && "$requirements_or" != "" ]]; then
found=false
for requirement in $requirements_or; do
requirement=$(echo $requirement | xargs)
found_line=$(grep -E "^$requirement" $SDKCONFIG_DIR/$target/sdkconfig)
if [[ "$found_line" != "" ]]; then
found=true
break
fi
done
if [[ "$found" == "false" ]]; then
continue 2
fi
fi
fi
fi
echo $sketch >> sketches.txt
Expand Down
23 changes: 21 additions & 2 deletions .github/scripts/tests_run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,9 @@ function run_test() {
return 0
fi

# Check if the sketch requires any configuration options
# Check if the sketch requires any configuration options (AND)
requirements=$(jq -r '.requires[]? // empty' $sketchdir/ci.json)
if [[ "$requirements" != "null" ]] || [[ "$requirements" != "" ]]; then
if [[ "$requirements" != "null" && "$requirements" != "" ]]; then
for requirement in $requirements; do
requirement=$(echo $requirement | xargs)
found_line=$(grep -E "^$requirement" "$SDKCONFIG_PATH")
Expand All @@ -49,6 +49,25 @@ function run_test() {
fi
done
fi

# Check if the sketch requires any configuration options (OR)
requirements_or=$(jq -r '.requires_any[]? // empty' $sketchdir/ci.json)
if [[ "$requirements_or" != "null" && "$requirements_or" != "" ]]; then
found=false
for requirement in $requirements_or; do
requirement=$(echo $requirement | xargs)
found_line=$(grep -E "^$requirement" "$SDKCONFIG_PATH")
if [[ "$found_line" != "" ]]; then
found=true
break
fi
done
if [[ "$found" == "false" ]]; then
printf "\033[93mTarget $target meets none of the requirements in requires_any for $sketchname. Skipping.\033[0m\n"
printf "\n\n\n"
return 0
fi
fi
fi

if [ $len -eq 1 ]; then
Expand Down
16 changes: 9 additions & 7 deletions docs/en/contributing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -172,12 +172,12 @@ And in the ``README.md`` file:
By default, the CI system will use the FQBNs specified in the ``.github/scripts/sketch_utils.sh`` file to compile the sketches.
Currently, the default FQBNs are:

* ``espressif:esp32:esp32:PSRAM=enabled,FlashMode=dio``
* ``espressif:esp32:esp32s2:PSRAM=enabled,FlashMode=dio``
* ``espressif:esp32:esp32s3:PSRAM=opi,USBMode=default,FlashMode=dio``
* ``espressif:esp32:esp32c3:FlashMode=dio``
* ``espressif:esp32:esp32c6:FlashMode=dio``
* ``espressif:esp32:esp32h2:FlashMode=dio``
* ``espressif:esp32:esp32:PSRAM=enabled``
* ``espressif:esp32:esp32s2:PSRAM=enabled``
* ``espressif:esp32:esp32s3:PSRAM=opi,USBMode=default``
* ``espressif:esp32:esp32c3``
* ``espressif:esp32:esp32c6``
* ``espressif:esp32:esp32h2``

There are two ways to alter the FQBNs used to compile the sketches: by using the ``fqbn`` or ``fqbn_append`` fields in the ``ci.json`` file.

Expand Down Expand Up @@ -408,7 +408,9 @@ CI JSON File
The ``ci.json`` file is used to specify how the test suite and sketches will handled by the CI system. It can contain the following fields:

* ``requires``: A list of configurations in ``sdkconfig`` that are required to run the test suite. The test suite will only run on the targets
that have the required configurations. By default, no configurations are required.
that have **ALL** the required configurations. By default, no configurations are required.
* ``requires_any``: A list of configurations in ``sdkconfig`` that are required to run the test suite. The test suite will only run on the targets
that have **ANY** of the required configurations. By default, no configurations are required.
* ``targets``: A dictionary that specifies the targets for which the tests will be run. The key is the target name and the value is a boolean
that specifies if the test should be run for that target. By default, all targets are enabled as long as they have the required configurations
specified in the ``requires`` field. This field is also valid for examples.
Expand Down
Loading