Changed: Move to semantic versioning for app and library versions and add commit hash and github to APK file names

agnostic-apollo · agnostic-apollo · commit 885fdefe3d2c · 2022-01-23T01:36:41.000+05:00
The `versionName` will now follow semantic version `2.0.0` spec in the format `major.minor.patch(-prerelease)(+buildmetadata)`. This will make versioning the prerelease and github debug builds versions easier and follow a spec. The @termux devs should make sure that when bumping `versionName` in `build.gradle` files and when creating a tag for new releases on github that they include the patch number as well, like `v0.1.0` instead of just `v0.1`. The `build.gradle` files and `attach_debug_apks_to_release` workflow will now validate the version as well and the build/attachment will fail if `versionName` does not follow the spec. https://semver.org/spec/v2.0.0.html APKs released on github for debug build workflows and releases are now referred as `Github` releases as per 7b10a35 and 94e01d6, so APK filenames have been modified to include `github` in the filename. The APKs are still debuggable, so that tag remains too. For github workflows the apk filename format will be `termux-app_<current_version>+<last_commit_hash>-github-debug_<arch>.apk`, like `termux-app_v0.1.0+xxxxxxxx-github-debug_arm64-v8a.apk` and for github releases it will be `termux-app_<release_version>+github-debug_<arch>.apk`, like `termux-app_v0.1+github-debug_arm64-v8a.apk`. The `last_commit_hash` will be the first `8` characters of the commit hash. The `<last_commit_hash>-github-debug` will act as `buildmetadata` and will not affect versioning precedence. For github workflows triggered by `push` and `pull_request` triggers, `<current_version>+<last_commit_hash>` will be used as new `versionName`, like `v0.1.0+xxxxxxxx`. This will make tracking which build a user is using easier and help in resolving issues as well. Note that users using github releases and termux devs using `$TERMUX_VERSION` environment variables in scripts should take commit hash into consideration and possibly use something like `dpkg --compare-versions "$TERMUX_VERSION" ge 0.1` where appropriate instead of mathematical comparisons. The `app/build.gradle` now also supports following `TERMUX_` scoped environmental variables and `RELEASE_TAG` variable will not be used anymore since it may conflict with possibly other variables used by users. They will also allow enabling split APKs for both debug and release builds. - `TERMUX_APP_VERSION_NAME` will be used as `versionName` if its set. - `TERMUX_APK_VERSION_TAG` will be used as `termux-app_<TERMUX_APK_VERSION_TAG>_<arch>.apk` if its set. The `_<arch>` will only exist for split APKs. - `TERMUX_SPLIT_APKS_FOR_DEBUG_BUILDS` will define whether split APKs should be enabled for debug builds. Default value is `1`. - `TERMUX_SPLIT_APKS_FOR_RELEASE_BUILDS` will define whether split APKs should be enabled for release builds. Default value is `0` since F-Droid does not support split APKs, check termux#1904. So based on above, if in future github releases are to be converted to `release` builds instead of `debug` builds, something like following can be done and even a workflow can be created for it. Users can also build split APKs release builds for themselves if they want. ``` export TERMUX_SPLIT_APKS_FOR_RELEASE_BUILDS=1 ./gradlew assembleRelease -Pandroid.injected.signing.store.file="$(pwd)/app/dev_keystore.jks" -Pandroid.injected.signing.store.password=xrj45yWGLbsO7W0v -Pandroid.injected.signing.key.alias=alias -Pandroid.injected.signing.key.password=xrj45yWGLbsO7W0v ``` The APK will be found at `./app/build/outputs/apk/release/termux-app_<version>_<arch>.apk` The `TERMUX_SPLIT_APKS_FOR_DEBUG_BUILDS` can be set to `0` to disable building split APKs which may be helpful for users building termux on device considering they will extra space and build time. Instructions for building are at termux/termux-packages#7227 (comment). ``` export TERMUX_SPLIT_APKS_FOR_DEBUG_BUILDS=0 ./gradlew assembleDebug ``` The APK will be found at `./app/build/outputs/apk/debug/termux-app_debug_universal.apk` Note that F-Droid uses algorithm at https://gitlab.com/fdroid/fdroidserver/-/blob/2.1a0/fdroidserver/build.py#L746 to automatically detect built APKs, so ensure any modifications to location or file name are compliant. Current updates should be. Auto updates are detected by checkupdates bot at https://gitlab.com/fdroid/fdroidserver/-/blob/master/fdroidserver/checkupdates.py
diff --git a/.github/workflows/attach_debug_apks_to_release.yml b/.github/workflows/attach_debug_apks_to_release.yml
@@ -6,31 +6,71 @@ on:
       - published
 
 jobs:
-  build:
+  attach-apks:
     runs-on: ubuntu-latest
     env:
       GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
     steps:
-      - name: Set tag
-        id: vars
-        run: echo ::set-output name=tag::${GITHUB_REF/refs\/tags\//}
       - name: Clone repository
         uses: actions/checkout@v2
         with:
           ref: ${{ env.GITHUB_REF }}
-      - name: Build
-        env:
-          RELEASE_TAG: ${{ steps.vars.outputs.tag }}
-        run: ./gradlew assembleDebug
-      - name: Attach debug APKs to release
-        env:
-          RELEASE_TAG: ${{ steps.vars.outputs.tag }}
-        run: >-
-          hub release edit
-          -m ""
-          -a ./app/build/outputs/apk/debug/termux-app-universal-$RELEASE_TAG-debug.apk
-          -a ./app/build/outputs/apk/debug/termux-app-arm64-v8a-$RELEASE_TAG-debug.apk
-          -a ./app/build/outputs/apk/debug/termux-app-armeabi-v7a-$RELEASE_TAG-debug.apk
-          -a ./app/build/outputs/apk/debug/termux-app-x86_64-$RELEASE_TAG-debug.apk
-          -a ./app/build/outputs/apk/debug/termux-app-x86-$RELEASE_TAG-debug.apk
-          $RELEASE_TAG
+
+      - name: Build and attach APKs to release
+        shell: bash {0}
+        run: |
+          exit_on_error() {
+              echo "$1"
+              echo "Deleting '$RELEASE_VERSION_NAME' release and '$GITHUB_REF' tag"
+              hub release delete "$RELEASE_VERSION_NAME"
+              git push --delete origin "$GITHUB_REF"
+              exit 1
+          }
+
+          echo "Setting vars"
+          RELEASE_VERSION_NAME="${GITHUB_REF/refs\/tags\//}"
+          if ! printf "%s" "${RELEASE_VERSION_NAME/v/}" | grep -qP '^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$'; then
+           exit_on_error "The versionName '${RELEASE_VERSION_NAME/v/}' is not a valid version as per semantic version '2.0.0' spec in the format 'major.minor.patch(-prerelease)(+buildmetadata)'. https://semver.org/spec/v2.0.0.html."
+          fi
+
+          APK_DIR_PATH="./app/build/outputs/apk/debug"
+          APK_VERSION_TAG="$RELEASE_VERSION_NAME+github-debug"
+          APK_BASENAME_PREFIX="termux-app_$APK_VERSION_TAG"
+
+          echo "Building APKs for '$RELEASE_VERSION_NAME' release"
+          export TERMUX_APK_VERSION_TAG="$APK_VERSION_TAG" # Used by app/build.gradle
+          if ! ./gradlew assembleDebug; then
+            exit_on_error "Build failed for '$RELEASE_VERSION_NAME' release."
+          fi
+
+          echo "Validating APKs"
+          for abi in universal arm64-v8a armeabi-v7a x86_64 x86; do
+            if ! test -f "$APK_DIR_PATH/${APK_BASENAME_PREFIX}_$abi.apk"; then
+              files_found="$(ls "$APK_DIR_PATH")"
+              exit_on_error "Failed to find built APK at '$APK_DIR_PATH/${APK_BASENAME_PREFIX}_$abi.apk'. Files found: "$'\n'"$files_found"
+            fi
+          done
+
+          echo "Generating sha25sums file"
+          if ! (cd "$APK_DIR_PATH"; sha256sum \
+            "${APK_BASENAME_PREFIX}_universal.apk" \
+            "${APK_BASENAME_PREFIX}_arm64-v8a.apk" \
+            "${APK_BASENAME_PREFIX}_armeabi-v7a.apk" \
+            "${APK_BASENAME_PREFIX}_x86_64.apk" \
+            "${APK_BASENAME_PREFIX}_x86.apk" \
+            > sha256sums); then
+            exit_on_error "Generate sha25sums failed for '$RELEASE_VERSION_NAME' release."
+          fi
+
+          echo "Attaching APKs to github release"
+          if ! hub release edit \
+          -m "" \
+          -a "$APK_DIR_PATH/${APK_BASENAME_PREFIX}_universal.apk" \
+          -a "$APK_DIR_PATH/${APK_BASENAME_PREFIX}_arm64-v8a.apk" \
+          -a "$APK_DIR_PATH/${APK_BASENAME_PREFIX}_armeabi-v7a.apk" \
+          -a "$APK_DIR_PATH/${APK_BASENAME_PREFIX}_x86_64.apk" \
+          -a "$APK_DIR_PATH/${APK_BASENAME_PREFIX}_x86.apk" \
+          -a "$APK_DIR_PATH/sha256sums" \
+          "$RELEASE_VERSION_NAME"; then
+            exit_on_error "Attach APKs to release failed for '$RELEASE_VERSION_NAME' release."
+          fi
diff --git a/.github/workflows/debug_build.yml b/.github/workflows/debug_build.yml
@@ -4,53 +4,110 @@ on:
   push:
     branches:
       - master
-      - android-10
   pull_request:
     branches:
       - master
-      - android-10
 
 jobs:
   build:
     runs-on: ubuntu-latest
     steps:
-    - name: Clone repository
-      uses: actions/checkout@v2
-    - name: Build
-      run: |
-        ./gradlew assembleDebug
-    - name: Store generated universal APK file
-      uses: actions/upload-artifact@v2
-      with:
-        name: termux-app-universal
-        path: |
-            ./app/build/outputs/apk/debug/termux-app-universal-debug.apk
-            ./app/build/outputs/apk/debug/output-metadata.json
-    - name: Store generated arm64-v8a APK file
-      uses: actions/upload-artifact@v2
-      with:
-        name: termux-app-arm64-v8a
-        path: |
-          ./app/build/outputs/apk/debug/termux-app-arm64-v8a-debug.apk
-          ./app/build/outputs/apk/debug/output-metadata.json
-    - name: Store generated armeabi-v7a APK file
-      uses: actions/upload-artifact@v2
-      with:
-        name: termux-app-armeabi-v7a
-        path: |
-          ./app/build/outputs/apk/debug/termux-app-armeabi-v7a-debug.apk
-          ./app/build/outputs/apk/debug/output-metadata.json
-    - name: Store generated x86_64 APK file
-      uses: actions/upload-artifact@v2
-      with:
-        name: termux-app-x86_64
-        path: |
-          ./app/build/outputs/apk/debug/termux-app-x86_64-debug.apk
-          ./app/build/outputs/apk/debug/output-metadata.json
-    - name: Store generated x86 APK file
-      uses: actions/upload-artifact@v2
-      with:
-        name: termux-app-x86
-        path: |
-          ./app/build/outputs/apk/debug/termux-app-x86-debug.apk
-          ./app/build/outputs/apk/debug/output-metadata.json
+      - name: Clone repository
+        uses: actions/checkout@v2
+
+      - name: Build APKs
+        shell: bash {0}
+        run: |
+          exit_on_error() { echo "$1"; exit 1; }
+
+          echo "Setting vars"
+          # Set RELEASE_VERSION_NAME to "<CURRENT_VERSION_NAME>+<last_commit_hash>"
+          CURRENT_VERSION_NAME_REGEX='\s+versionName "([^"]+)"$'
+          CURRENT_VERSION_NAME="$(grep -m 1 -E "$CURRENT_VERSION_NAME_REGEX" ./app/build.gradle | sed -r "s/$CURRENT_VERSION_NAME_REGEX/\1/")"
+          RELEASE_VERSION_NAME="v$CURRENT_VERSION_NAME+${GITHUB_SHA:0:7}" # The "+" is necessary so that versioning precedence is not affected
+          if ! printf "%s" "${RELEASE_VERSION_NAME/v/}" | grep -qP '^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$'; then
+           exit_on_error "The versionName '${RELEASE_VERSION_NAME/v/}' is not a valid version as per semantic version '2.0.0' spec in the format 'major.minor.patch(-prerelease)(+buildmetadata)'. https://semver.org/spec/v2.0.0.html."
+          fi
+
+          APK_DIR_PATH="./app/build/outputs/apk/debug"
+          APK_VERSION_TAG="$RELEASE_VERSION_NAME-github-debug" # Note the "-", GITHUB_SHA will already have "+" before it
+          APK_BASENAME_PREFIX="termux-app_$APK_VERSION_TAG"
+
+          # Used by attachment steps later
+          echo "APK_DIR_PATH=$APK_DIR_PATH" >> $GITHUB_ENV
+          echo "APK_VERSION_TAG=$APK_VERSION_TAG" >> $GITHUB_ENV
+          echo "APK_BASENAME_PREFIX=$APK_BASENAME_PREFIX" >> $GITHUB_ENV
+
+          echo "Building APKs for '$RELEASE_VERSION_NAME' build"
+          export TERMUX_APP_VERSION_NAME="${RELEASE_VERSION_NAME/v/}" # Used by app/build.gradle
+          export TERMUX_APK_VERSION_TAG="$APK_VERSION_TAG" # Used by app/build.gradle
+          if ! ./gradlew assembleDebug; then
+            exit_on_error "Build failed for '$RELEASE_VERSION_NAME' build."
+          fi
+
+          echo "Validating APKs"
+          for abi in universal arm64-v8a armeabi-v7a x86_64 x86; do
+            if ! test -f "$APK_DIR_PATH/${APK_BASENAME_PREFIX}_$abi.apk"; then
+              files_found="$(ls "$APK_DIR_PATH")"
+              exit_on_error "Failed to find built APK at '$APK_DIR_PATH/${APK_BASENAME_PREFIX}_$abi.apk'. Files found: "$'\n'"$files_found"
+            fi
+          done
+
+          echo "Generating sha25sums file"
+          if ! (cd "$APK_DIR_PATH"; sha256sum \
+            "${APK_BASENAME_PREFIX}_universal.apk" \
+            "${APK_BASENAME_PREFIX}_arm64-v8a.apk" \
+            "${APK_BASENAME_PREFIX}_armeabi-v7a.apk" \
+            "${APK_BASENAME_PREFIX}_x86_64.apk" \
+            "${APK_BASENAME_PREFIX}_x86.apk" \
+            > sha256sums); then
+            exit_on_error "Generate sha25sums failed for '$RELEASE_VERSION_NAME' release."
+          fi
+
+      - name: Attach universal APK file
+        uses: actions/upload-artifact@v2
+        with:
+          name: ${{ env.APK_BASENAME_PREFIX }}_universal
+          path: |
+            ${{ env.APK_DIR_PATH }}/${{ env.APK_BASENAME_PREFIX }}_universal.apk
+            ${{ env.APK_DIR_PATH }}/output-metadata.json
+
+      - name: Attach arm64-v8a APK file
+        uses: actions/upload-artifact@v2
+        with:
+          name: ${{ env.APK_BASENAME_PREFIX }}_arm64-v8a
+          path: |
+            ${{ env.APK_DIR_PATH }}/${{ env.APK_BASENAME_PREFIX }}_arm64-v8a.apk
+            ${{ env.APK_DIR_PATH }}/output-metadata.json
+
+      - name: Attach armeabi-v7a APK file
+        uses: actions/upload-artifact@v2
+        with:
+          name: ${{ env.APK_BASENAME_PREFIX }}_armeabi-v7a
+          path: |
+            ${{ env.APK_DIR_PATH }}/${{ env.APK_BASENAME_PREFIX }}_armeabi-v7a.apk
+            ${{ env.APK_DIR_PATH }}/output-metadata.json
+
+      - name: Attach x86_64 APK file
+        uses: actions/upload-artifact@v2
+        with:
+          name: ${{ env.APK_BASENAME_PREFIX }}_x86_64
+          path: |
+            ${{ env.APK_DIR_PATH }}/${{ env.APK_BASENAME_PREFIX }}_x86_64.apk
+            ${{ env.APK_DIR_PATH }}/output-metadata.json
+
+      - name: Attach x86 APK file
+        uses: actions/upload-artifact@v2
+        with:
+          name: ${{ env.APK_BASENAME_PREFIX }}_x86
+          path: |
+            ${{ env.APK_DIR_PATH }}/${{ env.APK_BASENAME_PREFIX }}_x86.apk
+            ${{ env.APK_DIR_PATH }}/output-metadata.json
+
+      - name: Attach sha256sums file
+        uses: actions/upload-artifact@v2
+        with:
+          name: sha256sums
+          path: |
+            ${{ env.APK_DIR_PATH }}/sha256sums
+            ${{ env.APK_DIR_PATH }}/output-metadata.json
diff --git a/.github/workflows/trigger_library_builds_on_jitpack.yml b/.github/workflows/trigger_library_builds_on_jitpack.yml
@@ -6,19 +6,16 @@ on:
       - published
 
 jobs:
-  build:
+  trigger-termux-library-builds:
     runs-on: ubuntu-latest
     steps:
-      - name: Set tag
-        id: vars
-        run: echo ::set-output name=tag::${GITHUB_REF/refs\/tags\/v/}
-      - name: Echo tag
-        run: echo "Triggering termux library builds on jitpack for '${{ steps.vars.outputs.tag }}' after waiting for 3 mins"
+      - name: Set vars
+        run: echo "TERMUX_LIB_VERSION=${GITHUB_REF/refs\/tags\/v/}" >> $GITHUB_ENV # Do not include "v" prefix
+      - name: Echo release
+        run: echo "Triggering termux library builds on jitpack for '$TERMUX_LIB_VERSION' release after waiting for 3 mins"
       - name: Trigger termux library builds on jitpack
-        env:
-          RELEASE_VERSION: ${{ steps.vars.outputs.tag }}
         run: |
           sleep 180 # It will take some time for the new tag to be detected by Jitpack
-          curl --max-time 600 --no-progress-meter https://jitpack.io/com/termux/termux-app/terminal-emulator/$RELEASE_VERSION/terminal-emulator-$RELEASE_VERSION.pom
-          curl --max-time 600 --no-progress-meter https://jitpack.io/com/termux/termux-app/terminal-view/$RELEASE_VERSION/terminal-view-$RELEASE_VERSION.pom
-          curl --max-time 600 --no-progress-meter https://jitpack.io/com/termux/termux-app/termux-shared/$RELEASE_VERSION/termux-shared-$RELEASE_VERSION.pom
+          curl --max-time 600 --no-progress-meter "https://jitpack.io/com/termux/termux-app/terminal-emulator/$TERMUX_LIB_VERSION/terminal-emulator-$TERMUX_LIB_VERSION.pom"
+          curl --max-time 600 --no-progress-meter "https://jitpack.io/com/termux/termux-app/terminal-view/$TERMUX_LIB_VERSION/terminal-view-$TERMUX_LIB_VERSION.pom"
+          curl --max-time 600 --no-progress-meter "https://jitpack.io/com/termux/termux-app/termux-shared/$TERMUX_LIB_VERSION/termux-shared-$TERMUX_LIB_VERSION.pom"
diff --git a/app/build.gradle b/app/build.gradle
@@ -5,6 +5,10 @@ plugins {
 android {
     compileSdkVersion project.properties.compileSdkVersion.toInteger()
     ndkVersion = System.getenv("JITPACK_NDK_VERSION") ?: project.properties.ndkVersion
+    def appVersionName = System.getenv("TERMUX_APP_VERSION_NAME") ?: ""
+    def apkVersionTag = System.getenv("TERMUX_APK_VERSION_TAG") ?: ""
+    def splitAPKsForDebugBuilds = System.getenv("TERMUX_SPLIT_APKS_FOR_DEBUG_BUILDS") ?: "1"
+    def splitAPKsForReleaseBuilds = System.getenv("TERMUX_SPLIT_APKS_FOR_RELEASE_BUILDS") ?: "0" // F-Droid does not support split APKs #1904
 
     dependencies {
         implementation "androidx.annotation:annotation:1.2.0"
@@ -27,7 +31,10 @@ android {
         minSdkVersion project.properties.minSdkVersion.toInteger()
         targetSdkVersion project.properties.targetSdkVersion.toInteger()
         versionCode 117
-        versionName "0.117"
+        versionName "0.117.0"
+
+        if (appVersionName) versionName = appVersionName
+        validateVersionName(versionName)
 
         manifestPlaceholders.TERMUX_PACKAGE_NAME = "com.termux"
         manifestPlaceholders.TERMUX_APP_NAME = "Termux"
@@ -46,7 +53,8 @@ android {
 
         splits {
             abi {
-                enable gradle.startParameter.taskNames.any { it.contains("Debug") }
+                enable ((gradle.startParameter.taskNames.any { it.contains("Debug") } && splitAPKsForDebugBuilds == "1") ||
+                    (gradle.startParameter.taskNames.any { it.contains("Release") } && splitAPKsForReleaseBuilds == "1"))
                 reset ()
                 include 'x86', 'x86_64', 'armeabi-v7a', 'arm64-v8a'
                 universalApk true
@@ -106,8 +114,10 @@ android {
         variant.outputs.all { output ->
             if (variant.buildType.name == "debug") {
                 def abi = output.getFilter(com.android.build.OutputFile.ABI)
-                def releaseTag = System.getenv("RELEASE_TAG")
-                outputFileName = new File("termux-app-" + (abi ? abi : "universal") + (releaseTag ? "-" + releaseTag : "") + "-debug.apk")
+                outputFileName = new File("termux-app_" + (apkVersionTag ? apkVersionTag : "debug") + "_" + (abi ? abi : "universal") + ".apk")
+            } else if (variant.buildType.name == "release") {
+                def abi = output.getFilter(com.android.build.OutputFile.ABI)
+                outputFileName = new File("termux-app_" + (apkVersionTag ? apkVersionTag : "release") + "_" + (abi ? abi : "universal") + ".apk")
             }
         }
     }
@@ -125,6 +135,13 @@ task versionName {
     }
 }
 
+def validateVersionName(String versionName) {
+    // https://semver.org/spec/v2.0.0.html#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string
+    // ^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$
+    if (!java.util.regex.Pattern.matches("^(0|[1-9]\\d*)\\.(0|[1-9]\\d*)\\.(0|[1-9]\\d*)(?:-((?:0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\\.(?:0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\\+([0-9a-zA-Z-]+(?:\\.[0-9a-zA-Z-]+)*))?\$", versionName))
+        throw new GradleException("The versionName '"  + versionName + "' is not a valid version as per semantic version '2.0.0' spec in the format 'major.minor.patch(-prerelease)(+buildmetadata)'. https://semver.org/spec/v2.0.0.html.")
+}
+
 def downloadBootstrap(String arch, String expectedChecksum, String version) {
     def digest = java.security.MessageDigest.getInstance("SHA-256")
 
