AppImage support with release workflow

.github/workflows/app-release.yml

@@ -0,0 +1,45 @@
+name: Release Scan AppImage
+
+on:
+  push:
+    tags:
+    - 'v*'
+
+jobs:
+  build:
+    runs-on: ubuntu-20.04
+    steps:
+    - uses: actions/checkout@v2
+    - name: Install dependencies
+      run: |
+        sudo apt-get update -y
+        sudo apt-get install -y appstream python3.8 python3.8-dev python3-pip python3-setuptools patchelf desktop-file-utils libgdk-pixbuf2.0-dev
+        # Install appimagetool AppImage
+        sudo wget https://github.com/AppImage/AppImageKit/releases/download/continuous/appimagetool-x86_64.AppImage -O /usr/local/bin/appimagetool
+        sudo chmod +x /usr/local/bin/appimagetool
+        sudo pip3 install git+https://github.com/AppImageCrafters/appimage-builder.git
+    - name: Build AppImage
+      run: |
+        appimage-builder --recipe appimage-builder.yml --skip-test
+      env:
+        UPDATE_INFO: gh-releases-zsync|ShiftLeftSecurity|sast-scan|latest|*x86_64.AppImage.zsync
+    - name: Create Release
+      id: create_release
+      uses: actions/create-release@v1
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      with:
+        tag_name: ${{ github.ref }}
+        release_name: Release ${{ github.ref }}
+        draft: false
+        prerelease: false
+    - name: Upload Release Asset
+      id: upload-release-asset
+      uses: actions/upload-release-asset@v1
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      with:
+        upload_url: ${{ steps.create_release.outputs.upload_url }}
+        asset_path: ./*.AppImage*
+        asset_name: scan
+        asset_content_type:  	application/octet-stream

.github/workflows/appimage.yml

@@ -0,0 +1,26 @@
+name: Scan AppImage
+
+on: [push]
+
+jobs:
+  build:
+    runs-on: ubuntu-20.04
+    steps:
+    - uses: actions/checkout@v2
+    - name: Install dependencies
+      run: |
+        sudo apt-get update -y
+        sudo apt-get install -y appstream python3.8 python3.8-dev python3-pip python3-setuptools patchelf desktop-file-utils libgdk-pixbuf2.0-dev
+        # Install appimagetool AppImage
+        sudo wget https://github.com/AppImage/AppImageKit/releases/download/continuous/appimagetool-x86_64.AppImage -O /usr/local/bin/appimagetool
+        sudo chmod +x /usr/local/bin/appimagetool
+        sudo pip3 install git+https://github.com/AppImageCrafters/appimage-builder.git
+    - name: Build AppImage
+      run: |
+        appimage-builder --recipe appimage-builder.yml --skip-test
+      env:
+        UPDATE_INFO: gh-releases-zsync|ShiftLeftSecurity|sast-scan|latest|*x86_64.AppImage.zsync
+    - uses: actions/upload-artifact@v2
+      with:
+        name: AppImage
+        path: './*.AppImage*'

.gitignore

@@ -136,3 +136,5 @@ dmypy.json
 .pyre/
 .coverage
 reports/
+AppDir/
+appimage-builder-cache/

README.md

@@ -9,7 +9,7 @@
 ╚══════╝╚═╝  ╚═╝╚═╝╚═╝        ╚═╝   ╚══════╝╚══════╝╚═╝        ╚═╝       ╚══════╝ ╚═════╝╚═╝  ╚═╝╚═╝  ╚═══╝
 ```
 
-ShiftLeft Scan is a free open-source security tool for modern DevOps teams. This repo builds `shiftleft/sast-scan`, a container image that powers the ShiftLeft Scan product. Scan products are open-source under a GNU GPL 3.0 or later (GPL-3.0-or-later) license.
+ShiftLeft Scan is a free open-source security tool for modern DevOps teams. With an integrated multi-scanner based design, ShiftLeft Scan can detect various kinds of security flaws in your application and infrastructure code in a single fast scan. Scan products are open-source under a GNU GPL 3.0 or later (GPL-3.0-or-later) license.
 
 [![Build Status](https://dev.azure.com/shiftleftsecurity/sl-appthreat/_apis/build/status/ShiftLeftSecurity.sast-scan?branchName=master)](https://dev.azure.com/shiftleftsecurity/sl-appthreat/_build/latest?definitionId=11&branchName=master)
 
@@ -52,16 +52,24 @@ ShiftLeft Scan is a free open-source security tool for modern DevOps teams. This
 - Node.js 10
 - Yarnpkg
 
+Please visit the official [documentation](https://slscan.io) site for scan to learn about the configuration and CI/CD integration options.
+
 ## Getting started
 
-scan is ideal for use with CI and also as a pre-commit hook for local development.
+scan is ideal for use with CI and also as a pre-commit hook for local development. Scan is distributed as a container image `shiftleft/scan`, and as an AppImage for supported Linux distributions.
 
 ### Scanning projects locally
 
-Scan Python project
+Easy one-liner command below:
 
 ```bash
-docker run --rm -e "WORKSPACE=${PWD}" -v $PWD:/app shiftleft/scan scan --src /app --type python
+sh <(curl https://slscan.sh)
+```
+
+The above command simply invokes the below docker run command.
+
+```bash
+docker run --rm -e "WORKSPACE=${PWD}" -v $PWD:/app shiftleft/scan scan --build
 ```
 
 On Windows, the command changes slightly depending on the terminal.
@@ -113,10 +121,6 @@ docker run --rm -e "WORKSPACE=${PWD}" -v ~/.gradle:/.gradle -v <source path>:/ap
 
 Feel free to skip `--type` to enable auto-detection. Or pass comma-separated values if the project has multiple types.
 
-### Detailed documentation
-
-Please visit the official [documentation](https://slscan.io) site for scan to learn about the configuration and CI/CD integration options.
-
 ## Viewing reports
 
 Reports would be produced in the directory specified for `--out_dir`. In the above examples, it is set to `reports` which will be a directory under the source code root directory.

app-build.sh

@@ -0,0 +1,6 @@
+#!/usr/bin/env bash
+
+rm -rf AppDir appimage-builder-cache
+rm *.AppImage*
+UPDATE_INFO="gh-releases-zsync|ShiftLeftSecurity|sast-scan|latest|*x86_64.AppImage.zsync" appimage-builder --recipe appimage-builder.yml --skip-test
+rm -rf AppDir appimage-builder-cache

appimage-builder.yml

@@ -0,0 +1,101 @@
+version: 1
+script:
+  # Remove any previous build
+  - rm -rf AppDir  | true
+  # Make usr and icons dirs
+  - mkdir -p AppDir/usr/src
+  - mkdir -p AppDir/usr/local/lib
+  - mkdir -p AppDir/usr/share/{metainfo,icons}
+  # Copy the python application code into the AppDir
+  - cp appimage-reqs.sh scan lib tools_config AppDir/usr/src -r
+  - cp tools_config/scan.png AppDir/usr/share/icons/
+  # Install application dependencies
+  - python3 -m pip install --no-cache-dir --ignore-installed --prefix=/usr --root=AppDir -r ./requirements.txt
+  - mv AppDir/usr/bin/scan AppDir/usr/bin/depscan
+  - chmod +x AppDir/usr/src/appimage-reqs.sh && AppDir/usr/src/appimage-reqs.sh AppDir
+  - npm install --only=production --no-save --prefix AppDir/usr/local/lib yarn @appthreat/cdxgen @microsoft/rush
+  - mkdir -p AppDir/opt/phpsast && cd AppDir/opt/phpsast && composer init --quiet && composer require --quiet --no-cache --dev vimeo/psalm
+  - cd AppDir/opt/phpsast && composer require --quiet --no-cache --dev phpstan/phpstan
+
+AppDir:
+  path: ./AppDir
+
+  app_info:
+    id: io.shiftleft.scan
+    name: scan
+    summary: ShiftLeft Scan is a free open-source security tool for modern DevOps teams
+    metadata_license: FSFAP
+    project_license: GPL-3.0-or-later
+    project_group: ShiftLeftSecurity
+    icon: utilities-terminal
+    version: 1.7.0
+    # Set the python executable as entry point
+    exec: usr/bin/python3.8
+    # Set the application main script path as argument. Use '$@' to forward CLI parameters
+    exec_args: "$APPDIR/usr/src/scan $@"
+
+  apt:
+    arch: amd64
+    sources:
+      - sourceline: 'deb [arch=amd64] http://archive.ubuntu.com/ubuntu/ bionic main restricted universe multiverse'
+        key_url: 'http://keyserver.ubuntu.com/pks/lookup?op=get&search=0x3b4fe6acc0b21f32'
+      - sourceline: deb http://archive.ubuntu.com/ubuntu/ bionic-updates main restricted universe multiverse
+      - sourceline: deb http://archive.ubuntu.com/ubuntu/ bionic-security main restricted universe multiverse
+    include:
+      - python3.8
+      - python3-pkg-resources
+      - git
+      - php
+      - php-json
+      - php-pear
+      - php-mbstring
+      - composer
+      - tar
+      - shellcheck
+      - jq
+  files:
+    exclude:
+    - usr/share/man
+    - usr/share/doc/*/README.*
+    - usr/share/doc/*/changelog.*
+    - usr/share/doc/*/NEWS.*
+    - usr/share/doc/*/TODO.*
+
+  runtime:
+    env:
+      PATH: '${APPDIR}/usr/bin:${APPDIR}/usr/bin/nodejs:${PATH}:${APPDIR}/opt/phpsast/vendor/bin:${APPDIR}/usr/local/lib/node_modules/.bin:'
+      PYTHONHOME: '${APPDIR}/usr'
+      PYTHONPATH: '${APPDIR}/usr/lib/python3.8/site-packages'
+      PYTHONUNBUFFERED: '1'
+      APP_SRC_DIR: '${APPDIR}/usr/src'
+      TOOLS_CONFIG_DIR: '${APPDIR}/usr/src/tools_config'
+      DEPSCAN_CMD: '${APPDIR}/usr/bin/depscan'
+      PMD_CMD: '${APPDIR}/opt/pmd-bin/bin/run.sh pmd'
+      SPOTBUGS_HOME: '${APPDIR}/opt/spotbugs'
+
+  test:
+    fedora:
+      image: appimagecrafters/tests-env:fedora-30
+      command: ./AppRun --help
+      use_host_x: true
+    debian:
+      image: appimagecrafters/tests-env:debian-stable
+      command: ./AppRun --help
+      use_host_x: true
+    arch:
+      image: appimagecrafters/tests-env:archlinux-latest
+      command: ./AppRun --help
+      use_host_x: true
+    centos:
+      image: appimagecrafters/tests-env:centos-7
+      command: ./AppRun --help
+      use_host_x: true
+    ubuntu:
+      image: appimagecrafters/tests-env:ubuntu-xenial
+      command: ./AppRun --help
+      use_host_x: true
+
+AppImage:
+  update-information: !ENV ${UPDATE_INFO}
+  sign-key: None
+  arch: x86_64

appimage-reqs.sh

@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+APPDIR=$1
+OPTDIR=${APPDIR}/opt
+GOSEC_VERSION=2.3.0
+TFSEC_VERSION=0.21.0
+KUBESEC_VERSION=2.4.0
+KUBE_SCORE_VERSION=1.7.0
+DETEKT_VERSION=1.10.0
+GITLEAKS_VERSION=4.3.1
+SC_VERSION=2020.1.4
+PMD_VERSION=6.24.0
+FSB_VERSION=1.10.1
+FB_CONTRIB_VERSION=7.4.7
+SB_VERSION=4.0.1
+NODE_VERSION=14.5.0
+export PATH=$PATH:${APPDIR}/usr/bin:
+
+curl -LO "https://nodejs.org/dist/v${NODE_VERSION}/node-v${NODE_VERSION}-linux-x64.tar.xz" \
+    && tar -C ${APPDIR}/usr/bin/ -xvf node-v${NODE_VERSION}-linux-x64.tar.xz \
+    && mv ${APPDIR}/usr/bin/node-v${NODE_VERSION}-linux-x64 ${APPDIR}/usr/bin/nodejs \
+    && chmod +x ${APPDIR}/usr/bin/nodejs/node \
+    && chmod +x ${APPDIR}/usr/bin/nodejs/npm \
+    && rm node-v${NODE_VERSION}-linux-x64.tar.xz
+curl -LO "https://github.com/securego/gosec/releases/download/v${GOSEC_VERSION}/gosec_${GOSEC_VERSION}_linux_amd64.tar.gz" \
+    && tar -C ${APPDIR}/usr/bin/ -xvf gosec_${GOSEC_VERSION}_linux_amd64.tar.gz \
+    && chmod +x ${APPDIR}/usr/bin/gosec \
+    && rm gosec_${GOSEC_VERSION}_linux_amd64.tar.gz
+curl -LO "https://github.com/dominikh/go-tools/releases/download/${SC_VERSION}/staticcheck_linux_amd64.tar.gz" \
+    && tar -C /tmp -xvf staticcheck_linux_amd64.tar.gz \
+    && chmod +x /tmp/staticcheck/staticcheck \
+    && cp /tmp/staticcheck/staticcheck ${APPDIR}/usr/bin/staticcheck \
+    && rm staticcheck_linux_amd64.tar.gz
+curl -L "https://github.com/zricethezav/gitleaks/releases/download/v${GITLEAKS_VERSION}/gitleaks-linux-amd64" -o "${APPDIR}/usr/bin/gitleaks" \
+    && chmod +x ${APPDIR}/usr/bin/gitleaks \
+    && curl -L "https://github.com/liamg/tfsec/releases/download/v${TFSEC_VERSION}/tfsec-linux-amd64" -o "${APPDIR}/usr/bin/tfsec" \
+    && chmod +x ${APPDIR}/usr/bin/tfsec \
+    && rm shellcheck-stable.linux.x86_64.tar.xz
+curl -L "https://github.com/zegl/kube-score/releases/download/v${KUBE_SCORE_VERSION}/kube-score_${KUBE_SCORE_VERSION}_linux_amd64" -o "${APPDIR}/usr/bin/kube-score" \
+    && chmod +x ${APPDIR}/usr/bin/kube-score \
+    && wget "https://github.com/pmd/pmd/releases/download/pmd_releases%2F${PMD_VERSION}/pmd-bin-${PMD_VERSION}.zip" \
+    && unzip -q pmd-bin-${PMD_VERSION}.zip -d ${OPTDIR}/ \
+    && rm pmd-bin-${PMD_VERSION}.zip \
+    && mv ${OPTDIR}/pmd-bin-${PMD_VERSION} ${OPTDIR}/pmd-bin
+curl -L "https://github.com/detekt/detekt/releases/download/v${DETEKT_VERSION}/detekt-cli-${DETEKT_VERSION}-all.jar" -o "${APPDIR}/usr/bin/detekt-cli.jar" \
+    && curl -LO "https://github.com/controlplaneio/kubesec/releases/download/v${KUBESEC_VERSION}/kubesec_linux_amd64.tar.gz" \
+    && tar -C ${APPDIR}/usr/bin/ -xvf kubesec_linux_amd64.tar.gz \
+    && rm kubesec_linux_amd64.tar.gz \
+    && curl -LO "https://repo.maven.apache.org/maven2/com/github/spotbugs/spotbugs/${SB_VERSION}/spotbugs-${SB_VERSION}.zip" \
+    && unzip -q spotbugs-${SB_VERSION}.zip -d ${OPTDIR}/ \
+    && rm spotbugs-${SB_VERSION}.zip
+curl -LO "https://repo1.maven.org/maven2/com/h3xstream/findsecbugs/findsecbugs-plugin/${FSB_VERSION}/findsecbugs-plugin-${FSB_VERSION}.jar" \
+    && mv findsecbugs-plugin-${FSB_VERSION}.jar ${OPTDIR}/spotbugs-${SB_VERSION}/plugin/findsecbugs-plugin.jar \
+    && curl -LO "https://repo1.maven.org/maven2/com/mebigfatguy/fb-contrib/fb-contrib/${FB_CONTRIB_VERSION}/fb-contrib-${FB_CONTRIB_VERSION}.jar" \
+    && mv fb-contrib-${FB_CONTRIB_VERSION}.jar ${OPTDIR}/spotbugs-${SB_VERSION}/plugin/fb-contrib.jar \
+    && mv ${OPTDIR}/spotbugs-${SB_VERSION} ${OPTDIR}/spotbugs

azure-pipelines.yml

@@ -1,6 +1,12 @@
 trigger:
-- master
-- feature/*
+  tags:
+    include:
+    - v*
+  branches:
+    include:
+    - master
+    - feature/*
+    - fix/*
 pool:
   vmImage: 'ubuntu-latest'
 variables:

lib/config.py

@@ -25,8 +25,7 @@
 
 # Depth of credscan
 credscan_depth = "5"
-work_dir = (Path(__file__).parent.parent).as_posix()
-credscan_config = os.path.join(work_dir, "credscan-config.toml")
+credscan_config = os.path.join(TOOLS_CONFIG_DIR, "credscan-config.toml")
 credscan_timeout = "2m"
 
 # Php memory limit

lib/executor.py

@@ -95,6 +95,8 @@ def exec_tool(args, cwd=None, env=utils.get_env(), stdout=subprocess.DEVNULL):
         LOG.info("=" * 80)
         LOG.debug('⚡︎ Executing "{}"'.format(" ".join(args)))
         stderr = subprocess.DEVNULL
+        if LOG.isEnabledFor(DEBUG):
+            stderr = subprocess.STDOUT
         cp = subprocess.run(
             args,
             stdout=stdout,

scan

@@ -477,7 +477,7 @@ def main():
     # Check if we should authenticate with inspect
     if not args.nocloud:
         inspect.authenticate()
-
+        config.set("DISABLE_TELEMETRY", "true")
     # Identify project type
     if not type:
         # Check the local config first. If not try auto detection