[Question]: IndexError while trying to stream a pg_dump to storage

[Ekerot]

Describe your question

I try to run this in a pipeline to take long retention backups for our databases.
When running with databases having a bit more data in it I get this unexpected error while running the following:

export SKIP=(
    azure_maintenance 
    azure_sys 
    postgres 
    template0 
    template1
)

run_dump() {
    pg_dump -Fc -Z6 "dbname=$dbname \
        host=$SERVER.postgres.database.azure.com port=5432 \
        sslrootcert=./scripts/combined.pem sslmode=verify-ca user=$PGUSER"
}

upload() {
    az storage blob upload --data @- --container-name backups \
        --name "$DATE-$dbname-$VERTICAL-$ENV" --account-name "$ACCOUNT" --sas-token "$BLOBTOKEN" 
}

for dbname in $(dblist); do
    if [[ " ${SKIP[*]} " =~ " ${dbname} " ]]; then
        echo "Not applicable "$dbname"!"    
    else
        echo "Backing up '$dbname'..."
        run_dump | upload
    fi
done

ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: 
Traceback (most recent call last):
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 106, in _expand_file_prefix
    return '='.join([arg_split[0], _maybe_load_file(arg_split[1])])
                                                    ~~~~~~~~~^^^
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/az/lib/python3.11/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 514, in execute
    args = _pre_command_table_create(self.cli_ctx, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 115, in _pre_command_table_create
    return _expand_file_prefixed_files(args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 110, in _expand_file_prefixed_files
    return [_expand_file_prefix(arg) for arg in args]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 110, in <listcomp>
    return [_expand_file_prefix(arg) for arg in args]
            ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 108, in _expand_file_prefix
    return _maybe_load_file(arg_split[0])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 95, in _maybe_load_file
    return _load_file(poss_file)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 79, in _load_file
    content = sys.stdin.read()
              ^^^^^^^^^^^^^^^^

Versions

Cloud version

Environment type (Please select at least one enviroment where you face this issue)

• Self-Hosted
• Microsoft Hosted
• VMSS Pool
• Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Operation system

Ubuntu 22.04

Version controll system

No response

Azure DevOps Server Version (if applicable)

No response

[vmapetr]

Hi @Ekerot thanks for reporting!
We are working on more prioritized issues at the moment, but will get back to this one soon.