externalize settings

This demos how to externalize the lengthy vars configuration to a reference file, load it during the runtime

Why

It is ok to just put the a few vars configuration entries together in paralelle with tasks in one yml file. With project growing, we offten need to make the code tidy, clean and modulized. In our case, it refers to yml configurations. It will significantlly reduce the line of codes before you could have a quick glimpse of skeleton of the tasks

In this demo, we show that the global vars all go to the referece yml file in refdir. The refdir could be a global configurable settting apply to all

Application

You could configure the group relationship in scope element and offload the details to separate files

For example: You could use follow directory structure to manage your vars:

scope:\
    global.yml          -> store all global vars
    prod:\              -> prod group
      dr:\
        dr.yml
      prod:\
        prod.yml
        db.yml
    nonprod:\           -> nonprod group
      dev:\
        dev.yml         -> dev settings
      st:\
        st.yml
      staging:\
        staging.yml
      db.yml            -> common db setttings for all nonprod group

d0009-global.yml reference

    vars:
      a: global-a
      b: global-b
      c: global-c
      d: global-d
    

d0009-dev.yml reference

    vars:
      a: dev-a
      c: dev-c
    

Demo

source

Main task yaml file
    scopes:
    - name: global
      ref: d0009-global.yml
      refdir: ./tests/functests
    - name: prod
      members: [dr, prod]
      vars:
        a: prod-a
        c: prod-c
    - name: nonprod
      members:
      - dev
      - st
      - staging
      vars:
        a: non-prod-a
        b: non-prod-b
        c: non-prod-c
    - name: staging
      vars:
        a: staging-a
        b: staging-b
    - name: dev
      ref: d0009-dev.yml
    tasks:
    - name: task
      task:
      - func: shell
        do:
        - echo "test out the var scopes only"
    
Main log file
    loading [Config]:  ./tests/functests/upconfig.yml
    Main config:
                 Version -> 1.0.0
                  RefDir -> ./tests/functests
                 WorkDir -> cwd
              AbsWorkDir -> /up_project/up
                TaskFile -> c0009
                 Verbose -> vvv
              ModuleName -> self
               ShellType -> /bin/sh
           MaxCallLayers -> 8
                 Timeout -> 3600000
     MaxModuelCallLayers -> 256
               EntryTask -> task
      ModRepoUsernameRef -> 
      ModRepoPasswordRef -> 
    work dir: /up_project/up
    -exec task: task
    loading [Task]:  ./tests/functests/c0009
    module: [self], instance id: [dev], exec profile: []
    profile -  envVars:
    
    (*core.Cache)({
    })
    
    loading [ref vars]:  ./tests/functests/d0009-global.yml
    loading [ref vars]:  ./tests/functests/d0009-dev.yml
    Task1: [task ==> task:  ]
    -Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "c": "dev-c",
      "d": "global-d",
      "a": "dev-a",
      "b": "non-prod-b",
      "up_runtime_task_layer_number": 0
    })
    
    cmd( 1):
    echo "test out the var scopes only"
    
    -
    test out the var scopes only
    
    -
     .. ok
    . ok
    
Logs with different verbose level
Raw logs with different verbose level