use case 1

Showcase gradually the lengthy configuration evolve to simpler skeleton by using ref element to externalized task/flow. Please also checkout the changes of: use case 2

Externalized the task definition - c0060-task-ref.yml

    flow:
      - func: shell
        do:
          - echo "i am a very complicated flow of step1"
          - echo "i am a very complicated flow of step2"
          - echo "i am a very complicated flow of step3"
    
      - func: shell
        do:
          - echo "i am a very complicated flow of step4"
          - echo "{{.student_name}}"
    

Flow element

Introduce the flow element to notate a task reference

With ref element in main tasks file, this will make the entry point much cleaner and the main skeleton design stand out

Demo

source

Main task yaml file
    vars:
      student_name: tom
      classes: [1k, 2b, 3j, 4s]
    tasks:
    - name: task
      task:
      - func: shell
        do:
        - echo "task step 1"
        - echo "task step 2"
      - func: call
        do:
        - task_a
        - task_c
        - task_a
        if: '{{eq .student_name "tom"}}'
      - func: call
        do:
        - task_b
        loop: "classes"
        if: '{{eq .student_name "tom"}}'
      - func: shell
        do:
        - echo "task step 3"
        - echo "task step 4"
      - func: call
        do:
        - a_very_complicated_task
    - name: task_a
      task:
      - func: shell
        do:
        - echo "task_a_step1"
    - name: task_c
      task:
      - func: shell
        do:
        - echo "task_c_step1"
    - name: task_b
      task:
      - func: shell
        do:
        - echo "task_b_step1"
        - echo "{{.loopindex}} -> student nameed {{.student_name}} has been in class [{{.loopitem}}]"
    - name: a_very_complicated_task
      desc: "a_very_complicated_task"
      ref: c0060-task-ref.yml
    
Main log file
    loading [Config]:  ./tests/functests/upconfig.yml
    Main config:
                 Version -> 1.0.0
                  RefDir -> ./tests/functests
                 WorkDir -> cwd
              AbsWorkDir -> /up_project/up
                TaskFile -> c0060
                 Verbose -> vvv
              ModuleName -> self
               ShellType -> /bin/sh
           MaxCallLayers -> 8
                 Timeout -> 3600000
     MaxModuelCallLayers -> 256
               EntryTask -> task
      ModRepoUsernameRef -> 
      ModRepoPasswordRef -> 
    work dir: /up_project/up
    -exec task: task
    loading [Task]:  ./tests/functests/c0060
    module: [self], instance id: [dev], exec profile: []
    profile -  envVars:
    
    (*core.Cache)({
    })
    
    loading [flow ref]:  ./tests/functests/c0060-task-ref.yml
    Task1: [task ==> task:  ]
    -Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "up_runtime_task_layer_number": 0,
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      }
    })
    
    cmd( 1):
    echo "task step 1"
    
    -
    task step 1
    
    -
     .. ok
    cmd( 2):
    echo "task step 2"
    
    -
    task step 2
    
    -
     .. ok
    . ok
    -Step2:
    self: final context exec vars:
    
    (*core.Cache)({
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task step 2\"",
        Code: 0,
        Output: "task step 2",
        ErrMsg: ""
      }),
      "up_runtime_task_layer_number": 0
    })
    
    =Task2: [task ==> task_a:  ]
    --Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "up_runtime_task_layer_number": 1,
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task step 2\"",
        Code: 0,
        Output: "task step 2",
        ErrMsg: ""
      })
    })
    
    cmd( 1):
    echo "task_a_step1"
    
    -
    task_a_step1
    
    -
     .. ok
    . ok
    =Task3: [task ==> task_c:  ]
    --Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task_a_step1\"",
        Code: 0,
        Output: "task_a_step1",
        ErrMsg: ""
      }),
      "up_runtime_task_layer_number": 1
    })
    
    cmd( 1):
    echo "task_c_step1"
    
    -
    task_c_step1
    
    -
     .. ok
    . ok
    =Task2: [task ==> task_a:  ]
    --Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task_c_step1\"",
        Code: 0,
        Output: "task_c_step1",
        ErrMsg: ""
      }),
      "up_runtime_task_layer_number": 1
    })
    
    cmd( 1):
    echo "task_a_step1"
    
    -
    task_a_step1
    
    -
     .. ok
    . ok
    -Step3:
    self: final context exec vars:
    
    (*core.Cache)({
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task step 2\"",
        Code: 0,
        Output: "task step 2",
        ErrMsg: ""
      }),
      "up_runtime_task_layer_number": 1,
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      }
    })
    
    =Task4: [task ==> task_b:  ]
    --Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "loopindex1": 1,
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task step 2\"",
        Code: 0,
        Output: "task step 2",
        ErrMsg: ""
      }),
      "up_runtime_task_layer_number": 1,
      "loopitem": "1k",
      "loopindex": 0
    })
    
    cmd( 1):
    echo "task_b_step1"
    
    -
    task_b_step1
    
    -
     .. ok
    cmd( 2):
    echo "{{.loopindex}} -> student nameed {{.student_name}} has been in class [{{.loopitem}}]"
    
    -
    0 -> student nameed tom has been in class [1k]
    
    -
     .. ok
    . ok
    =Task4: [task ==> task_b:  ]
    --Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"0 -> student nameed tom has been in class [1k]\"",
        Code: 0,
        Output: "0 -> student nameed tom has been in class [1k]",
        ErrMsg: ""
      }),
      "up_runtime_task_layer_number": 1,
      "loopitem": "2b",
      "loopindex": 1,
      "loopindex1": 2,
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      }
    })
    
    cmd( 1):
    echo "task_b_step1"
    
    -
    task_b_step1
    
    -
     .. ok
    cmd( 2):
    echo "{{.loopindex}} -> student nameed {{.student_name}} has been in class [{{.loopitem}}]"
    
    -
    1 -> student nameed tom has been in class [2b]
    
    -
     .. ok
    . ok
    =Task4: [task ==> task_b:  ]
    --Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"1 -> student nameed tom has been in class [2b]\"",
        Code: 0,
        Output: "1 -> student nameed tom has been in class [2b]",
        ErrMsg: ""
      }),
      "up_runtime_task_layer_number": 1,
      "loopitem": "3j",
      "loopindex": 2,
      "loopindex1": 3
    })
    
    cmd( 1):
    echo "task_b_step1"
    
    -
    task_b_step1
    
    -
     .. ok
    cmd( 2):
    echo "{{.loopindex}} -> student nameed {{.student_name}} has been in class [{{.loopitem}}]"
    
    -
    2 -> student nameed tom has been in class [3j]
    
    -
     .. ok
    . ok
    =Task4: [task ==> task_b:  ]
    --Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "loopindex1": 4,
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"2 -> student nameed tom has been in class [3j]\"",
        Code: 0,
        Output: "2 -> student nameed tom has been in class [3j]",
        ErrMsg: ""
      }),
      "up_runtime_task_layer_number": 1,
      "loopitem": "4s",
      "loopindex": 3
    })
    
    cmd( 1):
    echo "task_b_step1"
    
    -
    task_b_step1
    
    -
     .. ok
    cmd( 2):
    echo "{{.loopindex}} -> student nameed {{.student_name}} has been in class [{{.loopitem}}]"
    
    -
    3 -> student nameed tom has been in class [4s]
    
    -
     .. ok
    . ok
    -Step4:
    self: final context exec vars:
    
    (*core.Cache)({
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task step 2\"",
        Code: 0,
        Output: "task step 2",
        ErrMsg: ""
      }),
      "up_runtime_task_layer_number": 1,
      "student_name": "tom"
    })
    
    cmd( 1):
    echo "task step 3"
    
    -
    task step 3
    
    -
     .. ok
    cmd( 2):
    echo "task step 4"
    
    -
    task step 4
    
    -
     .. ok
    . ok
    -Step5:
    self: final context exec vars:
    
    (*core.Cache)({
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task step 4\"",
        Code: 0,
        Output: "task step 4",
        ErrMsg: ""
      }),
      "up_runtime_task_layer_number": 1
    })
    
    =Task5: [task ==> a_very_complicated_task: a_very_complicated_task ]
    --Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task step 4\"",
        Code: 0,
        Output: "task step 4",
        ErrMsg: ""
      }),
      "up_runtime_task_layer_number": 1
    })
    
    cmd( 1):
    echo "i am a very complicated flow of step1"
    
    -
    i am a very complicated flow of step1
    
    -
     .. ok
    cmd( 2):
    echo "i am a very complicated flow of step2"
    
    -
    i am a very complicated flow of step2
    
    -
     .. ok
    cmd( 3):
    echo "i am a very complicated flow of step3"
    
    -
    i am a very complicated flow of step3
    
    -
     .. ok
    . ok
    --Step2:
    self: final context exec vars:
    
    (*core.Cache)({
      "up_runtime_task_layer_number": 1,
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"i am a very complicated flow of step3\"",
        Code: 0,
        Output: "i am a very complicated flow of step3",
        ErrMsg: ""
      })
    })
    
    cmd( 1):
    echo "i am a very complicated flow of step4"
    
    -
    i am a very complicated flow of step4
    
    -
     .. ok
    cmd( 2):
    echo "{{.student_name}}"
    
    -
    tom
    
    -
     .. ok
    . ok
    
Logs with different verbose level
Raw logs with different verbose level