Low Code Airflow

Workflow (Spark)

Here you can choose the spark workflow. This will generate an appropriate airflow operator depending on the fabric. The intent is to run the spark workflow in batch mode, essentially a spark-submit <cpu-memory-for-spark> <jar-files> <WorkflowMainClass> <workflow-args>

You can also choose job size, which indicates how much resources will be allocated while running this workflow.

You can also provide runtime configurations to the workflow. Here you can use airflow macros for the values.