1) Run diff as part of CI (on code change), that can be done via Github workflow (or other CI). The Github workflow takes in a YAML config file, and also is customizable so you can fit it according your needs. For example, one of our customers names SQL files after tables they populate, so when the bot sees a changed SQL file, it knows what table to compare it to.
2) If you are looking to run a diff within the data pipeline itself (e.g. in Airflow), you can wrap Datafold API in an Airflow operator to integrate it as a task in your DAG.
I hope that I covered your questions but please let me know if you are interested in specific use cases!
1) Run diff as part of CI (on code change), that can be done via Github workflow (or other CI). The Github workflow takes in a YAML config file, and also is customizable so you can fit it according your needs. For example, one of our customers names SQL files after tables they populate, so when the bot sees a changed SQL file, it knows what table to compare it to.
2) If you are looking to run a diff within the data pipeline itself (e.g. in Airflow), you can wrap Datafold API in an Airflow operator to integrate it as a task in your DAG.
I hope that I covered your questions but please let me know if you are interested in specific use cases!