tencent cloud

Tencent Cloud DataBuddy

DocumentaçãoTencent Cloud DataBuddyData EngineeringCI/CDPractice TutorialTutorial: Git Collaborative Development

Tutorial: Git Collaborative Development

Baixar
Modo Foco
Tamanho da Fonte
Última atualização: 2026-10-09 16:13:46
Traduzido por IA
Tip
Estimated time: 20 minutes. Learning objectives: Connect the Bundle project to a Git repository and configure a CI/CD pipeline for automated deployment.

Scenario Introduction

In team-based development, Code Review and automated pipelines are typically required to ensure deployment quality. This tutorial demonstrates how to connect a Bundle project with a Git repository to enable automatic deployment to the corresponding environment when code is pushed to different branches.
In this tutorial, you will:
1. Initialize the Bundle project as a Git repository and configure .gitignore.
2. Configure a multi-branch policy (dev → prod).
3. Configure key authentication on the CI/CD platform.
4. Write the pipeline configuration file (using GitLab CI and GitHub Actions as examples).
5. Verify the automated deployment process.

Prerequisites

You have completed the Develop with Bundle tutorial and acquired basic Bundle operation skills.
An existing Git repository (GitHub / GitLab / other Git platforms).
You have obtained TencentCloud API credentials (SecretId + SecretKey) for non-interactive authentication in the CI/CD pipeline.
The DataBuddy Studio feature has been activated.
Tip
To obtain API credentials, log in to the Tencent Cloud CAM console to create or view them.

Use Limits

Key-based authentication (non-interactive) is used in CI/CD pipelines, and browser-based login is not supported.
The pipeline Runner environment must be Linux. Installation commands differ on macOS / Windows, and this tutorial uses Linux as an example.
databuddy bundle deploy -s submits changes directly to the scheduling system (production environment). Ensure that approval protection is configured in the pipeline.

Step 1: Planning Branch Policies

This tutorial uses the dev and prod branches as examples to demonstrate the Targets for the development and production environments respectively:
Branch
Corresponding Target
Purpose
dev
dev
Development integration, daily development merge point
prod
prod
Production environment, only accepts MR/PR merges
Note:
The branch policy above is for demonstration purposes only. In practice, teams can design it flexibly based on their own needs. For example, each developer can use their own personal branch for development, and merge changes into the dev or prod branch through MR/PR to trigger deployment to the corresponding environment.

Step 2: Initializing the Git Repository

1. Go to the Bundle project directory and initialize the Git repository:
cd my_bundle
git init
2. Create a .gitignore file to exclude the local state directory:
# DataBuddy CLI state directory (auto-generated, contains binding relationships and local state, and should not be committed)
.bundle/
3. Complete the initial commit:
git add .
git commit -m "chore: init bundle project"
git remote add origin <your repository address>
git push -u origin dev
Note:
The .bundle/ directory contains local binding state and authentication cache, and must be added to .gitignore and not committed to the Git repository.

Step 3: Configuring Key Authentication

Browser-based interactive login cannot be used in CI/CD pipelines, and key-based authentication is required. DataBuddy CLI obtains authentication information by reading the ~/.databuddycfg configuration file.

GitLab CI

Configure CI/CD variables in your GitLab project: Settings → CI/CD → Variables, and add the following 4 variables:
Variable Name
Description
Option
WEDATA_HOST
DataBuddy Workspace address (full URL)
Masked
WEDATA_SECRET_ID
TencentCloud API secret ID
Masked
WEDATA_SECRET_KEY
TencentCloud API secret Key
Masked
WEDATA_REGION
Region, such as ap-beijing-fsi
-

GitHub Actions

Configure Secrets in your GitHub repository: Settings → Secrets and variables → Actions, and add the following 4 Secrets:
Secret name
Description
WEDATA_HOST
DataBuddy Workspace address (full URL)
WEDATA_SECRET_ID
TencentCloud API secret ID
WEDATA_SECRET_KEY
TencentCloud API secret Key
WEDATA_REGION
Region, such as ap-guangzhou
Warning:
Keys must never be written in plaintext to configuration files or committed to Git repositories. Always inject them through CI/CD Variables or Secrets.

Step 4: Writing CI/CD Pipeline Configurations

GitLab CI

Create a .gitlab-ci.yml file in the repository root directory:
stages:
  - validate
  - deploy

# ---- Global Configuration ----
variables:
  BUNDLE_DIR: my_bundle   # Change this to the actual bundle directory name.

before_script:
  # Install DataBuddy CLI (Go Binary Version)
  - curl 'https://wedata3-platform-bundle-cq-1257305158.cos.ap-chongqing.myqcloud.com/bundle_sh/install.sh' | sudo sh || true
  - databuddy --version
  # Write the authentication configuration file (keys are injected from CI/CD Variables).
  - |
    cat << EOF > "$HOME/.databuddycfg"
    host = ${WEDATA_HOST}
    secret_id = ${WEDATA_SECRET_ID}
    secret_key = ${WEDATA_SECRET_KEY}
    region = ${WEDATA_REGION}
    EOF
  - cd ${BUNDLE_DIR}

# ========== dev Branch: Deploy to Development Environment ==========

validate-dev:
  stage: validate
  script:
    - databuddy bundle validate -t dev
  rules:
    - if: '$CI_COMMIT_BRANCH == "dev"'

deploy-dev:
  stage: deploy
  script:
    - databuddy bundle deploy -t dev
  rules:
    - if: '$CI_COMMIT_BRANCH == "dev"'
  environment:
    name: development

# ========== prod Branch: Deploy to Production Environment ==========

validate-prod:
  stage: validate
  script:
    - databuddy bundle validate -t prod
  rules:
    - if: '$CI_COMMIT_BRANCH == "prod"'

deploy-prod:
  stage: deploy
  script:
    - databuddy bundle deploy -t prod -s
  rules:
    - if: '$CI_COMMIT_BRANCH == "prod"'
  environment:
    name: production

GitHub Actions

Create a .github/workflows/deploy.yml file in the repository:
name: DataBuddy Bundle CI/CD

on:
  push:
    branches: [dev, prod]
  pull_request:
    branches: [prod]

env:
  BUNDLE_DIR: my_bundle   # Change this to the actual bundle directory name.

jobs:
  # ========== dev Branch: Deploy to Development Environment ==========
  deploy-dev:
    if: github.ref == 'refs/heads/dev'
    runs-on: ubuntu-latest
    environment: development
    steps:
      - uses: actions/checkout@v4

      - name: Install DataBuddy CLI
        run: |
          curl 'https://wedata3-platform-bundle-cq-1257305158.cos.ap-chongqing.myqcloud.com/bundle_sh/install.sh' | sudo sh || true
          databuddy --version

      - name: Configure credentials
        run: |
          cat << EOF > "$HOME/.databuddycfg"
          host = ${{ secrets.WEDATA_HOST }}
          secret_id = ${{ secrets.WEDATA_SECRET_ID }}
          secret_key = ${{ secrets.WEDATA_SECRET_KEY }}
          region = ${{ secrets.WEDATA_REGION }}
          EOF

      - name: Validate
        run: |
          cd ${{ env.BUNDLE_DIR }}
          databuddy bundle validate -t dev

      - name: Deploy
        run: |
          cd ${{ env.BUNDLE_DIR }}
          databuddy bundle deploy -t dev

  # ========== prod Branch: Deploy to Production Environment ==========
  deploy-prod:
    if: github.ref == 'refs/heads/prod'
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: actions/checkout@v4

      - name: Install DataBuddy CLI
        run: |
          curl 'https://wedata3-platform-bundle-cq-1257305158.cos.ap-chongqing.myqcloud.com/bundle_sh/install.sh' | sudo sh || true
          databuddy --version

      - name: Configure credentials
        run: |
          cat << EOF > "$HOME/.databuddycfg"
          host = ${{ secrets.WEDATA_HOST }}
          secret_id = ${{ secrets.WEDATA_SECRET_ID }}
          secret_key = ${{ secrets.WEDATA_SECRET_KEY }}
          region = ${{ secrets.WEDATA_REGION }}
          EOF

      - name: Validate
        run: |
          cd ${{ env.BUNDLE_DIR }}
          databuddy bundle validate -t prod

      - name: Deploy
        run: |
          cd ${{ env.BUNDLE_DIR }}
          databuddy bundle deploy -t prod -s
Note:
Configure Required Reviewers on GitHub for the production environment to ensure that production deployments require manual approval.

Step 5: Daily Development Collaboration Process

After the pipeline configuration is complete, team members collaborate on a daily basis as follows:
1. Create a feature branch from the dev branch for development:
git checkout dev
git pull origin dev
git checkout -b feature/add-report-task
2. Edit the Bundle configuration and code files, and commit them after local validation passes:
# Local Validation
databuddy bundle validate -t dev -p <profile>

# Commit Code
git add .
git commit -m "feat(tasks): add daily report task"
git push origin feature/add-report-task
3. Create a Merge Request / Pull Request on the Git platform and merge it into the dev branch. The pipeline automatically runs validate and deploy to the development environment.
4. After verification in the development environment passes, create an MR to merge from dev into prod, and merge it after Code Review approval. The pipeline automatically deploys to the production environment.

Step 6: Verifying the Automation Process

After completing the above configuration, perform an end-to-end validation:
1. Modify the Bundle configuration on the feature branch, push it, and then create an MR to merge into dev.
2. Check whether the CI/CD pipeline automatically runs validate + deploy-dev.
3. In the development environment of the DataBuddy console, confirm that the workflow has been updated.
4. Create an MR to merge from dev into prod, and confirm that the deploy-prod pipeline runs successfully.
5. In the production environment of the DataBuddy console, confirm that the workflow has been deployed and is in the scheduled state.

FAQs

validate Failure in the Pipeline

Check the following:
Confirm that the BUNDLE_DIR variable matches the actual Bundle directory name.
Confirm that the four fields host, secret_id, secret_key, and region in ~/.databuddycfg are all correctly injected.
Confirm that the CI Runner network can access the DataBuddy API endpoint.

deploy Failure in the Pipeline

Confirm that the target Target (dev / prod) is correctly defined in databuddy.yml.
Confirm that the account associated with the API key has sufficient permissions in the target Workspace.

How to Roll Back

If an issue occurs in the production deployment:
1. Use git revert to revert the problematic commit.
2. Merge the revert commit into the prod branch.
3. The pipeline automatically runs deploy to restore remote resources to their previous state.

References

Develop with Bundle
Bundle Definition
Bundle Commands
CI/CD Overview
Anterior: Tutorial: Developing with BundlePróximo: Bundle Package Definition

Ajuda e Suporte

Esta página foi útil?

comentários

Search
    Enter keywords to search documentation