CI/CD Integration

Available to: Developers, Admins Minimum plan: Free

When a change to your data platform needs a matching change to reference data, you can make both together: change the code in a Git feature branch and the table definitions in your TitanRDM development branch. Your CI/CD pipeline then keeps them in step:

1. The pull request is merged → the pipeline promotes the matching TitanRDM promotion.
2. The release reaches an environment (Stage, Prod) → the pipeline deploys the TitanRDM changes to the matching environment, before the data platform code that uses them.

TitanRDM's branches, promotions and environments mirror the way most teams run Git and CI/CD, and the way platforms such as Databricks, Microsoft Fabric and Snowflake recommend working:

The pipeline addresses TitanRDM by external id, so it never needs TitanRDM's own ids:

• Promotion external id: the promotion's Pull Request, for example my-org/data-platform#42
• Environment external id: the release stage or deployment target, for example staging or prod

---

Pull Request or Commit?

A promotion's external id is its Pull Request, shown by that name on the promotion pages. You know the pull request when you create the promotion, it never changes, and it links straight to the review. Enter the repository and the pull request number in the same format your pipeline builds it:

You don't need to know the commit. When the pull request is merged, the pipeline sends the merge commit along with the promote request, and TitanRDM records it on the promotion. A release pipeline usually knows only the commit it is releasing, so at release time it can send the commits in the release, and TitanRDM finds the promotions they merged.

---

Before You Start

1. Create a service account integration. In Admin > Integrations, create an application with the api:read and api:write scopes. Under Service Account, choose a service account the application acts as. Create one first with New Service Account in Admin > Users, or enter a name such as GitHub Actions on the form to create one. Copy the client id and secret into your CI/CD system's secret store. See OAuth Applications.
2. Give the service account its permissions. Add it to the Approver group of the branch it promotes into (for example Branch main Approver) and to the Deployer group of each environment it deploys to (for example Environment Stage Deployer). See Permissions.
3. Set external ids on your environments in Admin > Environments, matching your release stages or deployment targets (for example staging, prod).
4. Developers enter the pull request on the promotion. When you create a promotion, enter the pull request in Pull Request, for example my-org/data-platform#42. You can also add it later from the promotion page.

A pull request identifies one active promotion. Once a promotion is cancelled, abandoned or rejected, its pull request can be used by a new promotion.

---

The Endpoints

All three endpoints take TitanRDM's external ids in the request body or query string. Every call is safe to repeat: a promotion that is already promoted, or an environment that is already up to date, is reported, not changed.

Promote: when a pull request is merged

POST /api/v1/cicd/promote
{
  "promotion_external_id": "my-org/data-platform#42",
  "merge_commit_sha": "9f3c2a…",
  "allow_missing": true
}

The result is promoted or already_promoted. The response includes the promotion and the deployments it created, one for each environment the target branch deploys to.

Deploy: when a release reaches an environment

POST /api/v1/cicd/deploy
{
  "environment_external_id": "staging",
  "commit_shas": ["9f3c2a…", "41be07…", "…"],
  "release_ref": "9f3c2a…"
}

TitanRDM deploys every pending deployment for the environment created by promotions up to and including the release's promotion, oldest first. If a release contains pull requests 41 and 42, both are deployed. Pull request 43, merged after the release was built, is not deployed.

Deployments you created yourself (not by a promotion) are never deployed by a release. They are listed in skipped.

Changes promoted after the release. A deployment applies each table as it is now on the branch. If a later pull request changed the same table, deploying the release would apply that later change too, before the code that uses it. TitanRDM refuses with 409 Conflict and lists the tables and later promotions in later_changes. Usually the right thing is to wait: the release that contains the later pull request will deploy both. To deploy anyway, repeat the request with "allow_later_changes": true.

Status: checks and approval gates

GET /api/v1/cicd/status?promotion_external_id=my-org/data-platform%2342
GET /api/v1/cicd/status?environment_external_id=prod

• For a promotion: its status, whether it can be promoted (promotable), its tables, and its deployment to each environment. Use it in a pull request check, for example to fail when the promotion exists but has been rejected.
• For an environment: the deployments waiting, in the order they will be applied, and the last release deployed (last_deployment, with its external_id release reference). Use it before an approval gate to show what will change.

Errors

---

Example: GitHub Actions

Store TRDM_URL (for example https://acme.app.titanrdm.com), TRDM_CLIENT_ID and TRDM_CLIENT_SECRET as repository secrets.

Promote when a pull request is merged:

name: Promote reference data
on:
  pull_request:
    types: [closed]
    branches: [main]

jobs:
  promote:
    if: github.event.pull_request.merged == true
    runs-on: ubuntu-latest
    steps:
      - name: Promote in TitanRDM
        env:
          TRDM_URL: ${{ secrets.TRDM_URL }}
          TRDM_CLIENT_ID: ${{ secrets.TRDM_CLIENT_ID }}
          TRDM_CLIENT_SECRET: ${{ secrets.TRDM_CLIENT_SECRET }}
          PROMOTION: ${{ github.repository }}#${{ github.event.pull_request.number }}
          MERGE_SHA: ${{ github.event.pull_request.merge_commit_sha }}
        run: |
          TOKEN=$(curl -sS --fail-with-body -u "$TRDM_CLIENT_ID:$TRDM_CLIENT_SECRET" \
            -d grant_type=client_credentials -d "scope=api api:read api:write" \
            "$TRDM_URL/oauth/token" | jq -r .access_token)
          jq -n --arg id "$PROMOTION" --arg sha "$MERGE_SHA" \
            '{promotion_external_id: $id, merge_commit_sha: $sha, allow_missing: true}' |
          curl -sS --fail-with-body -X POST "$TRDM_URL/api/v1/cicd/promote" \
            -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @-

Deploy in each release stage, before the data platform code. GitHub environments provide the approval gate for production:

jobs:
  deploy-staging:
    runs-on: ubuntu-latest
    environment: staging
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 200
      - name: Deploy reference data to TitanRDM
        env:
          TRDM_URL: ${{ secrets.TRDM_URL }}
          TRDM_CLIENT_ID: ${{ secrets.TRDM_CLIENT_ID }}
          TRDM_CLIENT_SECRET: ${{ secrets.TRDM_CLIENT_SECRET }}
        run: |
          TOKEN=$(curl -sS --fail-with-body -u "$TRDM_CLIENT_ID:$TRDM_CLIENT_SECRET" \
            -d grant_type=client_credentials -d "scope=api api:read api:write" \
            "$TRDM_URL/oauth/token" | jq -r .access_token)
          git rev-list --first-parent -n 200 HEAD |
          jq -R . | jq -s --arg env staging --arg ref "$GITHUB_SHA" \
            '{environment_external_id: $env, commit_shas: ., release_ref: $ref}' |
          curl -sS --fail-with-body -X POST "$TRDM_URL/api/v1/cicd/deploy" \
            -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @-
      # ...then deploy the data platform code, e.g. databricks bundle deploy -t staging

  deploy-prod:
    needs: deploy-staging
    runs-on: ubuntu-latest
    environment: prod
    steps:
      # the same steps, with environment_external_id "prod"

---

Example: Azure Pipelines

Azure DevOps names the merge commit of a completed pull request Merged PR 42: …, so a pipeline that runs on main can find the pull request from the commit message.

Promote when a pull request is merged (a pipeline triggered by commits to main):

trigger:
  branches:
    include: [main]

steps:
  - bash: |
      PR=$(git log -1 --format=%s | sed -n 's/^Merged PR \([0-9]*\):.*/\1/p')
      if [ -z "$PR" ]; then echo "Not a pull request merge"; exit 0; fi
      TOKEN=$(curl -sS --fail-with-body -u "$TRDM_CLIENT_ID:$TRDM_CLIENT_SECRET" \
        -d grant_type=client_credentials -d "scope=api api:read api:write" \
        "$TRDM_URL/oauth/token" | jq -r .access_token)
      jq -n --arg id "$(Build.Repository.Name)#$PR" --arg sha "$(Build.SourceVersion)" \
        '{promotion_external_id: $id, merge_commit_sha: $sha, allow_missing: true}' |
      curl -sS --fail-with-body -X POST "$TRDM_URL/api/v1/cicd/promote" \
        -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @-
    displayName: Promote reference data
    env:
      TRDM_URL: $(TRDM_URL)
      TRDM_CLIENT_ID: $(TRDM_CLIENT_ID)
      TRDM_CLIENT_SECRET: $(TRDM_CLIENT_SECRET)

Deploy in each release stage: use a deployment job that targets an Azure DevOps environment, which provides approvals and checks. Run the same deploy request as the GitHub example with the stage's environment_external_id, then deploy the data platform code. Check out enough history (fetchDepth: 200) for git rev-list, and use $(Build.SourceVersion) as the release_ref.

---

Databricks Asset Bundles

A Databricks Asset Bundle deploys jobs, pipelines and notebooks to a target such as dev, staging or prod. To keep reference data in step:

• Use the bundle target names as environment external ids. Give your TitanRDM Stage and Prod environments the external ids staging and prod. The pipeline can then pass the same name to both tools.
• Deploy TitanRDM first. Run the TitanRDM deploy request before databricks bundle deploy -t , so that new reference tables and columns exist before the jobs and pipelines that read them are updated. If TitanRDM refuses the release, the bundle isn't deployed either.
• Check during validation. Run the deploy request with "dry_run": true alongside databricks bundle validate to show what will change, or to stop early when later changes are in the way.

TARGET=staging
# ...TitanRDM deploy request with environment_external_id "$TARGET" (see the examples above)
databricks bundle deploy -t "$TARGET"

Developers work in the dev target and their own TitanRDM Dev environment. They deploy their own changes there from TitanRDM directly; the pipeline handles the shared targets.

Deploying from within the bundle. If your releases run only through bundles, add a task at the start of a job that calls the deploy endpoint. Store the client secret in a Databricks secret scope:

import requests

url = "https://acme.app.titanrdm.com"
client_id = dbutils.secrets.get("titanrdm", "client_id")
client_secret = dbutils.secrets.get("titanrdm", "client_secret")

token = requests.post(f"{url}/oauth/token", auth=(client_id, client_secret),
                      data={"grant_type": "client_credentials", "scope": "api api:read api:write"}).json()["access_token"]
response = requests.post(f"{url}/api/v1/cicd/deploy", headers={"Authorization": f"Bearer {token}"},
                         json={"environment_external_id": dbutils.widgets.get("target"),
                               "promotion_external_id": dbutils.widgets.get("promotion")})
response.raise_for_status()

The same pattern works for other platforms: deploy TitanRDM before dbt run, before a Microsoft Fabric deployment pipeline stage, or before Snowflake schema changes (for example schemachange or Terraform) in each environment.

---

Tips

• Retry safely. If a step fails part-way, run it again. Deployments that are already applied are skipped.
• Promotions that fail can be fixed in TitanRDM and promoted again by re-running the pipeline step.
• Without a service account, an integration can still use the Authorization Code flow for a user who holds the Approver and Deployer permissions. Pipelines then need to keep refreshing that user's token.
• The lower-level endpoints (POST /api/v1/promotions/{id}/promote, POST /api/v1/deployments/{id}/deploy) remain available when you want to control each deployment yourself. See the API Endpoints Reference.

---

Related Pages

• OAuth Applications
• API Endpoints Reference
• Promotions
• Deployments
• Environments