tencent cloud

Tencent Cloud DataBuddy

Rest API Data Source

Baixar
Modo Foco
Tamanho da Fonte
Última atualização: 2026-10-09 16:09:43
Traduzido e Verificado por IA

Overview

A REST API data source can read data exposed by third-party systems through HTTP/HTTPS interfaces, making it suitable for scenarios where data is obtained from the open APIs of business systems. DataBuddy data ingestion supports using REST API as a source, covering offline synchronization scenarios. This document describes its connection configuration, supported capabilities, prerequisites, and common issues.

Supported Versions

Type
Supported Version
REST API
All Versions (HTTP/HTTPS)

Read Capability

Capability
System Support
Description
Offline sync (read/write)
✓ / -
Supports reading data as a source for offline synchronization tasks. The DataBuddy data source supports only read access and does not support writes.
For a complete comparison of data source capabilities, see Supported Data Sources and Read Capabilities.

Use Limits

Limit
Description
Response Format
JSON format data must be returned.
Authentication Method
Supports no authentication / Basic / Token / OAuth2 password mode / OAuth2 client mode

Supported Authentication Methods

Authentication Method
Description
No Authentication
The API does not require authentication.
Basic
Username + Password
Token
Fixed Token
OAuth2 Password Mode
Username + Password for Token Exchange
OAuth2 Client Mode
ClientId + ClientSecret for Token Exchange

Prerequisites

Confirm the API address, request method, and authentication information with the API provider, and make sure that the API is accessible from the DataBuddy runtime environment. The API response must be in JSON format and have a stable field structure.

Creating a Data Source

Operation Steps

1. Log in to the DataBuddy console.
2. Switch to the target region and Workspace at the top.
3. Go to Data Integration > Access Method > Data Source Management (or Platform Management > Data Source Management).
4. Click Add Data Source and select REST API.
5. Enter the connection configuration and authentication information. For details, see Parameter Description.
6. Click Test Connectivity (see Data Source Connectivity Test).
7. After the test passes, click Create Only or Create and Connect Task.

Parameter Description

Parameter
Description
Required
Default Value
Data Source Name
Unique identifier of the data source within the workspace
Yes
-
Request URL
Complete API URL
Yes
-
Authentication Method
No authentication / Basic / Token / OAuth2 password mode / OAuth2 client mode
Yes
No Authentication
Username / Password
Account for Basic or OAuth2 password mode
Required for the corresponding authentication method
-
Token
Fixed token for Token authentication
Required when the authentication method is Token
-
ClientId / ClientSecret
Credentials for OAuth2 client mode
Required when the authentication method is OAuth2 client mode
-
Token Endpoint / Scope
Token endpoint and scope in OAuth2 mode
Required for OAuth2 authentication
-

Using in Data Ingestion Tasks

Offline Data Read

Configuration Item
Description
Source Connection
A configured REST API data source. Required. Click Test to verify connectivity.
Read Method
Parse data: Parse the JSON content returned by the API and write it to the target table based on field mappings.
Raw file: Do not parse the response content. Write the raw data returned by the API to the target as a whole. For regular structured data ingestion, select Parse data.
Request method
HTTP method of the API, POST / GET
Returned data type
Data format of the API response. Currently, JSON is supported.
JSON data path
Use JSONPath to locate the level where records reside in the response, for example, $.data.list. It must point to an array of records or a single record object. After configuration, the source field JSONPath in field mapping uses a single record as the root node.
Returned data structure
Single record: The response is a single JSON object.
Array: The response is an array of JSON objects, with each element representing a record. Note that in array mode, each record must be a JSON object. A two-dimensional array will cause parsing to fail.

Request header.
API request headers. Fill them in as prompted on the page, for example, {"Accept-Language":"zh-cn"}. If authentication information is required, such as Authorization, enter it here as well.
Request parameters.
Optional. For GET requests, enter the query parameters appended to the URL, for example, name=wedata&age=10. For POST requests, enter the request body parameters.

Advanced Settings
Click Settings to configure advanced parameters. The default settings meet general scenarios. For large data volumes, split the data into multiple tasks based on the API query parameters and read them separately.


Best Practices

Practice
Recommendation
Authentication Security
Prioritize OAuth2 and avoid plaintext passwords.
Pagination Handling
Confirm the API pagination mechanism to avoid data omission.
Rate Limiting Control
Set request frequency appropriately to avoid triggering API rate limiting.

FAQs

Q: Authentication Failed?

A: Check whether the authentication method and credentials are correct, and confirm that the token has not expired.

Q: Incomplete Data?

A: Confirm that the API pagination parameters are set correctly and that the response is a stable JSON structure.

References



Ajuda e Suporte

Esta página foi útil?

comentários