This document describes how to perform connectivity tests when a data source is being created or edited, and explains the dependencies between connectivity tests and compute resources and network settings.
Overview
A connectivity test initiates a test connection to the data source through the specified compute resource to verify whether the connection information of the data source (address, port, and authentication) is correct and whether the network environment of the compute resource can reach the data source.
All features in DataBuddy that depend on data sources (data ingestion, external Catalog, federated analytics, and so on) actually access data sources through compute resources. Therefore, the connectivity test result represents the network reachability during actual task execution and is more informative than a standalone network ping.
Prerequisites
Before you start this operation, make sure the following prerequisites are met:
Complete connection information and authentication information have been entered in the data source creation and editing panel. For details, see Global Data Source Configuration. At least one compute resource in the Running state exists in the current workspace.
If the data source is in your private VPC, complete the outbound VPC connectivity configuration in Compute Resource Network Settings first. Otherwise, the test will fail. Use Limits
Limit | Description |
Compute resource required. | Connectivity tests cannot be executed independently of compute resources. If no compute resources exist in the workspace, create a compute resource first. |
Detection Scope | Only network reachability and authentication validity are checked. Whether the target database tables exist or have read/write permissions is not checked. |
Test Result Not Persisted. | Connectivity test results are not saved to the data source details page. They only serve as real-time feedback during the creation/editing process. |
Test Does Not Affect Data Source Status. | A test failure does not prevent the data source from being saved. You can save the data source first and then troubleshoot network issues. |
Single Test Duration | The test process is a synchronous operation. Depending on the data source response speed, it typically completes within a few seconds to tens of seconds. |
Operation Steps
Step 1: Initiating a Test on the Data Source Creation or Editing Page
1. Go to the data source creation or editing panel.
2. Expand the Connectivity Test section at the bottom of the panel.
3. Select one or more compute resources.
4. Click Test Connectivity.
5. During the test, the button changes to a loading state. After the test is completed, the result is displayed:
Connection successful: A pop-up prompts Connection successful.
Connection failed: A pop-up prompts Connection failed: ${error cause}.
6. Close the pop-up to continue configuring other settings or save the data source.
Note:
Connectivity tests can be performed multiple times before you save the data source without affecting the state of the data source itself. We recommend that you perform at least one connectivity test before saving to ensure that tasks do not fail at runtime due to connection errors.
Step 2: Initiating a Test in the Source and Target Settings of a Data Ingestion Task
After you select an existing data source connection in the source settings or destination settings of a data integration task, the platform automatically initiates a connectivity test:
1. Choose Source Connection in Source & Destination Selection > Source.
2. Pending test is displayed below the connection options. Click Test to start testing connectivity.
3. If the connection is successful, a message prompts Connection successful. If the connection fails, a message prompts the failure and its cause.
4. Click Retry to trigger the connectivity test again.
Step 3: Adjusting Configurations Based on Test Results
Test Result | Handling Recommendation |
Connection Successful | A message Connection successful is displayed. You can continue to save the data source or go to the next step of task configuration. |
Connection Failed | A message Connection failed: ${Failure reason} is displayed. |
For common solutions to connection failures, see FAQ. Verification Result
After a successful connectivity test, you can observe the following changes:
1. On the data source creation and editing page, a prompt appears below the test button: Connection successful: Connected to ${connection address}.
2. In the source and destination settings of a sync task, a Connection successful tag appears next to the selected data source.
Test Results and Troubleshooting
Common Failure Causes
Failure reason | Troubleshooting Direction |
Network unreachable | The data source is in a user's private VPC, but the selected compute resource is not connected to this VPC. Go to Compute Resource Network Settings to add a VPC outbound connection. |
Public IP address not added to allowlist | The data source restricts IP address allowlist access, but the egress IP address of the selected compute resource is not in the allowlist. Use the fixed egress IP address of the compute resource and add it to the allowlist on the data source side. |
Authentication information error | The username, password, or key is incorrect, or the SSM credential has been deleted or expired. |
Data source version mismatch | The data source version field does not match the actual data source version (for example, version 8.0 is selected for a MySQL 5.6 database). |
Data source service not started | The service process of the data source is not started or is under maintenance. |
Relationship with Compute Resources
Connectivity test = Compute resource × Data source: The test result reflects not only whether the data source itself is reachable, but also the network reachability from the selected compute resource to the data source. Test results for the same data source may vary under different compute resources.
Scenario | Applicable Configuration |
The data source is in the Tencent Cloud private network (such as Tencent Cloud RDS). | Any running compute resource is sufficient. |
Data source in the user's private VPC. | |
Data source in on-premises IDC or third-party cloud. | First connect to the Tencent Cloud VPC through Tencent Cloud VPN / Direct Connect, and then configure network settings for the compute resource. |
Data source exposed to the public network (with IP address allowlist). | Add the fixed egress IP address of the compute resource to the allowlist on the data source side. |
FAQs
Q1: Connectivity Test Succeeds, but the Task Still Reports "Connection Failed" During Runtime
Make sure that the compute resource used at task runtime is the same as the one selected during the connectivity test. If they are different, the runtime compute resource may not be connected to the data source VPC. In this case, reconfigure the network settings.
Q2: How to Make a Data Source Accessible Across Multiple Compute Resources?
Each compute resource that needs to access this data source must complete the VPC outbound configuration in Network Settings separately. We recommend that you plan a unified compute resource pool within your team to avoid redoing network connectivity every time a new compute resource is created. Q3: Which Data Sources Are Not Supported by Connectivity Tests?
Some data sources (such as the built-in DataBuddy data source) are system preset sources and do not require or support connectivity testing. All other user-defined data sources support connectivity testing.
Q4: Connection Test Fails with "Access denied / Authentication Failed"
Possible causes:
The username or password is incorrect.
The SSM credential name or version is incorrect, causing the dynamically obtained password to not match the actual one.
The Kerberos keytab does not match the principal.
Solution:
1. Verify that the username and password are correct. If necessary, verify them directly in a client tool such as mysql, psql, or ftp.
3. If Kerberos is used, confirm that the keytab and principal are issued by the same KDC, and that the realm and KDC in the conf file match the target system.
Q5: Is It Common for Tasks to Fail After a Successful Test?
Possible causes:
The account used to run the task is different from the test account. Real-time synchronization runs as the "Owner" identity.
The database, table, or topic does not exist, or you do not have the required read/write permissions.
The network changes while the task is running, such as when the peer service goes offline or the IP address changes.
Solution:
1. On the data source details page, confirm that the Owner matches the runtime identity.
2. Verify whether the database tables, topics, volumes, and other resources exist.
3. View the runtime logs in task Ops monitoring to locate the specific exception.
Q6: Can Connectivity Tests Be Triggered via API?
A: Open API capabilities are to be supplemented.
References