Concept | Description |
Service (Endpoint) | An externally exposed inference endpoint that corresponds to a stable service name and invocation address. Multiple versions can be mounted under a service. |
Service Version (Served Entity) | A specific deployment under a service, corresponding to a model + a version + a set of resource configurations. Multiple versions can coexist within the same service, with requests distributed by traffic ratio. |
Replica | A running instance of the same service version. More replicas deliver higher throughput and better availability, with support for manual / scheduled / automatic / combined scaling. |
Compute resources | Computing power consumed by a service at runtime, with platform support for pay-as-you-go billing. Resource specifications determine the CPU / memory / GPU capacity of a single replica. |
Traffic ratio | When multiple versions coexist, requests are distributed to different versions by ratio (for example, 80 / 20), facilitating canary releases and A/B testing. |
Inference Table | A Delta table managed by Catalog that automatically collects request and response payloads, serving as the fact table for monitoring data drift, performance, and fairness. |
Authentication Token | After authentication is enabled for a service, the caller must carry a Bearer Token in the request Header, which is generated and rotated by the platform. |
フィードバック