Feature/user resource quota#18202
Open
hongzhi-gao wants to merge 1 commit into
Open
Conversation
hongzhi-gao
force-pushed
the
feature/user-resource-quota
branch
2 times, most recently
from
014945f to
04d45c8
Compare
…UOTA. Introduce per-user min/max quotas for read/write CPU, memory, temp disk and disk IO, with ConfigNode persistence, DataNode acquire/release, THROTTLE compatibility, and related tests. Usage is reported via a dedicated low-frequency reportUserResourceUsage RPC (not main heartbeat); disk_io/temp_disk use fixed long units (bytes/sec and bytes).
hongzhi-gao
force-pushed
the
feature/user-resource-quota
branch
from
04d45c8 to
3bba33b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add a unified user-level resource quota framework so multi-tenant workloads can set per-user CPU / memory / temp-disk / disk-IO limits with min (guarantee) + max (cap), while staying compatible with existing
THROTTLE QUOTA.SQL & model
SET / SHOW / DELETE USER QUOTACPU,MEMORY,TEMP_DISK(capacity, Byte),DISK_IO(rate, reuse throttle)read_*/write_*); keepSET/SHOW THROTTLE QUOTAand mapcpu/mem→read_cpu_max/read_memory_maxquota_enable(enabled semantics; compatible with THROTTLE/SPACE)Runtime enforcement (DataNode-local)
UserResourceQuotaManager.acquire/releaseon query (DriverScheduler) and write pathsDELETE USER QUOTAclears config only and keeps in-flightinUseuntil tokens releaseCluster SHOW & defaults
SHOW USER QUOTA: NodeID = all Running DataNodes; DN reports inUse via heartbeat; CN aggregates; lost nodes omitteddn_quota_temp_disk_bytesdefault =min(100GiB, ΣdataDirs.totalSpace/10)at startup; explicit config overridesClusterCachePropagatorwhen availableTests
IoTDBUserResourceQuotaIT(SET/SHOW/DELETE, throttle mapping, lower max, NodeID)This PR has:
Key changed/added classes (or packages if there are too many classes) in this PR
iotdb-core/datanode/.../quotas/UserResourceQuotaManageriotdb-core/datanode/.../quotas/CapacityResourceLimiter/LimiterAcquireResultiotdb-core/datanode/.../quota/ShowUserResourceQuotaTask/SetUserResourceQuotaTask/DeleteUserResourceQuotaTaskiotdb-core/confignode/.../ClusterQuotaManageriotdb-core/confignode/.../quota/SetUserResourceQuotaPlan/DeleteUserResourceQuotaPlaniotdb-core/node-commons/.../quota/*iotdb-protocol/thrift-{commons,datanode,confignode}(quota + heartbeat usage snapshot)integration-test/.../IoTDBUserResourceQuotaIT