Perform rate and burst limit on filesystem operations
Maintainer(s):
dentiny
Installing and Loading
INSTALL rate_limit_fs FROM community;
LOAD rate_limit_fs;
Example
SELECT rate_limit_fs_wrap('RateLimitFsFakeFileSystem');
SELECT rate_limit_fs_quota('RateLimitFileSystem - RateLimitFsFakeFileSystem', 'read', 1000000, 'blocking');
About rate_limit_fs
This extension implements rate and burst limit for duckdb filesystems. It supports a few key features:
- Support filesystem and operation based limit.
- Support burst limit and GCRA-based rate limit.
- Supports wrap custom filesystems, so users don't need to change their query.
Added Functions
| function_name | function_type | description | comment | examples |
|---|---|---|---|---|
| rate_limit_fs_burst | scalar | Sets the burst capacity for read or write operations on a rate-limited filesystem. | NULL | [SELECT rate_limit_fs_burst('RateLimitFileSystem - LocalFileSystem', 'read', 10485760);] |
| rate_limit_fs_clear | scalar | Clears one or more rate-limit settings; '*' selects every filesystem or operation. | NULL | [SELECT rate_limit_fs_clear('RateLimitFileSystem - LocalFileSystem', 'read');] |
| rate_limit_fs_configs | table | Returns all configured filesystem operation quotas, modes, burst limits, and concurrency caps. | NULL | [SELECT * FROM rate_limit_fs_configs();] |
| rate_limit_fs_list_filesystems | table | Lists the filesystem implementations registered with DuckDB's virtual filesystem. | NULL | [SELECT * FROM rate_limit_fs_list_filesystems();] |
| rate_limit_fs_max_requests | scalar | Sets the maximum number of concurrent requests for a filesystem operation, or -1 for unlimited. | NULL | [SELECT rate_limit_fs_max_requests('RateLimitFileSystem - LocalFileSystem', 'read', 10);] |
| rate_limit_fs_quota | scalar | Sets the sustained rate limit for a filesystem operation in blocking or non-blocking mode. | NULL | [SELECT rate_limit_fs_quota('RateLimitFileSystem - LocalFileSystem', 'read', 1048576, 'blocking');] |
| rate_limit_fs_wrap | scalar | Wraps a registered filesystem with rate limiting and registers the wrapped filesystem. | NULL | [SELECT rate_limit_fs_wrap('LocalFileSystem');] |
Overloaded Functions
This extension does not add any function overloads.
Added Types
This extension does not add any types.
Added Settings
This extension does not add any settings.