Run duckdb query on remote server
Maintainer(s):
dentiny
Installing and Loading
INSTALL duckherder FROM community;
LOAD duckherder;
Example
SELECT duckherder_start_local_server(8815); ATTACH DATABASE 'localhost:8815' AS dh (TYPE duckherder);
About duckherder
This extension is built on storage extension, to execute query on remote server and transfer data back with arrow flight. For remote and distributed execution, there're two key components: driver and worker(s). As their names suggest, driver node is the control plane, and worker nodes do the real execution tasks. Users are allowed to implement their own driver and worker(s), and register to the duckdb client-side, as long they speaks duckherder dialect (i.e., grpc stubs and arrow flight). From users' perspective, all DML and DDL SQL statements should be used exactly the same as local duckdb.
Added Functions
| function_name | function_type | description | comment | examples |
|---|---|---|---|---|
| duckherder_clear_query_recorder_stats | scalar | Clears all query history records collected by the Duckherder query recorder. | NULL | [SELECT duckherder_clear_query_recorder_stats();] |
| duckherder_get_query_execution_stats | table | Returns execution mode, merge strategy, duration, worker, task, and start-time statistics for distributed queries. | NULL | [SELECT * FROM duckherder_get_query_execution_stats();] |
| duckherder_get_query_history | table | Returns recorded SQL queries and their observed execution latencies. | NULL | [SELECT * FROM duckherder_get_query_history();] |
| duckherder_get_worker_count | scalar | Returns the number of workers registered with the local Duckherder driver server. | NULL | [SELECT duckherder_get_worker_count();] |
| duckherder_load_extension | scalar | Loads an extension on the attached Duckherder server and attempts to load it on the client. | NULL | [SELECT duckherder_load_extension('parquet');] |
| duckherder_register_or_replace_driver | scalar | Registers a driver node, replacing the currently registered driver if one exists. | NULL | [SELECT duckherder_register_or_replace_driver('driver-1', 'grpc://localhost:8815');] |
| duckherder_register_worker | scalar | Registers a worker node with the local Duckherder driver server. | NULL | [SELECT duckherder_register_worker('worker-1', 'grpc://localhost:8816');] |
| duckherder_start_local_server | scalar | Starts a local Duckherder driver server on the given port, optionally with local workers. | NULL | [SELECT duckherder_start_local_server(8815, 4);] |
| duckherder_start_standalone_worker | scalar | Starts a standalone Duckherder worker on the given port. | NULL | [SELECT duckherder_start_standalone_worker(8816);] |
| duckherder_stop_local_server | scalar | Stops the local Duckherder driver server and its managed standalone workers. | NULL | [SELECT duckherder_stop_local_server();] |
| duckherder_unregister_remote_table | pragma | Removes a remote table mapping from the attached Duckherder catalog. | NULL | [PRAGMA duckherder_unregister_remote_table('orders');] |
Overloaded Functions
This extension does not add any function overloads.
Added Types
This extension does not add any types.
Added Settings
| name | description | input_type | scope | aliases |
|---|---|---|---|---|
| duckherder_retry_initial_backoff_ms | Initial Duckherder RPC retry backoff in milliseconds | BIGINT | GLOBAL | [] |
| duckherder_retry_jitter_ratio | Jitter ratio applied to Duckherder RPC retry backoff | DOUBLE | GLOBAL | [] |
| duckherder_retry_max_attempts | Maximum number of attempts for retryable Duckherder RPCs | UBIGINT | GLOBAL | [] |
| duckherder_retry_max_backoff_ms | Maximum Duckherder RPC retry backoff in milliseconds | BIGINT | GLOBAL | [] |