Run Python scripts
DPDispatcher can be used to directly run a single Python script:
dpdisp run script.py
The script must include inline script metadata compliant with PEP 723. An example of the script is shown below.
1# /// script
2# # dpdispatcher doesn't use `requires-python` and `dependencies`
3# requires-python = ">=3"
4# dependencies = [
5# ]
6# [tool.dpdispatcher]
7# work_base = "./"
8# forward_common_files=[]
9# backward_common_files=[]
10# [tool.dpdispatcher.machine]
11# batch_type = "Shell"
12# local_root = "./"
13# context_type = "LazyLocalContext"
14# [tool.dpdispatcher.resources]
15# number_node = 1
16# cpu_per_node = 1
17# gpu_per_node = 0
18# group_size = 0
19# [[tool.dpdispatcher.task_list]]
20# # no need to contain the script filename
21# command = "python"
22# # can be a glob pattern
23# task_work_path = "./"
24# forward_files = []
25# backward_files = ["log"]
26# ///
27
28print("hello world!")
The PEP 723 metadata entries for tool.dpdispatcher
are defined as follows:
- pep723:
- type:
dict
argument path:pep723
PEP 723 metadata
- work_base:
- type:
str
, optional, default:./
argument path:pep723/work_base
Base directory for the work
- forward_common_files:
- type:
typing.List[str]
, optional, default:[]
argument path:pep723/forward_common_files
Common files to forward to the remote machine
- backward_common_files:
- type:
typing.List[str]
, optional, default:[]
argument path:pep723/backward_common_files
Common files to backward from the remote machine
- machine:
- type:
dict
argument path:pep723/machine
Machine configuration. See related documentation for details.
- resources:
- type:
dict
argument path:pep723/resources
Resources configuration. See related documentation for details.
- task_list:
- type:
list
argument path:pep723/task_list
List of tasks to execute.
This argument takes a list with each element containing the following:
- command:
- type:
str
, optional, default:python
argument path:pep723/task_list/command
Python interpreter or launcher. No need to contain the Python script filename.
- task_work_path:
- type:
str
argument path:pep723/task_list/task_work_path
The dir where the command to be executed. Can be a glob pattern.
- forward_files:
- type:
typing.List[str]
, optional, default:[]
argument path:pep723/task_list/forward_files
The files to be uploaded in task_work_path before the task exectued.
- backward_files:
- type:
typing.List[str]
, optional, default:[]
argument path:pep723/task_list/backward_files
The files to be download to local_root in task_work_path after the task finished
- outlog:
- type:
str
|NoneType
, optional, default:log
argument path:pep723/task_list/outlog
The out log file name. redirect from stdout
- errlog:
- type:
str
|NoneType
, optional, default:err
argument path:pep723/task_list/errlog
The err log file name. redirect from stderr