Job runner
execution.core.job_runner
¶
Job Runner Module
This module provides the main job execution engine for ETL jobs. It handles job instantiation, execution, and result management.
JobRunner
¶
Main job execution engine for ETL jobs.
This class coordinates the execution of ETL jobs by using the JobFactory to create job instances and managing their lifecycle (run, commit, cleanup).
Attributes:
| Name | Type | Description |
|---|---|---|
job_config |
Configuration object containing job parameters |
|
factory |
JobFactory instance for creating job objects |
job_config = job_config
instance-attribute
¶
factory = JobFactory()
instance-attribute
¶
__init__(job_config: JobConfiguration)
¶
Initialize the JobRunner with a job configuration.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
job_config
|
JobConfiguration
|
Configuration object containing job parameters and settings |
required |
run(job_type: Optional[str] = None) -> Any
¶
Execute an ETL job with the specified type.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
job_type
|
Optional[str]
|
Optional job type string. If None, extracts from command line arguments |
None
|
Returns:
| Type | Description |
|---|---|
Any
|
The result returned by the job's run method |
Raises:
| Type | Description |
|---|---|
ValueError
|
If no job type is provided or found in command line arguments |
Exception
|
Re-raises any exception encountered during job execution |
list_jobs() -> Dict[str, str]
¶
Get a list of all available job types with descriptions.
Returns:
| Type | Description |
|---|---|
Dict[str, str]
|
Dictionary mapping job type strings to their descriptions |