Dataloaders
Shape the source data for modelling.
BaseDataLoader(param_grid=None)
Bases: ABC
Read and validate source snapshots and extract moment-aware modelling data.
A dataloader reads long stats and odds event snapshots and validates them. It derives the available
providers, markets and per-column metadata from the data. It then extracts moment-aware training and fixtures
data. This base class does everything except read the data source. A concrete dataloader implements the abstract
_load_snapshots method. It overrides the optional _list_all_params
hook when its data is downloadable.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
param_grid
|
ParamGrid | None
|
Selects the data to include. Keys are parameters like |
None
|
Attributes:
| Name | Type | Description |
|---|---|---|
param_grid_ |
list[dict]
|
The league/division/year combinations of the loaded data. |
stats_ |
DataFrame
|
The validated long |
odds_ |
DataFrame
|
The validated long |
drop_na_thres_ |
float
|
The checked value of |
odds_type_ |
str | None
|
The checked value of |
input_cols_ |
Index
|
The columns of |
output_cols_ |
Index | None
|
The columns of |
odds_cols_ |
Index
|
The columns of |
stats_schema_ |
Schema
|
The validated schema of the |
odds_schema_ |
Schema
|
The validated schema of the |
targets_ |
list[str]
|
The market columns the targets are built from. |
target_event_status_ |
str
|
The resolved status of the target moment. |
target_event_time_ |
Timedelta
|
The resolved time of the target moment. |
input_event_status_ |
str | None
|
The resolved status of the input horizon. |
input_event_time_ |
Timedelta
|
The resolved time of the input horizon. |
Examples:
>>> import pandas as pd
>>> from sportsbet.dataloaders import BaseDataLoader
>>>
>>> identity = {'date': pd.Timestamp('2024-08-16', tz='UTC'), 'league': 'England', 'division': 1,
... 'year': 2025, 'home_team': 'A', 'away_team': 'B'}
>>>
>>> class MyDataLoader(BaseDataLoader):
... 'A dataloader of data that is already on your machine.'
...
... def _load_snapshots(self):
... stats = pd.DataFrame([
... {**identity, 'event_status': 'preplay', 'event_time': pd.Timedelta(0), 'home_form': 1.0},
... {**identity, 'event_status': 'postplay', 'event_time': pd.Timedelta(0), 'home_win': 1},
... ])
... odds = pd.DataFrame([
... {**identity, 'event_status': 'preplay', 'event_time': pd.Timedelta(0),
... 'provider': 'acme', 'home_win': 2.5},
... ])
... return stats, odds
>>>
>>> dataloader = MyDataLoader()
>>> # The providers and the markets are derived from the data.
>>> dataloader.get_odds_types()
['acme']
>>> X, Y, O = dataloader.extract_train_data(odds_type='acme')
>>> list(Y.columns)
['home_win__postplay__0min']
>>> list(O.columns)
['acme__home_win__preplay__0min']
Source code in src/sportsbet/dataloaders/_base.py
202 203 | |
sources_
property
The data sources, empty for a dataloader carrying its own data.
extract_exploration_data(*, drop_na_thres=0.0, target_event_status=None, target_event_time=None, input_event_status=None, input_event_time=None)
Extract the moment-aware features on their own, for exploration.
Read more in the user guide.
It downloads the selected seasons. It returns the features X, with no targets and no odds. Use it to look at
a sport before you choose a param_grid or a model. Use it too when the source carries no odds and has nothing
to predict. The features follow the target moment and input horizon you pass.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
drop_na_thres
|
float
|
Threshold in |
0.0
|
target_event_status
|
str | None
|
|
None
|
target_event_time
|
Timedelta | None
|
In-play target time (e.g. |
None
|
input_event_status
|
str | None
|
Latest snapshot status to keep as a feature, one of |
None
|
input_event_time
|
Timedelta | None
|
Time of the input horizon (e.g. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
X |
DataFrame
|
The moment-aware features of the selected matches. |
Source code in src/sportsbet/dataloaders/_base.py
654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 | |
extract_fixtures_data()
Extract the fixtures data.
Read more in the user guide.
A fixture is a match that has not been played yet. This downloads the upcoming matches of the selected leagues.
It returns them shaped exactly like the training data, so a model trained on the history can bet on the
fixtures. The two share their columns, not their contents. param_grid chose the seasons to train on, and a
match still to be played is in none of them.
Call extract_train_data first, because it fixes those columns. The multi-output targets Y are always
None.
Returns:
| Type | Description |
|---|---|
(X, None, O)
|
The fixtures input data |
Raises:
| Type | Description |
|---|---|
ValueError
|
If it is called before |
Source code in src/sportsbet/dataloaders/_base.py
702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 | |
extract_train_data(*, drop_na_thres=None, odds_type=None, target_event_status=None, target_event_time=None, input_event_status=None, input_event_time=None)
Extract the moment-aware training data.
Read more in the user guide.
It downloads the selected seasons. It returns the historical data you build and backtest a betting strategy
on. Every snapshot before the target moment (target_event_status, target_event_time) becomes a feature in
X. An input horizon can cap which snapshots become features. The target-moment outcomes become the labels
Y. The odds become O. A dataloader that already downloaded reuses the snapshots it holds instead of
fetching again. A bare call on a reloaded dataloader rebuilds the same data offline. Save one with save.
When the odds source carries no markets, there is nothing to predict. Use extract_exploration_data to get
the features on their own.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
drop_na_thres
|
float | None
|
Threshold in |
None
|
odds_type
|
str | None
|
One of |
None
|
target_event_status
|
str | None
|
|
None
|
target_event_time
|
Timedelta | None
|
In-play target time (e.g. |
None
|
input_event_status
|
str | None
|
Latest snapshot status to keep as a feature, one of |
None
|
input_event_time
|
Timedelta | None
|
Time of the input horizon (e.g. |
None
|
Returns:
| Type | Description |
|---|---|
(X, Y, O)
|
Moment-aware features |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the selection yields no betting markets, so there is nothing
to predict. Pass an |
Source code in src/sportsbet/dataloaders/_base.py
559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 | |
get_odds_types()
Return the available odds types (providers) derived from the data.
Returns:
| Type | Description |
|---|---|
list[str]
|
The provider names the odds data carries, sorted. |
Source code in src/sportsbet/dataloaders/_base.py
262 263 264 265 266 267 268 269 | |
save(path)
Save the dataloader object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str
|
The path to save the object. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
self |
Self
|
The dataloader object. |
Source code in src/sportsbet/dataloaders/_base.py
758 759 760 761 762 763 764 765 766 767 768 769 770 771 | |
DataLoader(param_grid=None, stats=None, odds=None, aliases=None)
Bases: BaseDataLoader
Download source data and shape it into moment-aware modelling data.
There is one dataloader for every sport. The loader reads the sport off the statistics source and pairs it only with odds about the same sport.
It downloads the data into memory when you extract, and holds it on the object. Extract again and it downloads
again. Keep what you have with save, and read it back with load_dataloader.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
param_grid
|
ParamGrid | None
|
Selects the seasons to train on. Keys are |
None
|
stats
|
BaseStatsSource | None
|
The source of the statistics. |
None
|
odds
|
BaseOddsSource | None
|
The source of the odds, which may differ from the statistics source.
The default |
None
|
aliases
|
dict[str, str] | None
|
The team names of the odds source, mapped to the names of the statistics source, for the clubs the two feeds name differently. They are added to the ones the library already knows. |
None
|
Attributes:
| Name | Type | Description |
|---|---|---|
stats_ |
DataFrame
|
The downloaded statistics snapshots: the selected seasons, plus each selected league's season in progress. |
odds_ |
DataFrame
|
The downloaded odds snapshots of the selected provider. |
Examples:
>>> from sportsbet.dataloaders import DataLoader
>>> from sportsbet.sources import FootballDataOdds, FootballDataStats
>>> dataloader = DataLoader(
... param_grid={'league': ['Italy'], 'division': [1], 'year': [2024]},
... stats=FootballDataStats(),
... odds=FootballDataOdds(),
... )
>>> # The sources say what sport it is.
>>> dataloader.sport_
'soccer'
>>> # X, Y, O = dataloader.extract_train_data(odds_type='market_maximum')
Source code in src/sportsbet/dataloaders/_sourced.py
68 69 70 71 72 73 74 75 76 77 78 | |
sources_
property
The statistics and odds sources.
sport_
property
The sport the sources carry.
build_dataloader(stats, odds=None, leagues=None, divisions=None, years=None, odds_key_env=DEFAULT_KEY_ENV, odds_markets=None, odds_regions=None, odds_moments=None, aliases=None)
Build a dataloader from the names of its sources and the seasons to select.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
stats
|
str
|
The statistics source to read, one of the ready-made names ( |
required |
odds
|
str | None
|
The odds source to pair with the statistics, or |
None
|
leagues
|
list[str] | None
|
The leagues to select, or |
None
|
divisions
|
list[int] | None
|
The divisions to select, or |
None
|
years
|
list[int] | None
|
The years to select, or |
None
|
odds_key_env
|
str
|
The name of the environment variable holding the odds source's API key, read when the source needs one. |
DEFAULT_KEY_ENV
|
odds_markets
|
list[str] | None
|
The markets the odds source should price, or |
None
|
odds_regions
|
list[str] | None
|
The regions the odds source should price, or |
None
|
odds_moments
|
list[str] | None
|
The moments the odds source should price, each as |
None
|
aliases
|
list[str] | None
|
The teams the sources spell differently, each as |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
dataloader |
DataLoader
|
The dataloader that downloads and shapes the selected data. |
Raises:
| Type | Description |
|---|---|
BuildError
|
If a source name is unknown, or an argument is malformed. |
Source code in src/sportsbet/dataloaders/_factory.py
81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 | |
build_extraction_settings(odds_type=None, drop_na_thres=None, target_event_status=None, target_event_time=None, input_event_status=None, input_event_time=None)
Turn the extraction arguments into keyword arguments for extract_train_data.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
odds_type
|
str | None
|
The odds to extract, or |
None
|
drop_na_thres
|
float | None
|
The threshold to drop missing columns, or |
None
|
target_event_status
|
str | None
|
Where the targets are taken from, or |
None
|
target_event_time
|
str | None
|
The moment of the targets when they are in-play, as |
None
|
input_event_status
|
str | None
|
The latest snapshot kept as a feature, or |
None
|
input_event_time
|
str | None
|
The moment of the input horizon, as |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
settings |
dict[str, Any]
|
The keyword arguments, with the times as |
Examples:
>>> from sportsbet.dataloaders import build_extraction_settings
>>> build_extraction_settings(odds_type='market_average', target_event_time='45min')
{'odds_type': 'market_average', 'target_event_time': Timedelta('0 days 00:45:00')}
Source code in src/sportsbet/dataloaders/_extraction.py
12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | |
load_dataloader(path)
Load the dataloader object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str
|
The path of the dataloader pickled file. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
dataloader |
BaseDataLoader
|
The dataloader object. |
Examples:
>>> import tempfile
>>> from pathlib import Path
>>> from sportsbet.dataloaders import DataLoader, load_dataloader
>>> from sportsbet.sources import SampleSoccerOdds, SampleSoccerStats
>>> path = str(Path(tempfile.mkdtemp()) / 'dataloader.pkl')
>>> dataloader = DataLoader(
... param_grid={'league': ['England']}, stats=SampleSoccerStats(), odds=SampleSoccerOdds()
... )
>>> X, Y, O = dataloader.extract_train_data(odds_type='market_average')
>>> _ = dataloader.save(path)
>>> # It comes back knowing what it was told, so the fixtures take the shape the training data took.
>>> loaded = load_dataloader(path)
>>> loaded.param_grid_ == dataloader.param_grid_
True
>>> X_fix, _, O_fix = loaded.extract_fixtures_data()
>>> list(X_fix.columns) == list(X.columns)
True
Source code in src/sportsbet/dataloaders/_base.py
774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 | |