Extraction

Components that extract structured records from texts, images, audio, videos and multimodal samples.

`AudioExtractor(*, fields=None, instructions=None, k_shot=None, max_examples=8, backend=None, batch_size=8, on_error='raise', random_state=None)`

Bases: BaseExtractor

Extract a record of declared fields from every audio sample, learning from reference records in fit.

A field is declared as a description, as a dict of subfields for a nested record, or as a one-element list of either for a list of values or records. predict returns one dict per sample whose keys are exactly the field names, with None for absent fields, and transform returns one column per top-level field.

Parameters:

Name	Type	Description	Default
`fields`	`dict \| None`	The fields to extract, as a dict that maps every name to its declaration.	`None`
`instructions`	`str \| None`	Extra guidance on how to extract the fields.	`None`
`k_shot`	`int \| ArrayLike \| None`	How many training samples are passed to the backend as examples with their records. `None` passes up to `max_examples`, `0` passes none, an integer selects that many at random, and an array of indices selects those samples.	`None`
`max_examples`	`int \| None`	The maximum number of examples when `k_shot` is `None`.	`8`
`backend`	`str \| Backend \| None`	The backend, as `'<provider>'`, `'<provider>:<model>'` or a backend object. The local Hugging Face backend is used when it is `None`.	`None`
`batch_size`	`int`	The number of samples submitted to the backend at once.	`8`
`on_error`	`Literal['raise', 'ignore']`	`'raise'` raises when a sample fails, and `'ignore'` returns `None` for it and warns.	`'raise'`
`random_state`	`int \| None`	Controls the selection of examples and the seeds sent to the backend.	`None`

Attributes:

Name	Type	Description
`backend_`	`Backend`	The resolved backend.
`examples_`	`tuple`	The pairs of a sample and its reference record passed to the backend.

Examples:

>>> from skai.extraction import AudioExtractor
>>> import numpy as np
>>> extractor = AudioExtractor(fields={'topic': 'main topic'}, backend='dummy').fit()
>>> list(extractor.predict([(np.zeros(16000), 16000)])[0])
['topic']

Source code in src/skai/extraction/_audio.py

def __init__(
    self,
    *,
    fields: dict | None = None,
    instructions: str | None = None,
    k_shot: int | ArrayLike | None = None,
    max_examples: int | None = 8,
    backend: str | Backend | None = None,
    batch_size: int = 8,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.fields = fields
    self.instructions = instructions
    self.k_shot = k_shot
    self.max_examples = max_examples
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

`ImageExtractor(*, fields=None, instructions=None, k_shot=None, max_examples=8, backend=None, batch_size=8, on_error='raise', random_state=None)`

Bases: BaseExtractor

Extract a record of declared fields from every image, learning from reference records in fit.

A field is declared as a description, as a dict of subfields for a nested record, or as a one-element list of either for a list of values or records. predict returns one dict per sample whose keys are exactly the field names, with None for absent fields, and transform returns one column per top-level field.

Parameters:

Name	Type	Description	Default
`fields`	`dict \| None`	The fields to extract, as a dict that maps every name to its declaration.	`None`
`instructions`	`str \| None`	Extra guidance on how to extract the fields.	`None`
`k_shot`	`int \| ArrayLike \| None`	How many training samples are passed to the backend as examples with their records. `None` passes up to `max_examples`, `0` passes none, an integer selects that many at random, and an array of indices selects those samples.	`None`
`max_examples`	`int \| None`	The maximum number of examples when `k_shot` is `None`.	`8`
`backend`	`str \| Backend \| None`	The backend, as `'<provider>'`, `'<provider>:<model>'` or a backend object. The local Hugging Face backend is used when it is `None`.	`None`
`batch_size`	`int`	The number of samples submitted to the backend at once.	`8`
`on_error`	`Literal['raise', 'ignore']`	`'raise'` raises when a sample fails, and `'ignore'` returns `None` for it and warns.	`'raise'`
`random_state`	`int \| None`	Controls the selection of examples and the seeds sent to the backend.	`None`

Attributes:

Name	Type	Description
`backend_`	`Backend`	The resolved backend.
`examples_`	`tuple`	The pairs of a sample and its reference record passed to the backend.

Examples:

>>> from skai.extraction import ImageExtractor
>>> from PIL import Image
>>> extractor = ImageExtractor(fields={'color': 'main color'}, backend='dummy').fit()
>>> list(extractor.predict([Image.new('RGB', (8, 8))])[0])
['color']

Source code in src/skai/extraction/_image.py

def __init__(
    self,
    *,
    fields: dict | None = None,
    instructions: str | None = None,
    k_shot: int | ArrayLike | None = None,
    max_examples: int | None = 8,
    backend: str | Backend | None = None,
    batch_size: int = 8,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.fields = fields
    self.instructions = instructions
    self.k_shot = k_shot
    self.max_examples = max_examples
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

`MultimodalExtractor(*, fields=None, instructions=None, n_frames=8, k_shot=None, max_examples=8, backend=None, batch_size=4, on_error='raise', random_state=None)`

Bases: BaseExtractor

Extract a record of declared fields from every sample, learning from reference records in fit.

A sample is a dict, or a DataFrame row, of named texts, images, audio and videos, all sent in one request.

A field is declared as a description, as a dict of subfields for a nested record, or as a one-element list of either for a list of values or records. predict returns one dict per sample whose keys are exactly the field names, with None for absent fields, and transform returns one column per top-level field.

Parameters:

Name	Type	Description	Default
`fields`	`dict \| None`	The fields to extract, as a dict that maps every name to its declaration.	`None`
`instructions`	`str \| None`	Extra guidance on how to extract the fields.	`None`
`n_frames`	`int`	The number of frames sampled from every video.	`8`
`k_shot`	`int \| ArrayLike \| None`	How many training samples are passed to the backend as examples with their records. `None` passes up to `max_examples`, `0` passes none, an integer selects that many at random, and an array of indices selects those samples.	`None`
`max_examples`	`int \| None`	The maximum number of examples when `k_shot` is `None`.	`8`
`backend`	`str \| Backend \| None`	The backend, as `'<provider>'`, `'<provider>:<model>'` or a backend object. The local Hugging Face backend is used when it is `None`.	`None`
`batch_size`	`int`	The number of samples submitted to the backend at once.	`4`
`on_error`	`Literal['raise', 'ignore']`	`'raise'` raises when a sample fails, and `'ignore'` returns `None` for it and warns.	`'raise'`
`random_state`	`int \| None`	Controls the selection of examples and the seeds sent to the backend.	`None`

Attributes:

Name	Type	Description
`backend_`	`Backend`	The resolved backend.
`examples_`	`tuple`	The pairs of a sample and its reference record passed to the backend.

Examples:

>>> from skai.extraction import MultimodalExtractor
>>> from PIL import Image
>>> extractor = MultimodalExtractor(fields={'product': 'product name'}, backend='dummy').fit()
>>> list(extractor.predict([{'photo': Image.new('RGB', (8, 8)), 'notes': 'SkaiBox'}])[0])
['product']

Source code in src/skai/extraction/_multimodal.py

def __init__(
    self,
    *,
    fields: dict | None = None,
    instructions: str | None = None,
    n_frames: int = 8,
    k_shot: int | ArrayLike | None = None,
    max_examples: int | None = 8,
    backend: str | Backend | None = None,
    batch_size: int = 4,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.fields = fields
    self.instructions = instructions
    self.n_frames = n_frames
    self.k_shot = k_shot
    self.max_examples = max_examples
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

`TextExtractor(*, fields=None, instructions=None, k_shot=None, max_examples=8, backend=None, batch_size=16, on_error='raise', random_state=None)`

Bases: BaseExtractor

Extract a record of declared fields from every text, learning from reference records in fit.

A field is declared as a description, as a dict of subfields for a nested record, or as a one-element list of either for a list of values or records. predict returns one dict per sample whose keys are exactly the field names, with None for absent fields, and transform returns one column per top-level field.

Parameters:

Name	Type	Description	Default
`fields`	`dict \| None`	The fields to extract, as a dict that maps every name to its declaration.	`None`
`instructions`	`str \| None`	Extra guidance on how to extract the fields.	`None`
`k_shot`	`int \| ArrayLike \| None`	How many training samples are passed to the backend as examples with their records. `None` passes up to `max_examples`, `0` passes none, an integer selects that many at random, and an array of indices selects those samples.	`None`
`max_examples`	`int \| None`	The maximum number of examples when `k_shot` is `None`.	`8`
`backend`	`str \| Backend \| None`	The backend, as `'<provider>'`, `'<provider>:<model>'` or a backend object. The local Hugging Face backend is used when it is `None`.	`None`
`batch_size`	`int`	The number of samples submitted to the backend at once.	`16`
`on_error`	`Literal['raise', 'ignore']`	`'raise'` raises when a sample fails, and `'ignore'` returns `None` for it and warns.	`'raise'`
`random_state`	`int \| None`	Controls the selection of examples and the seeds sent to the backend.	`None`

Attributes:

Name	Type	Description
`backend_`	`Backend`	The resolved backend.
`examples_`	`tuple`	The pairs of a sample and its reference record passed to the backend.

Examples:

>>> from skai.extraction import TextExtractor
>>> extractor = TextExtractor(fields={'name': 'person name', 'city': 'city'}, backend='dummy').fit()
>>> sorted(extractor.predict(['Ada lives in Athens.'])[0])
['city', 'name']

Source code in src/skai/extraction/_text.py

def __init__(
    self,
    *,
    fields: dict | None = None,
    instructions: str | None = None,
    k_shot: int | ArrayLike | None = None,
    max_examples: int | None = 8,
    backend: str | Backend | None = None,
    batch_size: int = 16,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.fields = fields
    self.instructions = instructions
    self.k_shot = k_shot
    self.max_examples = max_examples
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

`VideoExtractor(*, fields=None, instructions=None, n_frames=8, k_shot=None, max_examples=8, backend=None, batch_size=4, on_error='raise', random_state=None)`

Bases: BaseExtractor

Extract a record of declared fields from every video, learning from reference records in fit.

The model sees frames sampled evenly over every video, and its audio track on backends that accept audio.

A field is declared as a description, as a dict of subfields for a nested record, or as a one-element list of either for a list of values or records. predict returns one dict per sample whose keys are exactly the field names, with None for absent fields, and transform returns one column per top-level field.

Parameters:

Name	Type	Description	Default
`fields`	`dict \| None`	The fields to extract, as a dict that maps every name to its declaration.	`None`
`instructions`	`str \| None`	Extra guidance on how to extract the fields.	`None`
`n_frames`	`int`	The number of frames sampled from every video.	`8`
`k_shot`	`int \| ArrayLike \| None`	How many training samples are passed to the backend as examples with their records. `None` passes up to `max_examples`, `0` passes none, an integer selects that many at random, and an array of indices selects those samples.	`None`
`max_examples`	`int \| None`	The maximum number of examples when `k_shot` is `None`.	`8`
`backend`	`str \| Backend \| None`	The backend, as `'<provider>'`, `'<provider>:<model>'` or a backend object. The local Hugging Face backend is used when it is `None`.	`None`
`batch_size`	`int`	The number of samples submitted to the backend at once.	`4`
`on_error`	`Literal['raise', 'ignore']`	`'raise'` raises when a sample fails, and `'ignore'` returns `None` for it and warns.	`'raise'`
`random_state`	`int \| None`	Controls the selection of examples and the seeds sent to the backend.	`None`

Attributes:

Name	Type	Description
`backend_`	`Backend`	The resolved backend.
`examples_`	`tuple`	The pairs of a sample and its reference record passed to the backend.

Examples:

>>> from skai.extraction import VideoExtractor
>>> import numpy as np
>>> from skai.core import Video
>>> extractor = VideoExtractor(fields={'place': 'where it happens'}, backend='dummy').fit()
>>> list(extractor.predict([Video(np.zeros((8, 8, 8, 3), dtype=np.uint8), fps=4)])[0])
['place']

Source code in src/skai/extraction/_video.py

def __init__(
    self,
    *,
    fields: dict | None = None,
    instructions: str | None = None,
    n_frames: int = 8,
    k_shot: int | ArrayLike | None = None,
    max_examples: int | None = 8,
    backend: str | Backend | None = None,
    batch_size: int = 4,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.fields = fields
    self.instructions = instructions
    self.n_frames = n_frames
    self.k_shot = k_shot
    self.max_examples = max_examples
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state