Generation

Components that generate texts, images, audio and videos from every input kind.

AudioToAudioGenerator(*, operation='separate', stem='vocals', voice=None, gap=0.0, output_format='object', output_dir=None, backend=None, batch_size=4, on_error='raise', random_state=None)

Bases: BaseMediaGenerator

Return one audio sample per audio sample: a separated stem, a denoised copy, another voice, or a joined list.

Parameters:

Name	Type	Description	Default
operation	Literal['separate', 'enhance', 'convert_voice', 'concatenate']	'separate' keeps stem, 'enhance' removes noise, 'convert_voice' says the same speech in voice, and 'concatenate' joins a list of audio samples with gap seconds between them, locally.	'separate'
stem	Literal['vocals', 'accompaniment']	'vocals' or 'accompaniment', the stem kept by a separation.	'vocals'
voice	VoiceSource | None	One recording or a list of recordings of the target voice. Required with operation='convert_voice'.	None
gap	float	The seconds of silence between joined audio samples.	0.0
output_format	Literal['object', 'bytes', 'path']	'object' returns Audio objects, 'bytes' WAV bytes, and 'path' the paths of new WAV files.	'object'
output_dir	str | PathLike | None	The directory of 'path' outputs. A new temporary directory is used when it is None.	None
backend	str | Backend | None	The backend, as '<provider>', '<provider>:<model>' or a backend object. The local Hugging Face backend is used when it is None, and an open-source model on Replicate for voice conversion. Concatenation needs no backend.	None
batch_size	int	The number of audio samples submitted to the backend at once.	4
on_error	Literal['raise', 'ignore']	'raise' raises when a sample fails, and 'ignore' returns None for it and warns.	'raise'
random_state	int | None	Controls the seeds sent to the backend.	None

Attributes:

Name	Type	Description
backend_	Backend | None	The resolved backend, or None for concatenation.
voice_	Audio | None	The joined reference audio of the target voice.

Examples:

>>> import numpy as np
>>> from skai.generation import AudioToAudioGenerator
>>> song = (np.zeros(32000), 16000)
>>> AudioToAudioGenerator(operation='separate', backend='dummy').fit().predict([song])[0].duration
2.0

Source code in src/skai/generation/_audio.py

def __init__(
    self,
    *,
    operation: Literal['separate', 'enhance', 'convert_voice', 'concatenate'] = 'separate',
    stem: Literal['vocals', 'accompaniment'] = 'vocals',
    voice: VoiceSource | None = None,
    gap: float = 0.0,
    output_format: Literal['object', 'bytes', 'path'] = 'object',
    output_dir: str | PathLike | None = None,
    backend: str | Backend | None = None,
    batch_size: int = 4,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.operation = operation
    self.stem = stem
    self.voice = voice
    self.gap = gap
    self.output_format = output_format
    self.output_dir = output_dir
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

AudioToTextGenerator(*, operation='transcribe', instructions=None, language=None, max_vocabulary=50, max_length=None, k_shot=None, max_examples=8, adaptation='examples', use_y='auto', backend=None, batch_size=8, on_error='raise', random_state=None)

Bases: BaseTextGenerator

Return one text per audio sample, by transcribing the speech or by following instructions.

With operation='transcribe' (the default), fit(X, y) with reference transcripts as y learns a vocabulary of the names, products and technical terms they contain, which the backend receives as context for spelling. With operation='instruct', the audio is described, summarized or answered about by an audio-language model, as the instructions ask.

Parameters:

Name	Type	Description	Default
operation	Literal['transcribe', 'instruct']	'transcribe' or 'instruct'.	'transcribe'
instructions	str | None	What to produce from every audio sample. Required with operation='instruct'.	None
language	str | None	The language of the speech, such as 'en'. It is detected when it is None.	None
max_vocabulary	int	The maximum number of terms in the vocabulary, the most frequent first.	50
max_length	int | Literal['auto'] | None	The maximum number of words of every output. 'auto' learns it from the references as the 90th percentile of their word counts, and None sets no limit.	None
k_shot	int | ArrayLike | None	How many training samples are passed to the backend as examples with their references, with operation='instruct'. None passes up to max_examples, 0 passes none, an integer selects that many at random, and an array of indices selects those samples.	None
max_examples	int | None	The maximum number of examples when k_shot is None.	8
adaptation	Literal['examples', 'finetune']	'examples' passes examples to the backend at prediction. 'finetune' trains a model on all the training data in fit, on backends that support fine-tuning, with operation='instruct'.	'examples'
use_y	Literal['auto'] | bool	'auto' learns from y in fit but not in fit_transform, which pipelines use for intermediate steps. True always learns from y, and False never does.	'auto'
backend	str | Backend | None	The backend, as '<provider>', '<provider>:<model>' or a backend object. The local Hugging Face backend is used when it is None.	None
batch_size	int	The number of audio samples submitted to the backend at once.	8
on_error	Literal['raise', 'ignore']	'raise' raises when a sample fails, and 'ignore' returns None for it and warns.	'raise'
random_state	int | None	Controls the selection of examples and the seeds sent to the backend.	None

Attributes:

Name	Type	Description
backend_	Backend	The resolved backend.
vocabulary_	tuple	The domain terms learned from the reference transcripts.
examples_	tuple	The pairs of a sample and its reference passed to the backend.
max_length_	int | None	The maximum number of words of every output.

Examples:

>>> import numpy as np
>>> from skai.generation import AudioToTextGenerator
>>> AudioToTextGenerator(backend='dummy').fit().predict([(np.zeros(16000), 16000)])[0]
'Speech of 1.00 seconds.'

Source code in src/skai/generation/_text.py

def __init__(
    self,
    *,
    operation: Literal['transcribe', 'instruct'] = 'transcribe',
    instructions: str | None = None,
    language: str | None = None,
    max_vocabulary: int = 50,
    max_length: int | Literal['auto'] | None = None,
    k_shot: int | ArrayLike | None = None,
    max_examples: int | None = 8,
    adaptation: Literal['examples', 'finetune'] = 'examples',
    use_y: Literal['auto'] | bool = 'auto',
    backend: str | Backend | None = None,
    batch_size: int = 8,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.operation = operation
    self.instructions = instructions
    self.language = language
    self.max_vocabulary = max_vocabulary
    self.max_length = max_length
    self.k_shot = k_shot
    self.max_examples = max_examples
    self.adaptation = adaptation
    self.use_y = use_y
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

AudioToVideoGenerator(*, operation='talking_head', avatar=None, output_format='object', output_dir=None, backend=None, batch_size=2, on_error='raise', random_state=None)

Bases: BaseMediaGenerator

Make a portrait speak every audio sample, as a talking head.

fit prepares avatar_ from avatar: converted to RGB, center-cropped to a square, and resized to the input size of the backend.

Parameters:

Name	Type	Description	Default
operation	Literal['talking_head']	'talking_head', the only operation.	'talking_head'
avatar	str | bytes | Image | ndarray | PurePath | None	The portrait, as a path, URL, bytes, PIL image or array.	None
output_format	Literal['object', 'bytes', 'path']	'object' returns Video objects, 'bytes' MP4 bytes, and 'path' the paths of new MP4 files.	'object'
output_dir	str | PathLike | None	The directory of 'path' outputs. A new temporary directory is used when it is None.	None
backend	str | Backend | None	The backend, as '<provider>', '<provider>:<model>' or a backend object. SadTalker on Replicate is used when it is None.	None
batch_size	int	The number of audio samples submitted to the backend at once.	2
on_error	Literal['raise', 'ignore']	'raise' raises when a sample fails, and 'ignore' returns None for it and warns.	'raise'
random_state	int | None	Controls the seeds sent to the backend.	None

Attributes:

Name	Type	Description
backend_	Backend	The resolved backend.
avatar_	Image	The prepared portrait.

Examples:

>>> import numpy as np
>>> from PIL import Image
>>> from skai.generation import AudioToVideoGenerator
>>> generator = AudioToVideoGenerator(avatar=Image.new('RGB', (640, 480)), backend='dummy').fit()
>>> generator.predict([(np.zeros(16000), 16000)])[0].duration
1.0

Source code in src/skai/generation/_video.py

def __init__(
    self,
    *,
    operation: Literal['talking_head'] = 'talking_head',
    avatar: str | bytes | Image.Image | np.ndarray | PurePath | None = None,
    output_format: Literal['object', 'bytes', 'path'] = 'object',
    output_dir: str | PathLike | None = None,
    backend: str | Backend | None = None,
    batch_size: int = 2,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.operation = operation
    self.avatar = avatar
    self.output_format = output_format
    self.output_dir = output_dir
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

ImageToImageGenerator(*, operation='edit', instructions=None, mask=None, scale=2, columns=None, tile=256, output_format='object', output_dir=None, backend=None, batch_size=4, on_error='raise', random_state=None)

Bases: BaseMediaGenerator

Return one image per image: edited, upscaled, without its background, or a collage of a list of images.

Parameters:

Name	Type	Description	Default
operation	Literal['edit', 'upscale', 'remove_background', 'collage']	'edit' follows instructions inside an optional mask, 'upscale' enlarges by scale, 'remove_background' returns an RGBA image with a transparent background, and 'collage' lays a list of images out in a grid, locally.	'edit'
instructions	str | None	The edit to apply. Required with operation='edit'.	None
mask	ImageSource | None	The region to edit, as a grayscale image where white marks the region. It is resized to every image.	None
scale	int	The factor of upscaling.	2
columns	int | None	The number of columns of a collage, chosen from the number of images when it is None.	None
tile	int	The side of every tile of a collage in pixels.	256
output_format	Literal['object', 'bytes', 'path']	'object' returns PIL images, 'bytes' PNG bytes, and 'path' the paths of new PNG files.	'object'
output_dir	str | PathLike | None	The directory of 'path' outputs. A new temporary directory is used when it is None.	None
backend	str | Backend | None	The backend, as '<provider>', '<provider>:<model>' or a backend object. The local Hugging Face backend is used when it is None. Collages need no backend.	None
batch_size	int	The number of images submitted to the backend at once.	4
on_error	Literal['raise', 'ignore']	'raise' raises when an image fails, and 'ignore' returns None for it and warns.	'raise'
random_state	int | None	Controls the seeds sent to the backend.	None

Attributes:

Name	Type	Description
backend_	Backend | None	The resolved backend, or None for collages.
mask_	Image | None	The read mask.

Examples:

>>> from PIL import Image
>>> from skai.generation import ImageToImageGenerator
>>> generator = ImageToImageGenerator(operation='upscale', scale=2, backend='dummy').fit()
>>> generator.predict([Image.new('RGB', (64, 48))])[0].size
(128, 96)

Source code in src/skai/generation/_image.py

def __init__(
    self,
    *,
    operation: Literal['edit', 'upscale', 'remove_background', 'collage'] = 'edit',
    instructions: str | None = None,
    mask: ImageSource | None = None,
    scale: int = 2,
    columns: int | None = None,
    tile: int = 256,
    output_format: Literal['object', 'bytes', 'path'] = 'object',
    output_dir: str | PathLike | None = None,
    backend: str | Backend | None = None,
    batch_size: int = 4,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.operation = operation
    self.instructions = instructions
    self.mask = mask
    self.scale = scale
    self.columns = columns
    self.tile = tile
    self.output_format = output_format
    self.output_dir = output_dir
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

ImageToTextGenerator(*, instructions=None, operation='instruct', format='text', max_length=None, k_shot=None, max_examples=8, adaptation='examples', use_y='auto', backend=None, batch_size=8, on_error='raise', random_state=None)

Bases: BaseTextGenerator

Return one text per image, by following instructions or by transcribing the text written in it.

With operation='instruct', the image is described, answered about or captioned as the instructions ask, and fit(X, y) with reference texts as y passes example pairs to the backend. With operation='transcribe', the text of the image is returned exactly, as plain text or as Markdown that keeps the page structure.

Parameters:

Name	Type	Description	Default
instructions	str | None	What to produce from every image, such as 'Describe the image.'. Required with operation='instruct'.	None
operation	Literal['instruct', 'transcribe']	'instruct' or 'transcribe'.	'instruct'
format	Literal['text', 'markdown']	'text' or 'markdown', the form of transcriptions.	'text'
max_length	int | Literal['auto'] | None	The maximum number of words of every output. 'auto' learns it from the references as the 90th percentile of their word counts, and None sets no limit.	None
k_shot	int | ArrayLike | None	How many training images are passed to the backend as examples with their references. None passes up to max_examples, 0 passes none, an integer selects that many at random, and an array of indices selects those images.	None
max_examples	int | None	The maximum number of examples when k_shot is None.	8
adaptation	Literal['examples', 'finetune']	'examples' passes examples to the backend at prediction. 'finetune' trains a model on all the training data in fit, on backends that support fine-tuning, and sends later requests to it.	'examples'
use_y	Literal['auto'] | bool	'auto' learns from y in fit but not in fit_transform, which pipelines use for intermediate steps. True always learns from y, and False never does.	'auto'
backend	str | Backend | None	The backend, as '<provider>', '<provider>:<model>' or a backend object. The local Hugging Face backend is used when it is None.	None
batch_size	int	The number of images submitted to the backend at once.	8
on_error	Literal['raise', 'ignore']	'raise' raises when an image fails, and 'ignore' returns None for it and warns.	'raise'
random_state	int | None	Controls the selection of examples and the seeds sent to the backend.	None

Attributes:

Name	Type	Description
backend_	Backend	The resolved backend.
examples_	tuple	The pairs of an image and its reference passed to the backend.
max_length_	int | None	The maximum number of words of every output.
model_	str	The id of the fine-tuned model. It is set only with adaptation='finetune'.

Examples:

>>> from PIL import Image
>>> from skai.generation import ImageToTextGenerator
>>> generator = ImageToTextGenerator(instructions='Describe the image.', backend='dummy').fit()
>>> generator.predict([Image.new('RGB', (64, 48))])[0]
'A 64x48 image.'

Source code in src/skai/generation/_text.py

def __init__(
    self,
    *,
    instructions: str | None = None,
    operation: Literal['instruct', 'transcribe'] = 'instruct',
    format: Literal['text', 'markdown'] = 'text',  # noqa: A002 - the name of the output format of transcriptions
    max_length: int | Literal['auto'] | None = None,
    k_shot: int | ArrayLike | None = None,
    max_examples: int | None = 8,
    adaptation: Literal['examples', 'finetune'] = 'examples',
    use_y: Literal['auto'] | bool = 'auto',
    backend: str | Backend | None = None,
    batch_size: int = 8,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.instructions = instructions
    self.operation = operation
    self.format = format
    self.max_length = max_length
    self.k_shot = k_shot
    self.max_examples = max_examples
    self.adaptation = adaptation
    self.use_y = use_y
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

ImageToVideoGenerator(*, operation='animate', instructions=None, duration=2.0, fps=None, output_format='object', output_dir=None, backend=None, batch_size=1, on_error='raise', random_state=None)

Bases: BaseMediaGenerator

Turn every image into a short video that starts from it, with optional instructions for the motion.

Parameters:

Name	Type	Description	Default
operation	Literal['animate']	'animate', the only operation.	'animate'
instructions	str | None	A description of the motion.	None
duration	float	The length of every video in seconds.	2.0
fps	int | None	The number of frames per second, or None for the model's default.	None
output_format	Literal['object', 'bytes', 'path']	'object' returns Video objects, 'bytes' MP4 bytes, and 'path' the paths of new MP4 files.	'object'
output_dir	str | PathLike | None	The directory of 'path' outputs. A new temporary directory is used when it is None.	None
backend	str | Backend | None	The backend, as '<provider>', '<provider>:<model>' or a backend object. The local Hugging Face backend is used when it is None.	None
batch_size	int	The number of images submitted to the backend at once.	1
on_error	Literal['raise', 'ignore']	'raise' raises when an image fails, and 'ignore' returns None for it and warns.	'raise'
random_state	int | None	Controls the seeds sent to the backend.	None

Attributes:

Name	Type	Description
backend_	Backend	The resolved backend.

Examples:

>>> from PIL import Image
>>> from skai.generation import ImageToVideoGenerator
>>> generator = ImageToVideoGenerator(duration=1, fps=8, backend='dummy').fit()
>>> generator.predict([Image.new('RGB', (64, 48))])[0].duration
1.0

Source code in src/skai/generation/_video.py

def __init__(
    self,
    *,
    operation: Literal['animate'] = 'animate',
    instructions: str | None = None,
    duration: float = 2.0,
    fps: int | None = None,
    output_format: Literal['object', 'bytes', 'path'] = 'object',
    output_dir: str | PathLike | None = None,
    backend: str | Backend | None = None,
    batch_size: int = 1,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.operation = operation
    self.instructions = instructions
    self.duration = duration
    self.fps = fps
    self.output_format = output_format
    self.output_dir = output_dir
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

MultimodalToAudioGenerator(*, operation='mix', volumes=None, duck_under=None, duck_for=None, duck_level=0.25, language=None, output_format='object', output_dir=None, backend=None, batch_size=16, on_error='raise', random_state=None)

Bases: BaseMediaGenerator

Mix the named tracks of every multimodal sample, or speak its text in its own voice.

With operation='mix' (the default), a sample is a dict, or a DataFrame row, of named tracks, mixed locally into one track as long as the longest. A track is audio, a video whose audio track is used, or a list of timed records with start and media, such as segmenter regions, each placed at its start. With operation='speech', the backend speaks the text of every sample in its voice, a voice name or reference audio of the voice to clone.

Parameters:

Name	Type	Description	Default
operation	Literal['mix', 'speech']	'mix' or 'speech'.	'mix'
volumes	dict[str, float] | None	The gain of every track by name, one when it is missing.	None
duck_under	list[str] | None	The names of the tracks to lower while duck_for sounds, such as ['music'].	None
duck_for	str | None	The name of the track whose sound lowers the others, such as 'voice'.	None
duck_level	float	The gain of the lowered tracks while duck_for sounds.	0.25
language	str | None	The language of the texts spoken with operation='speech'.	None
output_format	Literal['object', 'bytes', 'path']	'object' returns Audio objects, 'bytes' WAV bytes, and 'path' the paths of new WAV files.	'object'
output_dir	str | PathLike | None	The directory of 'path' outputs. A new temporary directory is used when it is None.	None
backend	str | Backend | None	The backend, as '<provider>', '<provider>:<model>' or a backend object. The local Hugging Face backend is used when it is None. Mixing needs no backend.	None
batch_size	int	The number of samples submitted to the backend at once.	16
on_error	Literal['raise', 'ignore']	'raise' raises when a sample fails, and 'ignore' returns None for it and warns.	'raise'
random_state	int | None	Controls the seeds sent to the backend.	None

Attributes:

Name	Type	Description
backend_	Backend | None	The resolved backend, or None for mixing.

Examples:

>>> import numpy as np
>>> from skai.generation import MultimodalToAudioGenerator
>>> sample = {'voice': (np.zeros(16000), 16000), 'music': (np.zeros(32000), 16000)}
>>> MultimodalToAudioGenerator(duck_under=['music'], duck_for='voice').fit().predict([sample])[0].duration
2.0

Source code in src/skai/generation/_audio.py

def __init__(
    self,
    *,
    operation: Literal['mix', 'speech'] = 'mix',
    volumes: dict[str, float] | None = None,
    duck_under: list[str] | None = None,
    duck_for: str | None = None,
    duck_level: float = 0.25,
    language: str | None = None,
    output_format: Literal['object', 'bytes', 'path'] = 'object',
    output_dir: str | PathLike | None = None,
    backend: str | Backend | None = None,
    batch_size: int = 16,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.operation = operation
    self.volumes = volumes
    self.duck_under = duck_under
    self.duck_for = duck_for
    self.duck_level = duck_level
    self.language = language
    self.output_format = output_format
    self.output_dir = output_dir
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

MultimodalToImageGenerator(*, operation='edit', output_format='object', output_dir=None, backend=None, batch_size=4, on_error='raise', random_state=None)

Bases: BaseMediaGenerator

Edit the image of every multimodal sample with the instructions of the same sample, inside its mask.

A sample is a dict, or a DataFrame row, with image and instructions, and an optional mask, a grayscale image where white marks the region to edit. It lets every image get its own edit, such as the translated text of a comic panel.

Parameters:

Name	Type	Description	Default
operation	Literal['edit']	'edit', the only operation.	'edit'
output_format	Literal['object', 'bytes', 'path']	'object' returns PIL images, 'bytes' PNG bytes, and 'path' the paths of new PNG files.	'object'
output_dir	str | PathLike | None	The directory of 'path' outputs. A new temporary directory is used when it is None.	None
backend	str | Backend | None	The backend, as '<provider>', '<provider>:<model>' or a backend object. The local Hugging Face backend is used when it is None.	None
batch_size	int	The number of samples submitted to the backend at once.	4
on_error	Literal['raise', 'ignore']	'raise' raises when a sample fails, and 'ignore' returns None for it and warns.	'raise'
random_state	int | None	Controls the seeds sent to the backend.	None

Attributes:

Name	Type	Description
backend_	Backend	The resolved backend.

Examples:

>>> from PIL import Image
>>> from skai.generation import MultimodalToImageGenerator
>>> sample = {'image': Image.new('RGB', (64, 48)), 'instructions': 'Add snow.'}
>>> MultimodalToImageGenerator(backend='dummy').fit().predict([sample])[0].size
(64, 48)

Source code in src/skai/generation/_image.py

def __init__(
    self,
    *,
    operation: Literal['edit'] = 'edit',
    output_format: Literal['object', 'bytes', 'path'] = 'object',
    output_dir: str | PathLike | None = None,
    backend: str | Backend | None = None,
    batch_size: int = 4,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.operation = operation
    self.output_format = output_format
    self.output_dir = output_dir
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

MultimodalToTextGenerator(*, instructions=None, operation='instruct', n_frames=8, max_length=None, k_shot=None, max_examples=4, adaptation='examples', use_y='auto', backend=None, batch_size=8, on_error='raise', random_state=None)

Bases: BaseTextGenerator

Return one text per multimodal sample by following instructions, with every named part in one request.

A sample is a dict, or a DataFrame row, of named texts, images, audio and videos. Strings are texts unless they are URLs or paths of existing media files. fit(X, y) with reference texts as y passes example pairs to the backend.

Parameters:

Name	Type	Description	Default
instructions	str | None	What to produce from every sample, such as 'Write a caption.'.	None
operation	Literal['instruct']	'instruct', the only operation.	'instruct'
n_frames	int	The number of frames sampled from every video part.	8
max_length	int | Literal['auto'] | None	The maximum number of words of every output. 'auto' learns it from the references as the 90th percentile of their word counts, and None sets no limit.	None
k_shot	int | ArrayLike | None	How many training samples are passed to the backend as examples with their references. None passes up to max_examples, 0 passes none, an integer selects that many at random, and an array of indices selects those samples.	None
max_examples	int | None	The maximum number of examples when k_shot is None.	4
adaptation	Literal['examples', 'finetune']	'examples' passes examples to the backend at prediction. 'finetune' trains a model on all the training data in fit, on backends that support fine-tuning, and sends later requests to it.	'examples'
use_y	Literal['auto'] | bool	'auto' learns from y in fit but not in fit_transform, which pipelines use for intermediate steps. True always learns from y, and False never does.	'auto'
backend	str | Backend | None	The backend, as '<provider>', '<provider>:<model>' or a backend object. The local Hugging Face backend is used when it is None.	None
batch_size	int	The number of samples submitted to the backend at once.	8
on_error	Literal['raise', 'ignore']	'raise' raises when a sample fails, and 'ignore' returns None for it and warns.	'raise'
random_state	int | None	Controls the selection of examples and the seeds sent to the backend.	None

Attributes:

Name	Type	Description
backend_	Backend	The resolved backend.
examples_	tuple	The pairs of a sample and its reference passed to the backend.
max_length_	int | None	The maximum number of words of every output.

Examples:

>>> from PIL import Image
>>> from skai.generation import MultimodalToTextGenerator
>>> generator = MultimodalToTextGenerator(instructions='Write a caption.', backend='dummy').fit()
>>> generator.predict([{'photo': Image.new('RGB', (8, 8)), 'notes': 'Crete'}])[0]
'photo: A 8x8 image. notes: Crete'

Source code in src/skai/generation/_text.py

def __init__(
    self,
    *,
    instructions: str | None = None,
    operation: Literal['instruct'] = 'instruct',
    n_frames: int = 8,
    max_length: int | Literal['auto'] | None = None,
    k_shot: int | ArrayLike | None = None,
    max_examples: int | None = 4,
    adaptation: Literal['examples', 'finetune'] = 'examples',
    use_y: Literal['auto'] | bool = 'auto',
    backend: str | Backend | None = None,
    batch_size: int = 8,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.instructions = instructions
    self.operation = operation
    self.n_frames = n_frames
    self.max_length = max_length
    self.k_shot = k_shot
    self.max_examples = max_examples
    self.adaptation = adaptation
    self.use_y = use_y
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

MultimodalToVideoGenerator(*, operation='lip_sync', burn=True, output_format='object', output_dir=None, backend=None, batch_size=1, on_error='raise', random_state=None)

Bases: BaseMediaGenerator

Return one video per multimodal sample: lip-synced to new speech, with a new soundtrack, or subtitled.

A sample is a dict, or a DataFrame row, of named parts. 'lip_sync' makes video speak audio with the backend, 'mux' replaces the soundtrack of video with audio, and 'subtitle' shows transcript as subtitles, both locally. A transcript is a string, or a list of timed records with start, end and text, such as the output of AudioSegmenter.

Parameters:

Name	Type	Description	Default
operation	Literal['lip_sync', 'mux', 'subtitle']	'lip_sync', 'mux' or 'subtitle'.	'lip_sync'
burn	bool	Whether subtitles are drawn on the frames. Otherwise they are attached as subtitles in SRT format and written next to the video.	True
output_format	Literal['object', 'bytes', 'path']	'object' returns Video objects, 'bytes' MP4 bytes, and 'path' the paths of new MP4 files.	'object'
output_dir	str | PathLike | None	The directory of 'path' outputs. A new temporary directory is used when it is None.	None
backend	str | Backend | None	The backend, as '<provider>', '<provider>:<model>' or a backend object. An open-source model on Replicate is used for lip sync when it is None. Muxing and subtitles need no backend.	None
batch_size	int	The number of samples submitted to the backend at once.	1
on_error	Literal['raise', 'ignore']	'raise' raises when a sample fails, and 'ignore' returns None for it and warns.	'raise'
random_state	int | None	Controls the seeds sent to the backend.	None

Attributes:

Name	Type	Description
backend_	Backend | None	The resolved backend, or None for muxing and subtitles.

Examples:

>>> import numpy as np
>>> from skai.core import Video
>>> from skai.generation import MultimodalToVideoGenerator
>>> sample = {'video': Video(np.zeros((8, 8, 8, 3), dtype=np.uint8), fps=4), 'audio': (np.zeros(8000), 16000)}
>>> MultimodalToVideoGenerator(operation='mux').fit().predict([sample])[0].audio.duration
2.0

Source code in src/skai/generation/_video.py

def __init__(
    self,
    *,
    operation: Literal['lip_sync', 'mux', 'subtitle'] = 'lip_sync',
    burn: bool = True,
    output_format: Literal['object', 'bytes', 'path'] = 'object',
    output_dir: str | PathLike | None = None,
    backend: str | Backend | None = None,
    batch_size: int = 1,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.operation = operation
    self.burn = burn
    self.output_format = output_format
    self.output_dir = output_dir
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

TextToAudioGenerator(*, operation='speech', voice=None, language=None, duration=10.0, output_format='object', output_dir=None, backend=None, batch_size=16, on_error='raise', random_state=None)

Bases: BaseMediaGenerator

Return one audio sample per text: the text spoken, or music or a sound it describes.

With operation='speech', voice is a voice name of the backend, or one recording or a list of recordings of the voice to clone, which fit joins into voice_.

Parameters:

Name	Type	Description	Default
operation	Literal['speech', 'music', 'sound']	'speech', 'music' or 'sound'.	'speech'
voice	VoiceSource | None	A voice name, or one recording or a list of recordings of the target voice.	None
language	str | None	The language of the text.	None
duration	float	The length of music and sounds in seconds.	10.0
output_format	Literal['object', 'bytes', 'path']	'object' returns Audio objects, 'bytes' WAV bytes, and 'path' the paths of new WAV files.	'object'
output_dir	str | PathLike | None	The directory of 'path' outputs. A new temporary directory is used when it is None.	None
backend	str | Backend | None	The backend, as '<provider>', '<provider>:<model>' or a backend object. The local Hugging Face backend is used when it is None.	None
batch_size	int	The number of texts submitted to the backend at once.	16
on_error	Literal['raise', 'ignore']	'raise' raises when a text fails, and 'ignore' returns None for it and warns.	'raise'
random_state	int | None	Controls the seeds sent to the backend.	None

Attributes:

Name	Type	Description
backend_	Backend	The resolved backend.
voice_	str | Audio | None	The voice name, or the joined reference audio of the voice to clone.

Examples:

>>> from skai.generation import TextToAudioGenerator
>>> generator = TextToAudioGenerator(operation='music', duration=5, backend='dummy').fit()
>>> generator.predict(['calm piano'])[0].duration
5.0

Source code in src/skai/generation/_audio.py

def __init__(
    self,
    *,
    operation: Literal['speech', 'music', 'sound'] = 'speech',
    voice: VoiceSource | None = None,
    language: str | None = None,
    duration: float = 10.0,
    output_format: Literal['object', 'bytes', 'path'] = 'object',
    output_dir: str | PathLike | None = None,
    backend: str | Backend | None = None,
    batch_size: int = 16,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.operation = operation
    self.voice = voice
    self.language = language
    self.duration = duration
    self.output_format = output_format
    self.output_dir = output_dir
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

TextToImageGenerator(*, operation='generate', size=(1024, 1024), style=None, style_images=None, negative_prompt=None, reference_images=None, reference_prompt=None, output_format='object', output_dir=None, backend=None, batch_size=4, on_error='raise', random_state=None)

Bases: BaseMediaGenerator

Create one image per prompt, in a style given or learned from reference images.

fit learns style_, a description of the shared style of style_images written by the backend's image description, and applies it to every prompt unless style is given. reference_images show a subject that every image keeps consistent, on backends that take reference images, and with reference_prompt instead, fit generates one reference image of the subject and sends it with every request.

Parameters:

Name	Type	Description	Default
operation	Literal['generate']	'generate', the only operation.	'generate'
size	tuple[int, int]	The width and height of every image in pixels.	(1024, 1024)
style	str | None	A description of the visual style. It takes precedence over the learned style.	None
style_images	list | None	Images whose shared style is learned in fit.	None
negative_prompt	str | None	A description of what the images must not show.	None
reference_images	list | None	Images of a subject that every image must show consistently.	None
reference_prompt	str | None	A description of the subject, from which fit generates the reference image when reference_images is not given.	None
output_format	Literal['object', 'bytes', 'path']	'object' returns PIL images, 'bytes' PNG bytes, and 'path' the paths of new PNG files.	'object'
output_dir	str | PathLike | None	The directory of 'path' outputs. A new temporary directory is used when it is None.	None
backend	str | Backend | None	The backend, as '<provider>', '<provider>:<model>' or a backend object. The local Hugging Face backend is used when it is None.	None
batch_size	int	The number of prompts submitted to the backend at once.	4
on_error	Literal['raise', 'ignore']	'raise' raises when a prompt fails, and 'ignore' returns None for it and warns.	'raise'
random_state	int | None	Controls the seeds sent to the backend.	None

Attributes:

Name	Type	Description
backend_	Backend	The resolved backend.
style_	str | None	The style learned from style_images.
reference_images_	tuple	The read or generated reference images.

Examples:

>>> from skai.generation import TextToImageGenerator
>>> generator = TextToImageGenerator(size=(64, 48), backend='dummy').fit()
>>> generator.predict(['a cat'])[0].size
(64, 48)

Source code in src/skai/generation/_image.py

def __init__(
    self,
    *,
    operation: Literal['generate'] = 'generate',
    size: tuple[int, int] = (1024, 1024),
    style: str | None = None,
    style_images: list | None = None,
    negative_prompt: str | None = None,
    reference_images: list | None = None,
    reference_prompt: str | None = None,
    output_format: Literal['object', 'bytes', 'path'] = 'object',
    output_dir: str | PathLike | None = None,
    backend: str | Backend | None = None,
    batch_size: int = 4,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.operation = operation
    self.size = size
    self.style = style
    self.style_images = style_images
    self.negative_prompt = negative_prompt
    self.reference_images = reference_images
    self.reference_prompt = reference_prompt
    self.output_format = output_format
    self.output_dir = output_dir
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

TextToTextGenerator(*, instructions=None, operation='instruct', max_length=None, learn_terms=False, k_shot=None, max_examples=8, adaptation='examples', use_y='auto', backend=None, batch_size=16, on_error='raise', random_state=None)

Bases: BaseTextGenerator

Return one text per text by following instructions, such as a summary, a translation or an answer.

fit(X, y) with reference texts as y passes k_shot text and reference pairs to the backend as examples, learns max_length_ with max_length='auto', and learns a glossary of consistently rendered terms with learn_terms=True.

Parameters:

Name	Type	Description	Default
instructions	str | None	What to produce from every text, such as 'Summarize the text.' or 'Translate into Greek.'.	None
operation	Literal['instruct']	'instruct', the only operation.	'instruct'
max_length	int | Literal['auto'] | None	The maximum number of words of every output. 'auto' learns it from the references as the 90th percentile of their word counts, and None sets no limit.	None
learn_terms	bool	Whether fit learns a glossary of the terms that appear in at least two texts, mapped to the term their references share.	False
k_shot	int | ArrayLike | None	How many training texts are passed to the backend as examples with their references. None passes up to max_examples, 0 passes none, an integer selects that many at random, and an array of indices selects those texts.	None
max_examples	int | None	The maximum number of examples when k_shot is None.	8
adaptation	Literal['examples', 'finetune']	'examples' passes examples to the backend at prediction. 'finetune' trains a model on all the training data in fit, on backends that support fine-tuning, and sends later requests to it.	'examples'
use_y	Literal['auto'] | bool	'auto' learns from y in fit but not in fit_transform, which pipelines use for intermediate steps. True always learns from y, and False never does.	'auto'
backend	str | Backend | None	The backend, as '<provider>', '<provider>:<model>' or a backend object. The local Hugging Face backend is used when it is None.	None
batch_size	int	The number of texts submitted to the backend at once.	16
on_error	Literal['raise', 'ignore']	'raise' raises when a text fails, and 'ignore' returns None for it and warns.	'raise'
random_state	int | None	Controls the selection of examples and the seeds sent to the backend.	None

Attributes:

Name	Type	Description
backend_	Backend	The resolved backend.
examples_	tuple	The pairs of a text and its reference passed to the backend.
max_length_	int | None	The maximum number of words of every output.
glossary_	tuple	Pairs of a term and its required rendering.
model_	str	The id of the fine-tuned model. It is set only with adaptation='finetune'.

Examples:

>>> from skai.generation import TextToTextGenerator
>>> generator = TextToTextGenerator(instructions='Summarize the text.', max_length=3, backend='dummy').fit()
>>> generator.predict(['One two three four five.']).tolist()
['One two three']

Source code in src/skai/generation/_text.py

def __init__(
    self,
    *,
    instructions: str | None = None,
    operation: Literal['instruct'] = 'instruct',
    max_length: int | Literal['auto'] | None = None,
    learn_terms: bool = False,
    k_shot: int | ArrayLike | None = None,
    max_examples: int | None = 8,
    adaptation: Literal['examples', 'finetune'] = 'examples',
    use_y: Literal['auto'] | bool = 'auto',
    backend: str | Backend | None = None,
    batch_size: int = 16,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.instructions = instructions
    self.operation = operation
    self.max_length = max_length
    self.learn_terms = learn_terms
    self.k_shot = k_shot
    self.max_examples = max_examples
    self.adaptation = adaptation
    self.use_y = use_y
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

TextToVideoGenerator(*, operation='generate', duration=4.0, size=(1280, 720), fps=None, style=None, negative_prompt=None, output_format='object', output_dir=None, backend=None, batch_size=1, on_error='raise', random_state=None)

Bases: BaseMediaGenerator

Create one video per prompt, of a duration and size.

Parameters:

Name	Type	Description	Default
operation	Literal['generate']	'generate', the only operation.	'generate'
duration	float	The length of every video in seconds.	4.0
size	tuple[int, int]	The width and height of every video in pixels.	(1280, 720)
fps	int | None	The number of frames per second, or None for the model's default.	None
style	str | None	A description of the visual style.	None
negative_prompt	str | None	A description of what the videos must not show.	None
output_format	Literal['object', 'bytes', 'path']	'object' returns Video objects, 'bytes' MP4 bytes, and 'path' the paths of new MP4 files.	'object'
output_dir	str | PathLike | None	The directory of 'path' outputs. A new temporary directory is used when it is None.	None
backend	str | Backend | None	The backend, as '<provider>', '<provider>:<model>' or a backend object. The local Hugging Face backend is used when it is None.	None
batch_size	int	The number of prompts submitted to the backend at once.	1
on_error	Literal['raise', 'ignore']	'raise' raises when a prompt fails, and 'ignore' returns None for it and warns.	'raise'
random_state	int | None	Controls the seeds sent to the backend.	None

Attributes:

Name	Type	Description
backend_	Backend	The resolved backend.

Examples:

>>> from skai.generation import TextToVideoGenerator
>>> generator = TextToVideoGenerator(duration=2, size=(320, 240), fps=8, backend='dummy').fit()
>>> generator.predict(['a boat at sea'])[0].duration
2.0

Source code in src/skai/generation/_video.py

def __init__(
    self,
    *,
    operation: Literal['generate'] = 'generate',
    duration: float = 4.0,
    size: tuple[int, int] = (1280, 720),
    fps: int | None = None,
    style: str | None = None,
    negative_prompt: str | None = None,
    output_format: Literal['object', 'bytes', 'path'] = 'object',
    output_dir: str | PathLike | None = None,
    backend: str | Backend | None = None,
    batch_size: int = 1,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.operation = operation
    self.duration = duration
    self.size = size
    self.fps = fps
    self.style = style
    self.negative_prompt = negative_prompt
    self.output_format = output_format
    self.output_dir = output_dir
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

VideoToAudioGenerator(*, operation='extract', output_format='object', output_dir=None, on_error='raise')

Bases: BaseMediaGenerator

Return the mono audio track of every video, or silence as long as a video without one, locally.

Parameters:

Name	Type	Description	Default
operation	Literal['extract']	'extract', the only operation.	'extract'
output_format	Literal['object', 'bytes', 'path']	'object' returns Audio objects, 'bytes' WAV bytes, and 'path' the paths of new WAV files.	'object'
output_dir	str | PathLike | None	The directory of 'path' outputs. A new temporary directory is used when it is None.	None
on_error	Literal['raise', 'ignore']	'raise' raises when a video fails, and 'ignore' returns None for it and warns.	'raise'

Attributes:

Name	Type	Description
backend_	None	Always None, since extraction runs locally.

Examples:

>>> import numpy as np
>>> from skai.core import Video
>>> from skai.generation import VideoToAudioGenerator
>>> video = Video(np.zeros((8, 8, 8, 3), dtype=np.uint8), fps=4)
>>> VideoToAudioGenerator().fit().predict([video])[0].duration
2.0

Source code in src/skai/generation/_audio.py

def __init__(
    self,
    *,
    operation: Literal['extract'] = 'extract',
    output_format: Literal['object', 'bytes', 'path'] = 'object',
    output_dir: str | PathLike | None = None,
    on_error: Literal['raise', 'ignore'] = 'raise',
) -> None:
    self.operation = operation
    self.output_format = output_format
    self.output_dir = output_dir
    self.on_error = on_error

VideoToImageGenerator(*, operation='thumbnail', n_frames=8, at=0.0, output_format='object', output_dir=None, backend=None, batch_size=4, on_error='raise', random_state=None)

Bases: BaseMediaGenerator

Return one image per video: the thumbnail chosen by the backend, or the frame at a time.

Parameters:

Name	Type	Description	Default
operation	Literal['thumbnail', 'frame']	'thumbnail' asks the backend for the frame that best represents the video among n_frames sampled frames, and 'frame' returns the frame at at seconds, locally.	'thumbnail'
n_frames	int	The number of candidate frames of a thumbnail.	8
at	float	The time of the frame in seconds, clamped to the video.	0.0
output_format	Literal['object', 'bytes', 'path']	'object' returns PIL images, 'bytes' PNG bytes, and 'path' the paths of new PNG files.	'object'
output_dir	str | PathLike | None	The directory of 'path' outputs. A new temporary directory is used when it is None.	None
backend	str | Backend | None	The backend, as '<provider>', '<provider>:<model>' or a backend object. The local Hugging Face backend is used when it is None. Frames need no backend.	None
batch_size	int	The number of videos submitted to the backend at once.	4
on_error	Literal['raise', 'ignore']	'raise' raises when a video fails, and 'ignore' returns None for it and warns.	'raise'
random_state	int | None	Controls the seeds sent to the backend.	None

Attributes:

Name	Type	Description
backend_	Backend | None	The resolved backend, or None for frames.

Examples:

>>> import numpy as np
>>> from skai.core import Video
>>> from skai.generation import VideoToImageGenerator
>>> video = Video(np.zeros((8, 48, 64, 3), dtype=np.uint8), fps=4)
>>> VideoToImageGenerator(operation='frame', at=1.0).fit().predict([video])[0].size
(64, 48)

Source code in src/skai/generation/_image.py

def __init__(
    self,
    *,
    operation: Literal['thumbnail', 'frame'] = 'thumbnail',
    n_frames: int = 8,
    at: float = 0.0,
    output_format: Literal['object', 'bytes', 'path'] = 'object',
    output_dir: str | PathLike | None = None,
    backend: str | Backend | None = None,
    batch_size: int = 4,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.operation = operation
    self.n_frames = n_frames
    self.at = at
    self.output_format = output_format
    self.output_dir = output_dir
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

VideoToTextGenerator(*, instructions=None, operation='instruct', n_frames=8, language=None, max_vocabulary=50, max_length=None, k_shot=None, max_examples=4, adaptation='examples', use_y='auto', backend=None, batch_size=4, on_error='raise', random_state=None)

Bases: BaseTextGenerator

Return one text per video, by following instructions or by transcribing its speech.

With operation='instruct' (the default), the model sees frames sampled evenly over every video, and the audio track on backends that accept audio, and produces what the instructions ask. With operation='transcribe', the speech of the audio track is transcribed, learning a vocabulary from reference transcripts as AudioToTextGenerator does, and a video without an audio track gives an empty transcript.

Parameters:

Name	Type	Description	Default
instructions	str | None	What to produce from every video. Required with operation='instruct'.	None
operation	Literal['instruct', 'transcribe']	'instruct' or 'transcribe'.	'instruct'
n_frames	int	The number of frames sampled from every video.	8
language	str | None	The language of the speech. It is detected when it is None.	None
max_vocabulary	int	The maximum number of terms in the vocabulary, the most frequent first.	50
max_length	int | Literal['auto'] | None	The maximum number of words of every output. 'auto' learns it from the references as the 90th percentile of their word counts, and None sets no limit.	None
k_shot	int | ArrayLike | None	How many training videos are passed to the backend as examples with their references, with operation='instruct'. None passes up to max_examples, 0 passes none, an integer selects that many at random, and an array of indices selects those videos.	None
max_examples	int | None	The maximum number of examples when k_shot is None.	4
adaptation	Literal['examples', 'finetune']	'examples' passes examples to the backend at prediction. 'finetune' trains a model on all the training data in fit, on backends that support fine-tuning, with operation='instruct'.	'examples'
use_y	Literal['auto'] | bool	'auto' learns from y in fit but not in fit_transform, which pipelines use for intermediate steps. True always learns from y, and False never does.	'auto'
backend	str | Backend | None	The backend, as '<provider>', '<provider>:<model>' or a backend object. The local Hugging Face backend is used when it is None.	None
batch_size	int	The number of videos submitted to the backend at once.	4
on_error	Literal['raise', 'ignore']	'raise' raises when a video fails, and 'ignore' returns None for it and warns.	'raise'
random_state	int | None	Controls the selection of examples and the seeds sent to the backend.	None

Attributes:

Name	Type	Description
backend_	Backend	The resolved backend.
vocabulary_	tuple	The domain terms learned from the reference transcripts.
examples_	tuple	The pairs of a video and its reference passed to the backend.
max_length_	int | None	The maximum number of words of every output.

Examples:

>>> import numpy as np
>>> from skai.core import Video
>>> from skai.generation import VideoToTextGenerator
>>> video = Video(np.zeros((8, 8, 8, 3), dtype=np.uint8), fps=4)
>>> VideoToTextGenerator(operation='transcribe', backend='dummy').fit().predict([video])[0]
''

Source code in src/skai/generation/_text.py

def __init__(
    self,
    *,
    instructions: str | None = None,
    operation: Literal['instruct', 'transcribe'] = 'instruct',
    n_frames: int = 8,
    language: str | None = None,
    max_vocabulary: int = 50,
    max_length: int | Literal['auto'] | None = None,
    k_shot: int | ArrayLike | None = None,
    max_examples: int | None = 4,
    adaptation: Literal['examples', 'finetune'] = 'examples',
    use_y: Literal['auto'] | bool = 'auto',
    backend: str | Backend | None = None,
    batch_size: int = 4,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.instructions = instructions
    self.operation = operation
    self.n_frames = n_frames
    self.language = language
    self.max_vocabulary = max_vocabulary
    self.max_length = max_length
    self.k_shot = k_shot
    self.max_examples = max_examples
    self.adaptation = adaptation
    self.use_y = use_y
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state

VideoToVideoGenerator(*, operation='reframe', aspect_ratio=9 / 16, subject='person', n_frames=8, output_format='object', output_dir=None, backend=None, batch_size=2, on_error='raise', random_state=None)

Bases: BaseMediaGenerator

Return one video per video, reframed to an aspect ratio following its main subject, or a joined list of clips.

Reframing detects subject in n_frames sampled frames with the backend and crops every frame around the detected centers, interpolated over time. Concatenation joins a list of clips locally, at the size and frame rate of the first clip.

Parameters:

Name	Type	Description	Default
operation	Literal['reframe', 'concatenate']	'reframe' or 'concatenate'.	'reframe'
aspect_ratio	float	The width divided by the height of reframed videos, such as 9 / 16 for vertical video.	9 / 16
subject	str	The kind of subject that reframing follows.	'person'
n_frames	int	The number of frames in which reframing detects the subject.	8
output_format	Literal['object', 'bytes', 'path']	'object' returns Video objects, 'bytes' MP4 bytes, and 'path' the paths of new MP4 files.	'object'
output_dir	str | PathLike | None	The directory of 'path' outputs. A new temporary directory is used when it is None.	None
backend	str | Backend | None	The backend, as '<provider>', '<provider>:<model>' or a backend object. The local Hugging Face backend is used when it is None. Concatenation needs no backend.	None
batch_size	int	The number of videos submitted to the backend at once.	2
on_error	Literal['raise', 'ignore']	'raise' raises when a video fails, and 'ignore' returns None for it and warns.	'raise'
random_state	int | None	Controls the seeds sent to the backend.	None

Attributes:

Name	Type	Description
backend_	Backend | None	The resolved backend, or None for concatenation.

Examples:

>>> import numpy as np
>>> from skai.core import Video
>>> from skai.generation import VideoToVideoGenerator
>>> clip = Video(np.zeros((4, 8, 8, 3), dtype=np.uint8), fps=4)
>>> VideoToVideoGenerator(operation='concatenate').fit().predict([[clip, clip, clip]])[0].duration
3.0

Source code in src/skai/generation/_video.py

def __init__(
    self,
    *,
    operation: Literal['reframe', 'concatenate'] = 'reframe',
    aspect_ratio: float = 9 / 16,
    subject: str = 'person',
    n_frames: int = 8,
    output_format: Literal['object', 'bytes', 'path'] = 'object',
    output_dir: str | PathLike | None = None,
    backend: str | Backend | None = None,
    batch_size: int = 2,
    on_error: Literal['raise', 'ignore'] = 'raise',
    random_state: int | None = None,
) -> None:
    self.operation = operation
    self.aspect_ratio = aspect_ratio
    self.subject = subject
    self.n_frames = n_frames
    self.output_format = output_format
    self.output_dir = output_dir
    self.backend = backend
    self.batch_size = batch_size
    self.on_error = on_error
    self.random_state = random_state