Skip to content

Neural networks

Neural networks: Hopfield's memory, its diluted version, reservoir computers, and Elman's recurrent network.

BinaryWeightElement(label, on=0.0)

Bases: Element

Weight of a binary perceptron, where the state 1 stands for +1 and the state 0 for -1.

Parameters:

Name Type Description Default
label str

The label of the weight or the group.

 required
on Annotated[float, StateSpec(domain=FiniteEnumeratedSet([0, 1]))]

Whether the weight is +1.

 0.0

Examples:

>>> from skcomplex.systems.computer_science.neural_networks import BinaryWeightElement
>>> BinaryWeightElement('weight', on=1.0).on
1.0
Source code in src/skcomplex/systems/computer_science/neural_networks/_perceptron.py 
36
37
38
def __init__(self, label: str, on: Annotated[float, StateSpec(domain=FiniteEnumeratedSet([0, 1]))] = 0.0) -> None:
    super().__init__(label)
    self.on = on

DilutedNeuronElement(label, visual=None, *, spin=1.0, pattern=1.0, beta=0.06)

Bases: Element

Binary neuron of an asymmetric diluted Hopfield network, updated in parallel with the others at every step.

The logit choice of diluted_hopfield draws the value s = ±1 of the neuron with probability 1 / (1 + exp(-2 s β h)), where h = Σ_j J_ij s_j is its local field, so its mean is tanh(β h).

Parameters:

Name Type Description Default
label str

The label of the neuron or the group.

 required
visual VisualSpec | None

How the element is drawn, or None for default_visual_.

 None
spin Annotated[float, StateSpec(domain=ZZ)]

The initial value, -1 or 1.

 1.0
pattern Annotated[float, ParameterSpec()]

The value of the first stored pattern at the neuron, whose overlap with the network is recorded.

 1.0
beta Annotated[float, ParameterSpec(domain=closed_open(0, oo))]

The inverse temperature β.

 0.06

Examples:

>>> from skcomplex.systems.computer_science.neural_networks import DilutedNeuronElement
>>> DilutedNeuronElement('neuron').beta
0.06
Source code in src/skcomplex/systems/computer_science/neural_networks/_diluted.py 
71
72
73
74
75
76
77
78
79
80
81
82
83
def __init__(
    self,
    label: str,
    visual: VisualSpec | None = None,
    *,
    spin: Annotated[float, StateSpec(domain=ZZ)] = 1.0,
    pattern: Annotated[float, ParameterSpec()] = 1.0,
    beta: Annotated[float, ParameterSpec(domain=RealSet.closed_open(0, oo))] = 0.06,
) -> None:
    super().__init__(label, visual=visual)
    self.spin = spin
    self.pattern = pattern
    self.beta = beta

HiddenUnitElement(label, visual=None, *, s=0.5, u=0.0, b=0.0)

Bases: Element

Hidden unit of a simple recurrent network, whose activity is a sigmoid of its recurrent input and its drive.

The drive u holds the weighted input of the previous step, and the bias b is adaptive, so training changes it.

Parameters:

Name Type Description Default
label str

The label of the element or group.

 required
visual VisualSpec | None

How the element is drawn, or None for default_visual_.

 None
s Annotated[float, StateSpec()]

The activity in (0, 1).

 0.5
u Annotated[float, StateSpec()]

The weighted input of the previous step.

 0.0
b Annotated[float, ParameterSpec(adaptive=True)]

The bias, an adaptive parameter.

 0.0

Examples:

>>> from skcomplex.systems.computer_science.neural_networks import HiddenUnitElement
>>> HiddenUnitElement('unit').s
0.5
Source code in src/skcomplex/systems/computer_science/neural_networks/_elman.py 
57
58
59
60
61
62
63
64
65
66
67
68
69
def __init__(
    self,
    label: str,
    visual: VisualSpec | None = None,
    *,
    s: Annotated[float, StateSpec()] = 0.5,
    u: Annotated[float, StateSpec()] = 0.0,
    b: Annotated[float, ParameterSpec(adaptive=True)] = 0.0,
) -> None:
    super().__init__(label, visual=visual)
    self.s = s
    self.u = u
    self.b = b

HopfieldUpdateInteraction(label, *, target)

Bases: MapInteraction

Asynchronous update of a Hopfield network, where every neuron in turn takes the sign of its weighted input.

The neurons update one after the other in a new random order every step, each reading the states the neurons before it set, so the energy -sum w_ij s_i s_j / 2 never increases. A neuron whose input is zero keeps its state.

Parameters:

Name Type Description Default
label str

The label of the interaction.

 required
target Annotated[str, RoleSpec()]

The path of the group of neurons.

 required

Examples:

>>> from skcomplex.systems.computer_science.neural_networks import HopfieldUpdateInteraction
>>> HopfieldUpdateInteraction('update', target='neurons').schedule
'random-sequential'
Source code in src/skcomplex/systems/computer_science/neural_networks/_hopfield.py 
 93
 94
 95
 96
 97
 98
 99
100
101
def __init__(self, label: str, *, target: Annotated[str, RoleSpec()]) -> None:
    super().__init__(
        label,
        target=target,
        state='spin',
        function=self._get_spin,
        schedule='random-sequential',
        form='agent',
    )

InputCouplingInteraction(label, *, target, source, D=0.0)

Bases: CouplingInteraction

Weighted input of a simple recurrent network, which sets the drive u = D x of every hidden unit each step.

The weights D are an adaptive matrix with one row per hidden unit and one column per input unit, so training changes them. The coupling runs in the phase drive of the step, which the system declares before the phases hidden of the recurrence and readout of the readout, so the hidden units read the drive of the same step.

Parameters:

Name Type Description Default
label str

The label of the interaction, unique among the parts of its system.

 required
target Annotated[str, RoleSpec()]

The path of the group of hidden units.

 required
source Annotated[str, RoleSpec()]

The path of the group of input units.

 required
D Annotated[object, ParameterSpec(shape=(target, source), adaptive=True)]

The input weights, a matrix of shape (hidden units, input units) or a number for every entry.

 0.0

Examples:

>>> from skcomplex.systems.computer_science.neural_networks import InputCouplingInteraction
>>> InputCouplingInteraction('input', target='hidden', source='inputs').write
'u'
Source code in src/skcomplex/systems/computer_science/neural_networks/_elman.py 
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
def __init__(
    self,
    label: str,
    *,
    target: Annotated[str, RoleSpec()],
    source: Annotated[str, RoleSpec()],
    D: Annotated[object, ParameterSpec(shape=('target', 'source'), adaptive=True)] = 0.0,
) -> None:
    super().__init__(
        label,
        target=target,
        source=source,
        read='x',
        write='u',
        function=self._input,
        mode='group',
        weights='D',
        form='map',
        phase='drive',
    )
    self.D = D

InputUnitElement(label, visual=None, *, x=0.0)

Bases: Element

Input unit of a simple recurrent network, whose value a dataset or a schedule sets at every step.

Parameters:

Name Type Description Default
label str

The label of the element or group.

 required
visual VisualSpec | None

How the element is drawn, or None for default_visual_.

 None
x Annotated[float, ParameterSpec()]

The input.

 0.0

Examples:

>>> from skcomplex.systems.computer_science.neural_networks import InputUnitElement
>>> InputUnitElement('bit').x
0.0
Source code in src/skcomplex/systems/computer_science/neural_networks/_elman.py 
91
92
93
94
95
96
97
98
99
def __init__(
    self,
    label: str,
    visual: VisualSpec | None = None,
    *,
    x: Annotated[float, ParameterSpec()] = 0.0,
) -> None:
    super().__init__(label, visual=visual)
    self.x = x

LorenzFlowElement(label, *, x=LORENZ_START[0], y=LORENZ_START[1], z=LORENZ_START[2])

Bases: Element

Point of the Lorenz flow that trains a reservoir, with the three variables of the flow.

Parameters:

Name Type Description Default
label str

The label of the point.

 required
x Annotated[float, StateSpec()]

The initial value of the first variable.

 LORENZ_START[0]
y Annotated[float, StateSpec()]

The initial value of the second variable.

 LORENZ_START[1]
z Annotated[float, StateSpec()]

The initial value of the third variable.

 LORENZ_START[2]

Examples:

>>> from skcomplex.systems.computer_science.neural_networks import LorenzFlowElement
>>> LorenzFlowElement('flow', z=2.0).z
2.0
Source code in src/skcomplex/systems/computer_science/neural_networks/_next_generation.py 
57
58
59
60
61
62
63
64
65
66
67
68
def __init__(
    self,
    label: str,
    *,
    x: Annotated[float, StateSpec()] = LORENZ_START[0],
    y: Annotated[float, StateSpec()] = LORENZ_START[1],
    z: Annotated[float, StateSpec()] = LORENZ_START[2],
) -> None:
    super().__init__(label)
    self.x = x
    self.y = y
    self.z = z

NeuronElement(label, visual=VisualSpec('agents', {'color_by': 'spin', 'color_range': (-1.0, 1.0)}), *, spin=1.0, weights=0.0)

Bases: Element

Binary neuron of a Hopfield network, which the update aligns with the sign of its weighted input.

The neuron sits on its node of a complete graph and holds the weights of its incoming links, one per node.

Parameters:

Name Type Description Default
label str

The label of the agent or group.

 required
visual VisualSpec | None

How the element is drawn, or None for default_visual_.

 VisualSpec('agents', {'color_by': 'spin', 'color_range': (-1.0, 1.0)})
spin Annotated[float, StateSpec(domain=ZZ)]

The initial state, -1 or 1.

 1.0
weights Annotated[object, StateSpec()]

The weights of the links from every node, an array with one value per node.

 0.0

Examples:

>>> from skcomplex.systems.computer_science.neural_networks import NeuronElement
>>> group = GroupElement('neurons', element=NeuronElement, size=4, values={'spin': 1.0})
>>> group.label, group.element.__name__, group.size
('neurons', 'NeuronElement', 4)
Source code in src/skcomplex/systems/computer_science/neural_networks/_hopfield.py 
56
57
58
59
60
61
62
63
64
65
66
def __init__(
    self,
    label: str,
    visual: VisualSpec | None = VisualSpec('agents', {'color_by': 'spin', 'color_range': (-1.0, 1.0)}),
    *,
    spin: Annotated[float, StateSpec(domain=ZZ)] = 1.0,
    weights: Annotated[object, StateSpec()] = 0.0,
) -> None:
    super().__init__(label, visual=visual)
    self.spin = spin
    self.weights = weights

OutputUnitElement(label, visual=None, *, y=0.5, c=0.0, target=0.0)

Bases: Element

Output unit of a simple recurrent network, whose activity predicts a target.

The bias c is adaptive, so training changes it, and the target is a parameter a dataset or a schedule sets.

Parameters:

Name Type Description Default
label str

The label of the element or group.

 required
visual VisualSpec | None

How the element is drawn, or None for default_visual_.

 None
y Annotated[float, StateSpec()]

The prediction in (0, 1).

 0.5
c Annotated[float, ParameterSpec(adaptive=True)]

The bias, an adaptive parameter.

 0.0
target Annotated[float, ParameterSpec()]

The value the prediction is compared with.

 0.0

Examples:

>>> from skcomplex.systems.computer_science.neural_networks import OutputUnitElement
>>> OutputUnitElement('prediction').target
0.0
Source code in src/skcomplex/systems/computer_science/neural_networks/_elman.py 
129
130
131
132
133
134
135
136
137
138
139
140
141
def __init__(
    self,
    label: str,
    visual: VisualSpec | None = None,
    *,
    y: Annotated[float, StateSpec()] = 0.5,
    c: Annotated[float, ParameterSpec(adaptive=True)] = 0.0,
    target: Annotated[float, ParameterSpec()] = 0.0,
) -> None:
    super().__init__(label, visual=visual)
    self.y = y
    self.c = c
    self.target = target

ReadoutCouplingInteraction(label, *, target, source, W=0.0)

Bases: CouplingInteraction

Readout of a simple recurrent network, y_i <- sigmoid(sum_j W_ij s_j + c_i) with the logistic sigmoid.

Every output unit reads the activities of the hidden units through the adaptive matrix W, adds its bias c, and takes the sigmoid of the sum as its next prediction.

Parameters:

Name Type Description Default
label str

The label of the interaction, unique among the parts of its system.

 required
target Annotated[str, RoleSpec()]

The path of the group of output units.

 required
source Annotated[str, RoleSpec()]

The path of the group of hidden units.

 required
W Annotated[object, ParameterSpec(shape=(target, source), adaptive=True)]

The readout weights, a matrix of shape (output units, hidden units) or a number for every entry.

 0.0

Examples:

>>> from skcomplex.systems.computer_science.neural_networks import ReadoutCouplingInteraction
>>> ReadoutCouplingInteraction('readout', target='outputs', source='hidden').write
'y'
Source code in src/skcomplex/systems/computer_science/neural_networks/_elman.py 
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
def __init__(
    self,
    label: str,
    *,
    target: Annotated[str, RoleSpec()],
    source: Annotated[str, RoleSpec()],
    W: Annotated[object, ParameterSpec(shape=('target', 'source'), adaptive=True)] = 0.0,
) -> None:
    super().__init__(
        label,
        target=target,
        source=source,
        read='s',
        write='y',
        function=self._activity,
        mode='group',
        weights='W',
        form='map',
        phase='readout',
        combine=self._activate,
    )
    self.W = W

RecurrenceCouplingInteraction(label, *, target, A=0.0)

Bases: CouplingInteraction

Recurrence of a simple recurrent network, s_i <- sigmoid(sum_j A_ij s_j + u_i + b_i) with the logistic sigmoid.

Every hidden unit reads the activities of every hidden unit, itself included, through the adaptive matrix A, adds its drive u and its bias b, and takes the sigmoid of the sum as its next activity (Gros 2015 Eq. 10.18).

Parameters:

Name Type Description Default
label str

The label of the interaction, unique among the parts of its system.

 required
target Annotated[str, RoleSpec()]

The path of the group of hidden units.

 required
A Annotated[object, ParameterSpec(shape=(target, target), adaptive=True)]

The recurrent weights, a square matrix with one row and one column per hidden unit, or a number for every entry.

 0.0

Examples:

>>> from skcomplex.systems.computer_science.neural_networks import RecurrenceCouplingInteraction
>>> RecurrenceCouplingInteraction('recurrence', target='hidden').include_self
True
Source code in src/skcomplex/systems/computer_science/neural_networks/_elman.py 
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
def __init__(
    self,
    label: str,
    *,
    target: Annotated[str, RoleSpec()],
    A: Annotated[object, ParameterSpec(shape=('target', 'target'), adaptive=True)] = 0.0,
) -> None:
    super().__init__(
        label,
        target=target,
        read='s',
        write='s',
        function=self._activity,
        mode='group',
        weights='A',
        include_self=True,
        form='map',
        phase='hidden',
        combine=self._activate,
    )
    self.A = A

ReservoirUnitElement(label, visual=None, *, r=0.0, w_in=1.0, leak=0.3, amplitude=1.0, frequency=0.2)

Bases: Element

Leaky tanh unit of an echo state network, with its activity and the weight, leak, and sine of its input.

The map of echo_state_network sets r(t + 1) = (1 - a) r + a tanh(sum_j W_ij r_j + w u(t)) with the input u(t) = A sin(f t).

Parameters:

Name Type Description Default
label str

The label of the unit or the group.

 required
visual VisualSpec | None

How the element is drawn, or None for default_visual_.

 None
r Annotated[float, StateSpec()]

The initial activity.

 0.0
w_in Annotated[float, ParameterSpec()]

The weight w of the input.

 1.0
leak Annotated[float, ParameterSpec(domain=open(0, oo))]

The leak rate a, 1 for units without memory of their own.

 0.3
amplitude Annotated[float, ParameterSpec()]

The amplitude A of the input.

 1.0
frequency Annotated[float, ParameterSpec()]

The angular frequency f of the input, per step.

 0.2

Examples:

>>> from skcomplex.systems.computer_science.neural_networks import ReservoirUnitElement
>>> ReservoirUnitElement('unit').leak
0.3
Source code in src/skcomplex/systems/computer_science/neural_networks/_echo_state.py 
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
def __init__(
    self,
    label: str,
    visual: VisualSpec | None = None,
    *,
    r: Annotated[float, StateSpec()] = 0.0,
    w_in: Annotated[float, ParameterSpec()] = 1.0,
    leak: Annotated[float, ParameterSpec(domain=RealSet.open(0, oo))] = 0.3,
    amplitude: Annotated[float, ParameterSpec()] = 1.0,
    frequency: Annotated[float, ParameterSpec()] = 0.2,
) -> None:
    super().__init__(label, visual=visual)
    self.r = r
    self.w_in = w_in
    self.leak = leak
    self.amplitude = amplitude
    self.frequency = frequency

diluted_hopfield(n_neurons=4000, *, connectivity=50.0, load=0.2, temperature_scale=3.0, overlap=0.3, random_state=0)

Return the asymmetric diluted Hopfield network, which recalls a stored pattern below its memory capacity.

The network stores p = load K random patterns in the couplings J_ij = C_ij Σ_μ ξ_i ξ_j, where every C_ij is 1 with probability K / N independently of C_ji, so the couplings are sparse and asymmetric. All neurons update together at the inverse temperature β = temperature_scale / K. Started with an overlap m with the first pattern, the overlap follows m(t + 1) = f(m(t)) for large K, which keeps a nonzero fixed point below the critical load 2 / π.

Parameters:

Name Type Description Default
n_neurons int

The number of neurons N.

 4000
connectivity float

The mean number K of inputs of a neuron.

 50.0
load float

The ratio p / K of the number of patterns to the connectivity.

 0.2
temperature_scale float

The product β K of the inverse temperature and the connectivity.

 3.0
overlap float

The initial overlap with the first pattern.

 0.3
random_state int | Generator | None

The seed or generator of the patterns, the dilution, and the initial values.

 0

Returns:

Name Type Description
system System

The system, with the values neurons.spin and the observable overlap.

Examples:

>>> from skcomplex.systems.computer_science.neural_networks import diluted_hopfield
>>> diluted_hopfield(200, connectivity=10).elements[0].prepare().size_
200
Source code in src/skcomplex/systems/computer_science/neural_networks/_diluted.py 
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
@register_system(
    field='computer_science/neural_networks',
    tags=('neural network', 'associative memory', 'spin glass', 'discrete map'),
    references=('Derrida, Gardner, and Zippelius 1987', 'Bertin 2021'),
    defaults={'until': 20, 'dt': 1},
    observables=lambda system: [ObservableSpec('overlap', _overlap, group='neurons')],
)
def diluted_hopfield(
    n_neurons: int = 4000,
    *,
    connectivity: float = 50.0,
    load: float = 0.2,
    temperature_scale: float = 3.0,
    overlap: float = 0.3,
    random_state: int | np.random.Generator | None = 0,
) -> System:
    """Return the asymmetric diluted Hopfield network, which recalls a stored pattern below its memory capacity.

    The network stores `p = load K` random patterns in the couplings `J_ij = C_ij Σ_μ ξ_i ξ_j`, where every `C_ij` is
    1 with probability `K / N` independently of `C_ji`, so the couplings are sparse and asymmetric. All neurons update
    together at the inverse temperature `β = temperature_scale / K`. Started with an overlap `m` with the first pattern,
    the overlap follows `m(t + 1) = f(m(t))` for large `K`, which keeps a nonzero fixed point below the critical load
    `2 / π`.

    Args:
        n_neurons:
            The number of neurons `N`.

        connectivity:
            The mean number `K` of inputs of a neuron.

        load:
            The ratio `p / K` of the number of patterns to the connectivity.

        temperature_scale:
            The product `β K` of the inverse temperature and the connectivity.

        overlap:
            The initial overlap with the first pattern.

        random_state:
            The seed or generator of the patterns, the dilution, and the initial values.

    Returns:
        system:
            The system, with the values `neurons.spin` and the observable `overlap`.

    Examples:
        >>> from skcomplex.systems.computer_science.neural_networks import diluted_hopfield
        >>> diluted_hopfield(200, connectivity=10).elements[0].prepare().size_
        200
    """
    rng = np.random.default_rng(random_state)
    n_patterns = max(1, round(load * connectivity))
    patterns = rng.choice([-1.0, 1.0], size=(n_patterns, n_neurons))
    spins = np.where(rng.random(n_neurons) < (1 + overlap) / 2, patterns[0], -patterns[0])
    neurons = GroupElement(
        'neurons',
        element=DilutedNeuronElement,
        size=n_neurons,
        values={
            'spin': spins,
            'pattern': patterns[0],
            'beta': temperature_scale / connectivity,
        },
    )
    glauber = ChoiceInteraction(
        'glauber',
        target='neurons',
        state='spin',
        options=(-1.0, 1.0),
        utility=functools.partial(_alignment, couplings=_couplings(patterns, connectivity, rng)),
    )
    return System(elements=[neurons], interactions=[glauber])

echo_state_network(n_units=200, *, spectral_radius=0.9, leak=0.3, connectivity=0.1, input_scaling=1.0, amplitude=1.0, frequency=0.2, random_state=0)

Return an echo state network, a reservoir of leaky tanh units with random recurrent weights driven by a sine.

The recurrent weights are sparse and Gaussian, scaled to a spectral radius below 1, so the reservoir has the echo state property: its activities forget their initial values and become a function of the input history alone. A readout trained on these activities, as skcomplex.core.analysis.ReservoirAnalysis does, turns the reservoir into a forecaster.

Parameters:

Name Type Description Default
n_units int

The number of units.

 200
spectral_radius float

The largest modulus of the eigenvalues of the recurrent weights.

 0.9
leak float

The leak rate of every unit, 1 for units without memory of their own.

 0.3
connectivity float

The share of the recurrent weights that are not zero.

 0.1
input_scaling float

The scale of the input weights, drawn uniformly from [-input_scaling, input_scaling].

 1.0
amplitude float

The amplitude of the sine input.

 1.0
frequency float

The angular frequency of the sine input, per step.

 0.2
random_state int | Generator | None

The seed or generator of the weights and of the initial activities.

 0

Returns:

Name Type Description
system System

The system, with the group units of activities r, one per unit.

Examples:

>>> from skcomplex.systems.computer_science.neural_networks import echo_state_network
>>> echo_state_network(50).elements[0].prepare().size_
50
Source code in src/skcomplex/systems/computer_science/neural_networks/_echo_state.py 
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
@register_system(
    field='computer_science/neural_networks',
    tags=('neural network', 'reservoir computing', 'echo state', 'discrete map'),
    references=('Jaeger 2001', 'Cestnik and Martens 2025'),
    defaults={'until': 200, 'dt': 1},
)
def echo_state_network(
    n_units: int = 200,
    *,
    spectral_radius: float = 0.9,
    leak: float = 0.3,
    connectivity: float = 0.1,
    input_scaling: float = 1.0,
    amplitude: float = 1.0,
    frequency: float = 0.2,
    random_state: int | np.random.Generator | None = 0,
) -> System:
    """Return an echo state network, a reservoir of leaky tanh units with random recurrent weights driven by a sine.

    The recurrent weights are sparse and Gaussian, scaled to a spectral radius below 1, so the reservoir has the echo
    state property: its activities forget their initial values and become a function of the input history alone. A
    readout trained on these activities, as `skcomplex.core.analysis.ReservoirAnalysis` does, turns the reservoir
    into a forecaster.

    Args:
        n_units:
            The number of units.

        spectral_radius:
            The largest modulus of the eigenvalues of the recurrent weights.

        leak:
            The leak rate of every unit, 1 for units without memory of their own.

        connectivity:
            The share of the recurrent weights that are not zero.

        input_scaling:
            The scale of the input weights, drawn uniformly from `[-input_scaling, input_scaling]`.

        amplitude:
            The amplitude of the sine input.

        frequency:
            The angular frequency of the sine input, per step.

        random_state:
            The seed or generator of the weights and of the initial activities.

    Returns:
        system:
            The system, with the group `units` of activities `r`, one per unit.

    Examples:
        >>> from skcomplex.systems.computer_science.neural_networks import echo_state_network
        >>> echo_state_network(50).elements[0].prepare().size_
        50
    """
    rng = np.random.default_rng(random_state)
    weights = _recurrent_weights(n_units, connectivity, spectral_radius, rng)
    units = GroupElement(
        'units',
        element=ReservoirUnitElement,
        size=n_units,
        values={
            'r': rng.uniform(-1.0, 1.0, n_units),
            'w_in': rng.uniform(-input_scaling, input_scaling, n_units),
            'leak': leak,
            'amplitude': amplitude,
            'frequency': frequency,
        },
    )
    reservoir = MapInteraction(
        'reservoir',
        target='units',
        state='r',
        function=functools.partial(_leaky_tanh, weights=weights),
    )
    return System(elements=[units], interactions=[reservoir])

elman_network(n_hidden=4, *, n_bits=3000, window=12, batch_size=16, learning_rate=0.05, truncation=None, random_state=0)

Return Elman's simple recurrent network, which learns to predict the next bit of a temporal XOR stream.

The nested discrete map net holds one input unit, n_hidden hidden units, and one output unit. The input sets the drive u = D x of the hidden units, the hidden units take s <- sigmoid(A s + u + b) (Gros 2015 Eq. 10.18), and the output predicts y <- sigmoid(W s + c), in three phases of every step, so the prediction of a step reads its input. The training interaction training trains D, A, W, b, and c with Adam on the squared error of predicting the next input from the inputs so far, over windows of window bits that start at every third bit of a stream in which every third bit is the exclusive or of the two before it. Only those bits can be predicted, so the metric xor_error falls toward 0 while random_error, the error on the other bits, stays near 0.25. One step of the system is one epoch.

Parameters:

Name Type Description Default
n_hidden int

The number of hidden units.

 4
n_bits int

The length of the stream.

 3000
window int

The number of bits of a sequence.

 12
batch_size int

The number of sequences of a batch.

 16
learning_rate float

The learning rate of Adam.

 0.05
truncation int | None

The number of steps the gradient flows back through the hidden activities, 1 for Elman's original rule that holds the previous activities as constant context, or None for full backpropagation through time.

 None
random_state int | Generator | None

The seed or generator of the stream and of the initial weights.

 0

Returns:

Name Type Description
system System

The system, with the nested system net and the training interaction training.

Examples:

>>> from skcomplex.systems.computer_science.neural_networks import elman_network
>>> [item.label for item in elman_network().systems]
['net']
Source code in src/skcomplex/systems/computer_science/neural_networks/_elman.py 
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
@register_system(
    field='computer_science/neural_networks',
    tags=('neural network', 'recurrent network', 'learning', 'backpropagation through time', 'discrete map'),
    references=('Gros 2015 §10.5.1', 'Elman 1990'),
    defaults={'until': 200, 'dt': 1},
)
def elman_network(
    n_hidden: int = 4,
    *,
    n_bits: int = 3000,
    window: int = 12,
    batch_size: int = 16,
    learning_rate: float = 0.05,
    truncation: int | None = None,
    random_state: int | np.random.Generator | None = 0,
) -> System:
    """Return Elman's simple recurrent network, which learns to predict the next bit of a temporal XOR stream.

    The nested discrete map `net` holds one input unit, `n_hidden` hidden units, and one output unit. The input sets
    the drive `u = D x` of the hidden units, the hidden units take `s <- sigmoid(A s + u + b)` (Gros 2015 Eq. 10.18),
    and the output predicts `y <- sigmoid(W s + c)`, in three phases of every step, so the prediction of a step reads
    its input. The training interaction `training` trains `D`, `A`, `W`, `b`, and `c` with Adam on the squared error
    of predicting the next input from the inputs so far, over windows of `window` bits that start at every third bit of
    a stream in which every third bit is the exclusive or of the two before it. Only those bits can be predicted, so
    the metric `xor_error` falls toward 0 while `random_error`, the error on the other bits, stays near 0.25. One step
    of the system is one epoch.

    Args:
        n_hidden:
            The number of hidden units.

        n_bits:
            The length of the stream.

        window:
            The number of bits of a sequence.

        batch_size:
            The number of sequences of a batch.

        learning_rate:
            The learning rate of Adam.

        truncation:
            The number of steps the gradient flows back through the hidden activities, 1 for Elman's original rule
            that holds the previous activities as constant context, or `None` for full backpropagation through time.

        random_state:
            The seed or generator of the stream and of the initial weights.

    Returns:
        system:
            The system, with the nested system `net` and the training interaction `training`.

    Examples:
        >>> from skcomplex.systems.computer_science.neural_networks import elman_network
        >>> [item.label for item in elman_network().systems]
        ['net']
    """
    rng = np.random.default_rng(random_state)
    data = _dataset(_temporal_xor(n_bits, rng), window)
    net = System(
        'net',
        elements=[
            GroupElement('inputs', element=InputUnitElement, size=1),
            GroupElement('hidden', element=HiddenUnitElement, size=n_hidden),
            GroupElement('outputs', element=OutputUnitElement, size=1),
        ],
        interactions=[
            InputCouplingInteraction(
                'input',
                target='hidden',
                source='inputs',
                D=rng.normal(0.0, WEIGHT_SCALE, (n_hidden, 1)),
            ),
            RecurrenceCouplingInteraction(
                'recurrence',
                target='hidden',
                A=rng.normal(0.0, WEIGHT_SCALE, (n_hidden, n_hidden)),
            ),
            ReadoutCouplingInteraction(
                'readout',
                target='outputs',
                source='hidden',
                W=rng.normal(0.0, WEIGHT_SCALE, (1, n_hidden)),
            ),
        ],
        time=DiscreteTime(schedule=PhasesScheduleSpec(PHASES)),
    )
    training = TrainingInteraction(
        'training',
        system='net',
        trained=TRAINED,
        loss=ObservableSpec('error', group='outputs', reduce='sum', expression=_squared_error),
        data=data,
        learning_rate=learning_rate,
        batch_size=batch_size,
        truncation=truncation,
        metrics=(
            ObservableSpec('xor_error', group='outputs', reduce='sum', expression=_squared_error),
            ObservableSpec('random_error', group='outputs', reduce='sum', expression=_squared_error),
        ),
    )
    return System('elman', systems=[net], interactions=[training], time=DiscreteTime())

hopfield(patterns=None, *, n_neurons=64, n_patterns=3, noise=0.2, random_state=0)

Return a Hopfield network that stores patterns with Hebb's rule and recalls the first from a corrupted copy.

Neurons update one at a time in random order, so the energy -Σ w_ij s_i s_j / 2 never increases and the network settles in a stored pattern when few patterns are stored, about fewer than 0.14 n for n neurons.

Parameters:

Name Type Description Default
patterns ArrayLike | None

The stored patterns of -1 and 1, one row per pattern, or None for random patterns.

 None
n_neurons int

The number of neurons of random patterns.

 64
n_patterns int

The number of random patterns.

 3
noise float

The probability that a neuron starts flipped from the first pattern.

 0.2
random_state int | Generator | None

The seed or generator of the random patterns and of the corruption.

 0

Returns:

Name Type Description
system System

The system, with the group neurons of states spin and weights on a complete graph.

Examples:

>>> from skcomplex.systems.computer_science.neural_networks import hopfield
>>> hopfield(n_neurons=16, n_patterns=2).elements[0].prepare().size_
16
Source code in src/skcomplex/systems/computer_science/neural_networks/_hopfield.py 
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
@register_system(
    field='computer_science/neural_networks',
    tags=('neural network', 'associative memory', 'agents', 'attractor'),
    references=('Hopfield 1982', 'Trzaska 2023'),
    defaults={'until': 10},
)
def hopfield(
    patterns: npt.ArrayLike | None = None,
    *,
    n_neurons: int = 64,
    n_patterns: int = 3,
    noise: float = 0.2,
    random_state: int | np.random.Generator | None = 0,
) -> System:
    """Return a Hopfield network that stores patterns with Hebb's rule and recalls the first from a corrupted copy.

    Neurons update one at a time in random order, so the energy `-Σ w_ij s_i s_j / 2` never increases and the network
    settles in a stored pattern when few patterns are stored, about fewer than `0.14 n` for `n` neurons.

    Args:
        patterns:
            The stored patterns of -1 and 1, one row per pattern, or `None` for random patterns.

        n_neurons:
            The number of neurons of random patterns.

        n_patterns:
            The number of random patterns.

        noise:
            The probability that a neuron starts flipped from the first pattern.

        random_state:
            The seed or generator of the random patterns and of the corruption.

    Returns:
        system:
            The system, with the group `neurons` of states `spin` and `weights` on a complete graph.

    Examples:
        >>> from skcomplex.systems.computer_science.neural_networks import hopfield
        >>> hopfield(n_neurons=16, n_patterns=2).elements[0].prepare().size_
        16
    """
    rng = np.random.default_rng(random_state)
    stored: Array = np.asarray(
        rng.choice([-1.0, 1.0], size=(n_patterns, n_neurons)) if patterns is None else patterns,
        dtype=np.float64,
    )
    size = stored.shape[1]
    spins = np.where(rng.random(size) < noise, -stored[0], stored[0])
    neurons = GroupElement(
        'neurons',
        element=NeuronElement,
        size=size,
        values={'spin': spins, 'weights': _hebbian_weights(stored)},
    )
    return System(
        elements=[neurons],
        interactions=[HopfieldUpdateInteraction('update', target='neurons')],
        space=NetworkSpace(nx.complete_graph(size)),
    )

next_generation_reservoir(series=None, *, analysis=None, variables=LORENZ_VARIABLES, delay=2, order=2, ridge=2.5e-06, standardize=False, samples=403, step=0.025)

Return a next-generation reservoir computer whose variables follow the ridge readout of their monomials.

Every variable keeps delay - 1 lags, and one map in the increment mode adds to it the readout of the constant, the variables with their lags, and the monomials of degree order of those. The readout comes from a fitted ReservoirAnalysis(kind='next_generation'), and the variables start at the end of its last series, so a run is the autonomous forecast from there. One step of the system stands for one sample of the series. The defaults are those of Gauthier et al. (2021) for the Lorenz flow: 400 training samples after a warmup of 2, one sample every 0.025 from an explicit Runge-Kutta 3(2) integrator, and the ridge 2.5e-6 on features that are not standardized. A ridge this small on these features diverges within one Lyapunov time when a series comes from a much more accurate integrator, and such a series needs standardize=True.

Parameters:

Name Type Description Default
series Array | None

The training series, one row per sample and one column per variable, or None for samples samples of the Lorenz flow every step.

 None
analysis ReservoirAnalysis | None

A fitted next-generation reservoir analysis to take the readout from, which replaces the fit.

 None
variables Sequence[str]

The names of the columns of the series.

 LORENZ_VARIABLES
delay int

The number of consecutive samples the features read.

 2
order int

The degree of the monomials.

 2
ridge float

The strength of the ridge penalty.

 2.5e-06
standardize bool

Whether the features read the standardized variables.

 False
samples int

The number of samples of the Lorenz flow.

 403
step float

The time between the samples of the Lorenz flow.

 0.025

Returns:

Name Type Description
system System

The system, with the variables reservoir.x and their lags reservoir.x_lag1 for the Lorenz flow.

Raises:

Type Description
ValidationError

The analysis is not a next-generation reservoir.

Examples:

>>> import numpy as np
>>> from skcomplex.systems.computer_science.neural_networks import next_generation_reservoir
>>> values = np.sin(0.1 * np.arange(200.0))[:, None]
>>> system = next_generation_reservoir(values, variables=['v'])
>>> [interaction.mode for interaction in system.interactions]
['increment']
Source code in src/skcomplex/systems/computer_science/neural_networks/_next_generation.py 
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
@register_system(
    field='computer_science/neural_networks',
    tags=('neural network', 'reservoir computing', 'next generation', 'lags', 'discrete map'),
    references=('Gauthier et al. 2021', 'Cestnik and Martens 2025'),
    defaults={'until': 200, 'dt': 1},
)
def next_generation_reservoir(
    series: Array | None = None,
    *,
    analysis: ReservoirAnalysis | None = None,
    variables: Sequence[str] = LORENZ_VARIABLES,
    delay: int = 2,
    order: int = 2,
    ridge: float = 2.5e-6,
    standardize: bool = False,
    samples: int = 403,
    step: float = 0.025,
) -> System:
    """Return a next-generation reservoir computer whose variables follow the ridge readout of their monomials.

    Every variable keeps `delay - 1` lags, and one map in the increment mode adds to it the readout of the constant, the
    variables with their lags, and the monomials of degree `order` of those. The readout comes from a fitted
    `ReservoirAnalysis(kind='next_generation')`, and the variables start at the end of its last series, so a run is the
    autonomous forecast from there. One step of the system stands for one sample of the series. The defaults are those
    of Gauthier et al. (2021) for the Lorenz flow: 400 training samples after a warmup of 2, one sample every 0.025 from
    an explicit Runge-Kutta 3(2) integrator, and the ridge 2.5e-6 on features that are not standardized. A ridge this
    small on these features diverges within one Lyapunov time when a series comes from a much more accurate integrator,
    and such a series needs `standardize=True`.

    Args:
        series:
            The training series, one row per sample and one column per variable, or `None` for `samples` samples of the
            Lorenz flow every `step`.

        analysis:
            A fitted next-generation reservoir analysis to take the readout from, which replaces the fit.

        variables:
            The names of the columns of the series.

        delay:
            The number of consecutive samples the features read.

        order:
            The degree of the monomials.

        ridge:
            The strength of the ridge penalty.

        standardize:
            Whether the features read the standardized variables.

        samples:
            The number of samples of the Lorenz flow.

        step:
            The time between the samples of the Lorenz flow.

    Returns:
        system:
            The system, with the variables `reservoir.x` and their lags `reservoir.x_lag1` for the Lorenz flow.

    Raises:
        ValidationError:
            The analysis is not a next-generation reservoir.

    Examples:
        >>> import numpy as np
        >>> from skcomplex.systems.computer_science.neural_networks import next_generation_reservoir
        >>> values = np.sin(0.1 * np.arange(200.0))[:, None]
        >>> system = next_generation_reservoir(values, variables=['v'])
        >>> [interaction.mode for interaction in system.interactions]
        ['increment']
    """
    fitted = analysis or _fit_reservoir(
        series,
        variables,
        delay=delay,
        order=order,
        ridge=ridge,
        standardize=standardize,
        samples=samples,
        step=step,
    )
    if fitted.kind != REQUIRED_KIND:
        error_msg = f'The readout comes from a {REQUIRED_KIND!r} reservoir analysis, not from a {fitted.kind!r} one.'
        raise ValidationError(error_msg)
    names = fitted.states_[: len(fitted.labels_)]
    reservoir = fitted.step_
    memory = fitted.samples_.series[-1][-fitted.delay :][::-1]
    initial = {name: memory[:, index].tolist() for index, name in enumerate(names)}
    element = _reservoir_element(initial)
    weights = np.asarray(reservoir.readout.Wout, dtype=float)
    bias = np.ravel(np.asarray(reservoir.readout.bias, dtype=float))
    maps = [
        MapInteraction(
            name,
            target='reservoir',
            state=name,
            mode='increment',
            function=functools.partial(
                _increment,
                names=names,
                delay=fitted.delay,
                powers=reservoir.monomials(),
                weights=weights[:, index],
                bias=float(bias[index]),
                mean=reservoir.mean,
                scale=reservoir.scale,
                spread=float(reservoir.scale[index]),
            ),
        )
        for index, name in enumerate(names)
    ]
    return System(elements=[element], interactions=maps, time=DiscreteTime())

perceptron_satisfiability(n_weights=21, *, load=1.5, random_state=0)

Return a binary perceptron that must store random patterns, revealed one per step, as cardinality constraints.

A pattern xi of output sigma holds when sigma sum_i w_i xi_i > 0, so with an odd number N of weights w_i = +-1, when at least (N + 1) / 2 of the literals w_i = sigma xi_i hold. The patterns are satisfiable up to the critical load p / N of about 0.83 in the limit of many weights (Bertin 2021 §6.3.3, Ex. 6.4). The patterns are drawn from random_state, so a disorder average sweeps it through GridDesign.

Parameters:

Name Type Description Default
n_weights int

The number of weights, which is odd.

 21
load float

The number of patterns over the number of weights.

 1.5
random_state int | Generator | None

The seed or generator of the patterns.

 0

Returns:

Name Type Description
system System

The system, with the group weights and the reports decide.satisfiable and decide.revealed.

Raises:

Type Description
ValidationError

The number of weights is even, so a pattern can tie.

Examples:

>>> from skcomplex.systems.computer_science.neural_networks import perceptron_satisfiability
>>> perceptron_satisfiability(3, load=1.0).interactions[0].constraints[0].bound
2
Source code in src/skcomplex/systems/computer_science/neural_networks/_perceptron.py 
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
@register_system(
    field='computer_science/neural_networks',
    tags=('perceptron', 'satisfiability', 'disorder', 'critical load'),
    references=('Bertin 2021', 'Krauth and Mezard 1989'),
    defaults={'until': 31},
)
def perceptron_satisfiability(
    n_weights: int = 21,
    *,
    load: float = 1.5,
    random_state: int | np.random.Generator | None = 0,
) -> System:
    """Return a binary perceptron that must store random patterns, revealed one per step, as cardinality constraints.

    A pattern `xi` of output `sigma` holds when `sigma sum_i w_i xi_i > 0`, so with an odd number `N` of weights
    `w_i = +-1`, when at least `(N + 1) / 2` of the literals `w_i = sigma xi_i` hold. The patterns are satisfiable up to
    the critical load `p / N` of about 0.83 in the limit of many weights (Bertin 2021 §6.3.3, Ex. 6.4). The patterns are
    drawn from `random_state`, so a disorder average sweeps it through `GridDesign`.

    Args:
        n_weights:
            The number of weights, which is odd.

        load:
            The number of patterns over the number of weights.

        random_state:
            The seed or generator of the patterns.

    Returns:
        system:
            The system, with the group `weights` and the reports `decide.satisfiable` and `decide.revealed`.

    Raises:
        ValidationError:
            The number of weights is even, so a pattern can tie.

    Examples:
        >>> from skcomplex.systems.computer_science.neural_networks import perceptron_satisfiability
        >>> perceptron_satisfiability(3, load=1.0).interactions[0].constraints[0].bound
        2
    """
    if n_weights % 2 == 0:
        error_msg = f'perceptron_satisfiability has n_weights={n_weights!r}. Use an odd number, so no pattern ties.'
        raise ValidationError(error_msg)
    rng = np.random.default_rng(random_state)
    n_patterns = round(load * n_weights)
    patterns = rng.choice([-1, 1], size=(n_patterns, n_weights)) * rng.choice([-1, 1], size=(n_patterns, 1))
    indices = np.arange(1, n_weights + 1)
    constraints = [
        CardinalitySpec(tuple((indices * pattern).tolist()), (n_weights + 1) // 2, 'at-least') for pattern in patterns
    ]
    weights = GroupElement('weights', element=BinaryWeightElement, size=n_weights)
    decide = SatisfiabilityInteraction('decide', target='weights', state='on', constraints=constraints)
    return System(elements=[weights], interactions=[decide], time=DiscreteTime())