Debug learning rate so large weights diverge in the single-neuron trainer

Do not start with a speculative fix for the single-neuron trainer. Force the failure on purpose, save the before output, then change one cause at a time. Retries are allowed only for transient conditions—not for bad input that will fail forever on neural-nets-by-building.

import math
def sigmoid(z): return 1/(1+math.exp(-max(-60,min(60,z))))
for z in [-1000,0,1000]: print(z,sigmoid(z))

Expected evidence: finite 0.0, 0.5, and 1.0 values. If you cannot reproduce on demand, you do not yet control the failure mode for neural-nets-by-building.

After repair, rerun the exact reproduction command. Keep the failing fixture as a regression seed for the observability page. For the single-neuron logic-gate trainer, remember the claim you are restoring: after training, AND (or OR) inputs map to the expected 0/1 outputs.

Store a one-command reproduction for: learning rate so large weights diverge, or claiming XOR success with one neuron. The command should use AND gate truth table or a minimal mutant of it. Paste the failing output into notes/failure-before.txt (or your shell scrollback as copied text). After the fix, paste notes/failure-after.txt and keep both.

Retries belong only on transient faults. If the failure is bad input, a bad allowlist, or a logic bug in the single-neuron trainer, retrying will amplify cost without repairing trust around learn weights for a linearly separable gate by gradient steps you can print.

Classify the failure as prevent, detect, contain, or recover—using this lab’s language, not a generic poster. For neural-nets-by-building, the first fix should usually be detect+prevent at the boundary, because learning rate so large weights diverge, or claiming XOR success with one neuron is cheaper to stop early than to explain in production prose.

At the debugging stage for neural-nets-by-building, the job is narrower than finishing a product demo. You are creating one progressive evidence piece about AND gate truth table that later pages inherit without redefining success. Keep that fixture small enough to inspect by hand, keep outputs copy-pasteable as text, and refuse to narrate this baseline as if it were a production SLA: untrained weight predictions on the gate table.

For this page specifically, success looks like before/after evidence for the characteristic failure while still centering the user decision to learn weights for a linearly separable gate by gradient steps you can print. If you cannot point to a file, command, or assertion that proves that for the single-neuron trainer, stay on this page instead of advancing.

Glossary: deep learning · Glossary: gradient descent

Previous · Next