A Codebook-and-Language-Model Framework for Predicting Seizure Onset from EEG Signals ^†

Abstract

Roughly 50 million people worldwide live with epilepsy, and about a third of them do not respond to anti-epileptic drugs, which makes predicting a seizure before it begins clinically meaningful. We propose a new end-to-end framework based on large language models (LLMs) for predicting seizure onset using electroencephalogram (EEG) data. The framework rests on two contributions: (i) a vocabulary that transcribes multichannel EEG signals into a discrete codebook that is interpretable by LLM-style decoders, and (ii) the exploration of two LLM variants, GPT-2 and Llama-3, adapted to this vocabulary through transfer learning by fine-tuning, with the goal of predicting seizure onset with the lowest possible error. The vocabulary is obtained by fitting a k-means codebook on one-second multichannel EEG patches from the training cohort; the LLM variants are then trained on the resulting token stream with a causal pre-training pass followed by a fine-tuning pass that flags preictal windows. We evaluate the framework on the CHB-MIT scalp EEG dataset under a strict person-disjoint cross-patient split (16 training, 3 validation and 5 test cases, with 22 held-out seizure events) and, because the interictal class dominates the windows, report prediction-focused metrics rather than accuracy: pre-onset sensitivity (Sens_p), false alarms per hour (FAR/h), lead time, and sensitivity at fixed FAR. The configuration that maximises sensitivity (codebook $K=256$, GPT-style LLM, focal loss plus weighted sampler) predicts 16 of 22 test seizures ahead of onset (Sens_p $= 72.7\%$) with a median lead time of 280 s, at the price of 4.17 false alarms per hour. For the implemented 600 s prediction horizon, a rate-matched homogeneous Poisson process at 4.165 alarms/h yields a 50.1% chance-alarm probability, compared with the observed 72.7% event sensitivity. A persistence-smoothing operating point pulls FAR below 0.5/h but trades away most of the sensitivity. These results demonstrate an end-to-end route from discrete EEG motifs to pre-onset alarms under a strict person-disjoint protocol. The resulting sensitivity–FAR frontier provides a transparent cross-patient baseline and identifies representation learning and alarm-policy calibration as complementary directions for improving the framework.

1. Introduction

Epilepsy is a chronic neurological disorder in which abnormal hypersynchronous cortical activity produces recurrent seizures. The World Health Organization estimates that more than 50 million people live with epilepsy and that about one third do not respond adequately to medications [1]. For refractory patients, a warning issued before onset could support protective actions, assistive notification, or closed-loop stimulation. The target is therefore not merely the detection of seizures after onset but the prediction of seizures from preictal EEG with positive lead time.

Many CHB-MIT seizure-prediction studies use hand-crafted spectral or nonlinear features, CNN, recurrent networks, or hybrid CNN–LSTM models [2,3,4]. These systems can report high performance within the patient, but recent methodological audits show that overlapping windows, non-person-disjoint partitions, and loose prediction/detection definitions can inflate apparent performance [5,6]. We therefore frame the study as held-out cross-patient prediction and credit only alarms that occur strictly before the annotated onset.

Large language models are attractive for EEG because decoder-only transformers learn long-range dependencies from token streams [7,8,9]. The obstacle is that EEG is continuous, multichannel, and high-rate, whereas language models expect discrete vocabularies. Foundation EEG models such as LaBraM and NeuroLM address this through large-scale pre-training, but at substantial data and compute cost [10,11]. This paper explores the opposite end of the spectrum: a compact, training-only k-means codebook that turns raw EEG into one-second motifs, followed by small GPT-2 and Llama-3-style decoders fine-tuned for the prediction of the onset of seizures.

Contributions are twofold. First, we define an EEG vocabulary in which each token is the nearest centroid of a one-second, 23-channel patch fitted from training subjects only. Second, we compare EEGGPT and EEGLlama under the same causal pre-training and supervised fine-tuning protocol. The evaluation reports codebook quality, window-level discrimination, event-level prediction sensitivity, false alarms per hour, and lead time.

2. Materials and Methods

2.1. Dataset and Person-Disjoint Split

We used the CHB-MIT scalp EEG database [2], sampled at 256 Hz with 23 bipolar channels and 198 annotated seizure events. Because chb01 and chb21 correspond to the same person, both cases were assigned to the same data partition to prevent identity leakage. The final split is person-disjoint: 16 training cases, 3 validation cases, and 5 test cases representing 4 distinct persons. The closed test fold contains 22 seizure events and is used only once for the final event-level evaluation. Table 1 reports the case, person, and event counts assigned to each partition.

Table 1. Person-disjoint CHB-MIT split used in this work.

Split	Cases	Persons	Events
Train	16	16	121
Validation	3 (chb03, chb05, chb14)	3	20
Test	5 (chb01+chb21, chb04, chb09, chb22)	4	22

2.2. EEG Preprocessing and Positive-Window Definition

EDF recordings are resampled at 256 Hz, restricted to 23 canonical bipolar channels, band-pass filtered from 0.5 to 70 Hz, notched at 60 Hz, and standardized per channel. Missing channels are filled with zero and recorded. Windows with maximum absolute amplitude above 3000 $\mathsf{\mu }$V are discarded as saturation artifacts while preserving high-amplitude peri-ictal activity.

Each positive sample is a 60-second window positioned around the onset of the seizure. The parameter $\rho$ controls how many seconds of that window are preictal. Four settings are evaluated: opt-A ($\rho =30$), opt-B ($\rho =50$), opt-C ($\rho =55$), and opt-D ($\rho =60$). opt-D is the strict prediction case because the window contains no ictal samples. Table 2 summarizes the preictal and ictal composition of the four configurations. For each seizure and $\rho$, eight positive windows are generated with small random changes. Negative windows are sampled at least 1200 s away from any seizure boundary, avoiding a shortcut in which negatives come only from seizure-free files.

Table 2. Positive-window configurations. $\rho$ is the number of preictal seconds inside each 60 s positive window.

Config	$\mathit{\rho }$ (s)	Preictal	Ictal	Framing
opt-A	30	30 s	30 s	Detection-like
opt-B	50	50 s	10 s	Mostly preictal
opt-C	55	55 s	5 s	Near-prediction
opt-D	60	60 s	0 s	Pure prediction

2.3. Codebook-and-LLM Framework

The proposed framework has two modules (Figure 1). The first module fits a vocabulary of k-means in raw training EEG patches. A 60-second window of shape $(C=23, T=\mathrm{15,360})$ is divided into 60 patches of shape $(23,256)$; each patch is replaced by the index of its nearest centroid in $\mathcal{B}=\{c_1,\dots ,c_K\}$. Each centroid is an EEG motif: a one-second multichannel prototype summarizing a recurrent region of the training patch space. The frozen codebook then tokenizes the validation and test subjects, avoiding tokenizer leakage.

Figure 1. End-to-end codebook-and-language-model framework. Raw multichannel EEG is segmented into one-second patches, quantized into discrete motifs, modeled by a compact decoder-only LLM, and converted into event-level alarms through thresholding, persistence, and cooldown.

The second module consumes the token stream with a compact decoder-only LLM. EEGGPT uses learned positional embeddings, GELU feed-forward blocks, and pre-LayerNorm. EEGLlama follows the LlamaModel recipe with RMSNorm, rotary positional embeddings, and SwiGLU blocks. Both use $L=4$ layers, $h=4$ heads, $d_{\mathrm{model}}=128$, $d_{\mathrm{ffn}}=512$, dropout 0.1, and a maximum length of 64 tokens. Both models are trained first for the prediction of the causal next-token in the EEG vocabulary and then fine-tuned with a $\left[\mathrm{CLS}\right]$-pooled binary classifier. We compare BCE and focal loss with weighted sampling.

2.4. Evaluation

In the codebook stage, we monitor validation reconstruction RMSE, active-token use, and event sensitivity at FAR $= 0.5$/h; $K=256$ is retained as the operating vocabulary because it combines the highest observed mean controlled-FAR sensitivity with a reconstruction error on the plateau of the sweep. In the LLM stage, validation $F_1$, AUC, and MCC provide complementary checkpoint diagnostics. For continuous evaluation, the fine-tuned model is applied with a 4 s stride. An alarm is fired when the probability exceeds the validation-selected threshold $\tau =0.5$ for $K_p$ consecutive windows, and a 10 min cooldown suppresses duplicate alarms. Alarms strictly before onset are counted as predictions, whereas alarms at or after onset are counted separately as detections. FAR/h is computed from alarms in interictal periods located at least 1200 s from a seizure boundary. To contextualize event sensitivity at a given alarm rate, a homogeneous Poisson process with rate r alarms/hour and eligible pre-onset horizon H hours has chance-alarm probability $P_{\mathrm{chance}}=1-\exp (-rH)$. The continuous-evaluation implementation used a maximum pre-onset lead of 600 s, hence $H=600/3600=1/6$ h. At the high-sensitivity operating point, $r=4.165$/h gives $P_{\mathrm{chance}}=1-\exp [-4.165(1/6)] =0.501$. This analytical reference assumes stationary, independent Poisson alarms and is used for descriptive context rather than as an inferential significance test.

3. Results and Discussion

3.1. Contribution (i): EEG Vocabulary

Table 3 summarizes the codebook sweep using reproducible descriptive quantities. RMSE improves only modestly beyond $K=128$, while $K=256$ gives the highest observed mean Sens_p at FAR $= 0.5$/h. We therefore retain $K=256$ as an operational choice rather than as a statistically superior vocabulary. With only three seeds, inferential comparisons are not reported.

Table 3. Codeboo sweep across three seeds. Results are descriptive; no inferential comparison is made for $n=3$.

K	RMSE Val ($\mathsf{\mu }$V)	Sens@0.5/h
32	82.79	0.58 ± 0.07
64	82.33	0.79 ± 0.07
128	82.51	0.73 ± 0.09
256	81.45	0.80 ± 0.07
512	81.26	0.71 ± 0.21
1024	81.08	0.73 ± 0.05

Note: Bold identifies the selected operational vocabulary ($K=256$) and its highest observed mean Sens@0.5/h; the selection is descriptive rather than inferential.

3.2. Contribution (ii): LLM Variants

Table 4 compares EEGGPT and EEGLlama under the same vocabulary and fine-tuning protocol. The window-level AUC values cluster around 0.5, while EEGGPT attains the highest mean $F_1$ within both evaluated objectives. We therefore use these metrics as checkpoint diagnostics and rely on the continuous event-level analysis to characterize how the learned probability streams translate into pre-onset alarms and false-alarm rates.

Table 4. Descriptive validation comparison of LLM variants across 10 training seeds.

Objective	Backbone	Val ${\mathit{F}}_1$	Val AUC	Val MCC
BCE	EEGGPT (K256)	0.041 ± 0.022	0.496 ± 0.057	−0.028 ± 0.025
BCE	EEGLlama (K256)	0.004 ± 0.005	0.478 ± 0.049	−0.020 ± 0.012
BCE	EEGGPT (K512)	0.058 ± 0.028	0.522 ± 0.040	0.002 ± 0.023
BCE	EEGLlama (K512)	0.037 ± 0.041	0.519 ± 0.028	0.030 ± 0.057
focal+samp.	EEGGPT (K256)	0.050 ± 0.025	0.521 ± 0.046	−0.019 ± 0.032
focal+samp.	EEGLlama (K256)	0.008 ± 0.010	0.470 ± 0.070	−0.017 ± 0.014

3.3. Integrated Continuous Prediction

The integrated system is evaluated on continuous held-out EDFs. Table 5 shows that the most sensitive configuration predicts 16 of 22 seizures before onset (Sens_p $= 0.727$) with a median lead time of 280 s at 4.165 FAR/h. This operating point produces, on average, one alarm every 14.4 min and therefore represents the sensitivity-oriented end of the observed frontier. With the implemented 10-min horizon, the rate-matched Poisson probability is 0.501, corresponding to approximately 11 of 22 events expected to contain at least one random alarm, whereas 16 events were predicted by the model. The observed sensitivity is therefore 22.6 percentage points above this analytical chance reference. This descriptive difference does not by itself establish statistical significance and does not make 4.165 FAR/h clinically acceptable. Increasing persistence moves the system toward lower-FAR operation by suppressing isolated probability excursions, although it also reduces the number of pre-onset alarms. The joint reporting of Sens_p, FAR/h, lead time, and chance context captures this clinically relevant operating trade-off more completely than a single window-level metric.

Table 5. Continuous event-level prediction on held-out test EEG.

Model	Obj./${\mathit{K}}_{\mathit{p}}$	Pred.	Det.	Miss.	Sensp	FAR/h
EEGGPT	BCE/1	8	8	6	0.364	2.339
EEGGPT	BCE/5	1	1	20	0.045	0.122
EEGLlama	BCE/3	1	0	21	0.045	0.202
EEGGPT	f+s/1	16	3	3	0.727	4.165
EEGGPT	f+s/5	12	7	3	0.545	3.528
EEGLlama	f+s/5	0	1	21	0.000	0.199

Note: The bold row identifies the highest-sensitivity operating point; the separately bolded FAR value (0.122/h) is the lowest observed false-alarm rate.

Figure 2 visualizes the sensitivity–FAR trade-off of the event-level operating points in Table 5; the inset enlarges the low-FAR region so that closely spaced configurations remain distinguishable.

Figure 2. Clinical operating frontier after event-level alarm post-processing. Markers show the sensitivity and FAR values reported in Table 5; the inset enlarges the overlapping low-FAR region without displacing the observations. The practical trade-off is between high pre-onset sensitivity and clinically tolerable FAR.

The high-sensitivity and low-FAR operating points characterize two complementary uses of the framework. The focal+sampler EEGGPT configuration emphasizes seizure coverage, whereas persistence-based configurations emphasize alarm suppression. Together they show that the discrete vocabulary and decoder can support a tunable event-level alarm pipeline. Controlled comparisons with token-histogram classifiers, raw-window convolutional models, and an EEGGPT variant trained without causal pre-training constitute the next experimental step for separating the contributions of tokenization, sequence modeling, and pre-training.

3.4. Clinical Interpretation and Protocol Safeguards

Figure 3 shows the sensitivity/FAR movement across positive-window definitions before applying stronger smoothing. Moving from detection-like windows toward stricter preictal windows increases the opportunity to learn pre-onset structure, but it also exposes the classifier to more ambiguous preictal/interictal boundaries. This explains why the best sensitivity point and the best deployment point do not coincide.

Figure 3. Prediction sensitivity and false-alarm rate across positive-window definitions at $\tau =0.5$ and persistence $K_p=1$.

Table 6 summarizes the methodological safeguards that distinguish this evaluation from the simpler CHB-MIT protocols. The tokenizer never sees the test subjects; the test fold is person-disjoint; thresholds are selected using the validation data; and alarms after the onset are explicitly separated from the pre-onset predictions. The pooled held-out evaluation measures performance across four distinct test persons using one global alarm policy. A patient-stratified extension reporting event counts, interictal exposure, FAR/h, sensitivity, and confidence intervals would quantify cross-patient heterogeneity and support the development of validation-only patient-adaptive operating points.

Table 6. Protocol safeguards and their impact on interpretation.

Safeguard	Implementation in This Study	Interpretation Impact
Person-disjoint split	Train, validation, and test cases do not share persons; chb01 and chb21 remain together.	Reduces identity leakage and makes the reported numbers closer to cross-patient deployment.
Training-only vocabulary	The k-means codebook is fitted only on training patches and then frozen.	Prevents the tokenizer from adapting to validation or test morphology.
Validation-only thresholding	Checkpoints and thresholds are selected before test evaluation.	Avoids threshold-on-test leakage and preserves the held-out meaning of Sensp and FAR/h.
Strict pre-onset credit	Alarms at or after onset are counted as detections, not predictions.	Separates clinical warning from post-onset detection and lowers inflated success counts.
Fixed global alarm policy	The same persistence/cooldown logic is applied across test subjects.	Reveals the current calibration gap and motivates patient-adaptive deployment rules.

3.5. Comparison with Prior CHB-MIT Studies

Table 7 shows how the proposed framework compares with representative CHB-MIT seizure-prediction reports. This comparison should not be interpreted as a direct leaderboard. Although this study uses held-out patients and strict pre-onset criteria, Truong et al. [3], CBAM-3D [12], CLEP [13], MVNN [14], and OPP [15] use protocols ranging from cross-validation within the patient to subject-specific calibration. Given these constraints, while the proposed method is not numerically dominant, it is transparent and lightweight, and it evaluates the more challenging task of predicting seizures in different patients.

Table 7. Representative CHB-MIT seizure-prediction comparison.

Method	Protocol	Op. Point	AUC	Sens.	FAR/h	Lead
Truong et al. [3]	within-patient CV	–	–	0.81	0.16	–
CBAM-3D [12]	LOOCV	–	0.980	0.984	0.017	–
CLEP [13]	transfer	–	–	0.967	0.072	–
MVNN [14]	mixed	–	1.000	1.000	0.000	650 s
OPP [15]	subject-specific	–	0.994	–	–	76.8 min
Proposed	held-out	high-sens.	0.471	0.727	4.165	280 s
		low-FAR	0.477	0.045	0.202	572 s

4. Conclusions and Future Work

We presented a lightweight codebook-and-language-model framework for strict prediction of seizure-onset from scalp EEG. The k-means vocabulary converts raw multichannel CHB-MIT recordings into one-second motifs, and compact decoder-only LLM variants learn over the resulting token stream. Under a held-out person-disjoint test protocol, the most sensitive EEGGPT operating point predicts 16 of 22 seizures before onset with a median lead time of 280 s, but at 4.17 FAR/h. Persistence smoothing reduces FAR below 0.5/h but collapses sensitivity. This study contributes a leakage-controlled pipeline that connects training-only EEG tokenization, compact decoder modeling, and continuous pre-onset alarm evaluation. Its principal empirical result is the quantified sensitivity–FAR frontier across model and persistence settings, which establishes a reproducible cross-patient baseline for subsequent improvements in representation learning and decision calibration.

For the specified 600 s prediction horizon, the high-sensitivity point exceeds the rate-matched analytical Poisson reference (72.7% observed versus 50.1% expected by chance). Nevertheless, this comparison is descriptive, and the associated 4.165 FAR/h remains unsuitable for unattended clinical deployment.

Future experiments will add matched token-histogram and raw-window baselines, isolate the contribution of causal pre-training, evaluate analytic and randomized chance-alarm controls, and report patient-level estimates with uncertainty. Larger unlabeled EEG corpora, trainable vector-quantized encoders, validation-only patient adaptation, and explainability analyses provide additional routes for improving the sensitivity–FAR frontier while preserving the person-disjoint evaluation protocol.