Se124M10KInfMinimalist

This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6063

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.3678	1.0	200	1.1110
0.2364	2.0	400	0.7918
0.2048	3.0	600	0.7298
0.1916	4.0	800	0.7019
0.1861	5.0	1000	0.6841
0.179	6.0	1200	0.6719
0.1767	7.0	1400	0.6665
0.1722	8.0	1600	0.6568
0.1714	9.0	1800	0.6538
0.1651	10.0	2000	0.6493
0.1674	11.0	2200	0.6446
0.1655	12.0	2400	0.6407
0.1643	13.0	2600	0.6384
0.1638	14.0	2800	0.6346
0.1619	15.0	3000	0.6347
0.1618	16.0	3200	0.6299
0.1625	17.0	3400	0.6292
0.1586	18.0	3600	0.6262
0.1589	19.0	3800	0.6235
0.1616	20.0	4000	0.6229
0.1609	21.0	4200	0.6218
0.1575	22.0	4400	0.6195
0.1601	23.0	4600	0.6200
0.1577	24.0	4800	0.6159
0.1593	25.0	5000	0.6171
0.1574	26.0	5200	0.6185
0.1582	27.0	5400	0.6139
0.1563	28.0	5600	0.6141
0.1563	29.0	5800	0.6146
0.1595	30.0	6000	0.6124
0.1575	31.0	6200	0.6126
0.1537	32.0	6400	0.6121
0.1559	33.0	6600	0.6104
0.1543	34.0	6800	0.6116
0.1562	35.0	7000	0.6098
0.1558	36.0	7200	0.6089
0.1551	37.0	7400	0.6089
0.1537	38.0	7600	0.6085
0.1526	39.0	7800	0.6084
0.1556	40.0	8000	0.6085
0.1548	41.0	8200	0.6080
0.1542	42.0	8400	0.6078
0.1581	43.0	8600	0.6071
0.1555	44.0	8800	0.6066
0.1547	45.0	9000	0.6064
0.1569	46.0	9200	0.6067
0.1524	47.0	9400	0.6063
0.1555	48.0	9600	0.6065
0.1543	49.0	9800	0.6065
0.1559	50.0	10000	0.6063

Framework versions

• PEFT 0.15.1
• Transformers 4.51.3
• Pytorch 2.6.0+cu118
• Datasets 3.5.0
• Tokenizers 0.21.1