Teaching responsible AI use in bioinformatics education

When integrating AI tools into a training curriculum, it is essential to equip learners not only with practical skills, but also with the ability to critically evaluate the outputs that these systems produce.

Properly applying AI in research requires critical scrutiny of the model rather than blind trust in its outputs. Researchers should understand how a model was built (including its training data and assumptions), how it has been validated, and how it performs on independent benchmarks. It is equally important to consider uncertainty and the potential for “hallucinations” or plausible but incorrect predictions, particularly when models are applied outside their intended domain. Interpreting results therefore involves examining confidence scores, comparing outputs with known evidence, and recognising that AI predictions are probabilistic estimates, not definitive answers.

A useful example is AlphaFold, whose official training materials explicitly caution users about limitations. They highlight that even advanced models like AlphaFold 3 can struggle with certain cases, such as proteins without homologous sequences, and may produce inaccurate or physically implausible predictions. The materials also emphasise how predictions have been validated against experimental data, while noting that benchmarking does not guarantee correctness in all scenarios. Importantly, users are advised to rely on confidence metrics, sample multiple predictions, and treat results as hypotheses requiring further validation, rather than as conclusive structural determinations.

You can see below AlphaFold training resources that explain limitations of the model: