The development of artificial intelligence in education not only increases access to information but also introduces a new epistemological risk: AI hallucinations, that is, content which appears credible yet lacks grounding in reality. This article analyzes the phenomenon from the perspective of Evidence-Based Management (EBMnt), arguing that the issue is not merely technological but fundamentally methodological and decision-related. In particular, AI hallucinations are compared with classical scientific malpractices such as HARKing and p-hacking, demonstrating that all these phenomena lead to the production of pseudo-evidence. Consequently, a redefinition of evidence becomes necessary, alongside a shift toward managing the credibility of information.
Credibility
For a long time, education operated under conditions of information scarcity. The key competence was the ability to search for and assimilate knowledge. With the emergence of AI tools, this situation has changed radically — information has become instantly available, personalized, and linguistically refined. However, this shift has revealed a new problem: not the lack of knowledge, but its overabundance in forms that are difficult to distinguish from reliable information. AI hallucinations are a symptom of a deeper transformation — from the problem of information accessibility to the problem of its credibility.
From the perspective of Evidence-Based Management, it is particularly significant that the source of information begins to simulate evidence. AI-generated responses may fulfill all superficial criteria of correctness while lacking empirical grounding.
An Epistemological or Merely Technical Phenomenon?
In educational practice, AI hallucinations manifest themselves in various forms: from subtle shifts in meaning, through incorrect interpretations, to entirely fictitious sources. Their defining characteristic, however, is not falsehood itself, but its credible form. AI hallucinations are “complete,” “elaborated,” and “persuasive” — they imitate the structure of knowledge rather than its sources. From an EBMnt perspective, this implies a crucial shift: from evaluating content to evaluating the process by which it is produced.
This approach is especially important in areas requiring specialized competencies — such as remote education, accounting, coaching, or craftsmanship — where apparent correctness may lead to real decision-making errors. In such fields, knowledge is not merely informational but actionable, making the issue of hallucinations particularly consequential.
AI hallucinations are often treated as a technological imperfection. However, their nature is more deeply rooted and resembles well-known methodological flaws in science. Two phenomena are especially relevant here:
- HARKing (Hypothesizing After the Results are Known) consists in formulating hypotheses only after obtaining the results and then presenting them as prior research assumptions.
- p-hacking consists in manipulating data analysis (for example, through variable or sample selection) in order to obtain statistically significant results.
Structurally, AI hallucinations work in an analogous way:
- the result (the answer) is generated first,
- the justification is added afterward,
- the whole is presented as a coherent and rational process.
In this way, an illusion of evidence emerges, one that results neither from data nor from methodology, but from alignment with the expectations of the audience.
This logic is increasingly relevant in education and management as well. In a world saturated with AI-generated content, it is not enough to expand access to information. The cognitive environment must be designed to support selection, verification, and credibility assessment. In other words, the challenge is not to see more, but to recognize more accurately what truly matters.
Knowledge or “Magic”?
Evidence-Based Management assumes the integration of multiple sources of evidence. Traditionally, the main problem was the limited availability of evidence or its uneven quality. Today, this situation is reversed — the challenge is no longer information scarcity, but its overproduction, in which distinguishing knowledge from its simulation becomes increasingly difficult. AI hallucinations, much like HARKing and p-hacking known from scientific methodology, lead to what may be described as pseudo-evidence. These are constructs that take the form of argument, use the language of science, and fit the cognitive expectations of the recipient, yet lack genuine grounding in empirical data. Their strength does not derive from truth, but from credible form.
In this context, artificial intelligence ceases to be a source of knowledge in the classical sense. Rather, it becomes a generator of possible narratives — linguistic hypotheses that must still be confronted with scientific research, operational data, practical experience, and stakeholder values. It is precisely this integration that constitutes the core of Evidence-Based Management and, at the same time, the fundamental mechanism of defense against pseudo-knowledge.
One may say that contemporary knowledge management increasingly resembles a choice between two orders: knowledge understood as a process of arriving at truth, and “magic,” which creates its convincing illusion. In the first case, we are dealing with the effort of verification, uncertainty, and methodological discipline. In the second, there appears the temptation of a quick effect, where the result matters regardless of its epistemic foundation. AI hallucinations situate themselves dangerously close to this latter order.
They produce answers that work — they are coherent, persuasive, and useful — but they are not necessarily true.
As a result, a shift occurs in which decisions may increasingly be made not on the basis of evidence, but on the basis of its simulation.
From the perspective of management practice and didactics, this means that the relationship between data, interpretation, and decision must be redefined. Particularly in digital environments, it becomes easy for the tool to begin replacing the cognitive process rather than supporting it. Experience at the intersection of entrepreneurship, education, and research shows that technology — however useful — must be balanced by methodological rigor. In this sense, it is no longer sufficient to manage information. What becomes necessary is the management of its credibility.
Ultimately, then, the question is not whether to use AI, but in what epistemological order we want to function: in a world of knowledge that requires verification, or in a world of “magic” that merely imitates it.
|
| Alex Andrews | pexels.com |
Hallucinations as a Cognitive Stress Test
Rather than treating AI hallucinations solely as a threat, they may also be understood as a diagnostic tool. They reveal not only flaws in the technology itself, but also weaknesses in the educational and decision-making systems within which they operate. When confronted with AI-generated content, it quickly becomes clear whether the user can distinguish information from evidence, understands the process by which knowledge is produced, and is able to recognize methodological errors — both those resulting from human simplifications and those generated algorithmically. In this sense, AI hallucinations function as a kind of stress test for Evidence-Based Management. A system that truly relies on evidence integration remains resilient; a system that merely declares such an orientation succumbs to the illusion of credibility.
AI hallucinations, like HARKing and p-hacking, share a common denominator: they produce images of reality that are persuasive yet epistemically fragile. Although they differ in their mechanisms — from algorithmic prediction to selective data interpretation — they lead to the same result: the erosion of decision quality. Consequently, the essential shift no longer concerns technology itself, but the way we think about knowledge and its role in action. What becomes crucial is the transition from information management, understood as collecting and processing data, to the management of information credibility, which assumes its constant verification and confrontation with multiple sources.
In a world saturated with AI-generated content, Evidence-Based Management ceases to be merely one possible method among many. It becomes a condition of meaningful decision-making. Without it, the risk grows of acting on the basis of what merely resembles evidence, but is not evidence at all. Ultimately, then, the problem does not reduce to the question of whether AI makes mistakes, but to a far more demanding one: whether the system in which we operate — educational, organizational, or cognitive — is capable of recognizing, understanding, and correcting those mistakes.
Source: M. Jabłoński, A. Jabłoński, P. Janulek, D. Dulęba, M. Glenszczyk,Treatise on the Principles of Evidence-Based Management – The Future of Management (TRAKTAT o zasadach zarządzania dowodowego – przyszłość zarządzania) 2025, CeDeWu, p. 190.
Posts
