Beyond Explainability: User Agency, Operational Control, and Interaction Design in Human–AI Systems: A Critical Narrative Review

Gajanan Ankatwar *

Computer Science and Applications, Datta Meghe Institute of Higher Education and Research, Wardha, Maharashtra, India.

*Author to whom correspondence should be addressed.


Abstract

Human–artificial intelligence (AI) systems increasingly influence decisions, workflows and creative production, yet the presence of explanations or a nominal human approval step does not establish that people can direct, question or meaningfully correct an AI-mediated outcome. This critical narrative review examines how user agency is constituted, constrained and evaluated through the interaction design of contemporary human–AI systems. A targeted, multidisciplinary examination of peer-reviewed research published from 1999 to 24 August 2026, supplemented by earlier foundational work where conceptually necessary, integrates human–computer interaction, automation psychology, decision science, organisational research and sociotechnical governance. The synthesis distinguishes explanation, comprehension, calibrated reliance, operational control and contestability rather than treating them as interchangeable manifestations of human-centred design. Experimental studies show that explanations may improve performance where their claims can be checked against task evidence, but may also amplify misplaced reliance, impose cognitive demands or fail to improve complementary human–AI performance. Studies of mixed-initiative interfaces, adjustable recommendations and co-creative tools demonstrate that users' opportunities to select, revise, refuse and renegotiate AI contributions are consequential, while perceived ownership or nominal choice does not necessarily entail consequential authority. Research in clinical, organisational and public-decision settings further indicates that time constraints, institutional rules and unequal access to appeals can determine whether interface controls are usable in practice. An integrative, explicitly provisional framework is advanced in which agency depends jointly on information, feasible action, consequential feedback, accountable recourse and distributional accessibility. The most pressing research needs concern longitudinal, ecologically valid and comparative evaluation of effective override, decision quality, learning, procedural fairness and unintended burdens. Explainability remains useful, but should be assessed as one component within a broader architecture of meaningful participation and control rather than as its proxy.

Keywords: Human-centred artificial intelligence, appropriate reliance, mixed-initiative interaction, algorithmic contestability, human–AI co-creation, decision authority, interaction evaluation, automation bias


How to Cite

Ankatwar, Gajanan. 2026. “Beyond Explainability: User Agency, Operational Control, and Interaction Design in Human–AI Systems: A Critical Narrative Review”. Journal of Engineering Research and Reports 28 (11):60-78. https://doi.org/10.9734/jerr/2026/v28i112042.

Downloads

Download data is not yet available.