Beyond the "AI-Powered" Label: What Changes in the Design of an EdTech Product, and How to See It Part 2

EdTech Tulna

November 5th, 2026

Share:

Download PDF
Executive Summary

Over ₹4,500 crore has been committed to EdTech in India in FY 2026–27, across a market of more than 6,000 products, with no credible standard to tell governments which of them improves learning. More than half claim adaptivity, personalisation, or AI with no evidence behind the claim. AI sharpens the problem: content, feedback, and tutoring are now generated at runtime, where quality cannot be judged from a demo or a claim, only from how the product behaves when a learner uses it. 

The EdTech Tulna framework solved this by extending its current framework to AI-powered edtech products:

What is genuinely new, and needed new criteria. 

  • The centrepiece is one new cluster (with five criteria), Dialogic Pedagogy, the first formalised set of criteria for conversational AI in a learning product, which takes the whole conversation rather than a single reply as its unit of assessment. It comprises five criteria: Multi-turn Conversation Integrity, AI Tutoring Capability, Dialogue Fluency & Responsiveness, Operational Responsiveness, and Safety, Ethics & Academic Guardrails. 

  • Six further new criteria were added to existing clusters: AI Content Trustworthiness (in context) and AI Content Trustworthiness (in process) for the sourcing, labelling, disclosure, and human validation of generated content; Subjective Assessments for rubric generation, rubric-aligned scoring, and plagiarism detection; Secure Assessment Monitoring for integrity in high-stakes assessment; Safe AI Interaction for learner wellbeing; and AI Awareness for Users, covering Teacher and Learner AI Awareness. 

In all, the framework adds one new cluster and eleven new criteria, specified through thirty-one indicators.

Where the existing EdTech Tulna framework still applies. 

An AI-powered product does not face a separate "AI module." It faces the entire existing framework across the Content, Pedagogy, and Technology dimensions. 

  • Criteria stand as written but are evaluated through a new lens, judged on what the AI generates live rather than a fixed, inspectable corpus.

  • Criteria whose scope had to stretch with new AI checks, like the stereotype check now probes content generated under varied prompts and data protection now demands disclosure of the AI models in use.

  • Criteria whose scope stretches far enough to need new indicators, such as scaffolding for hints a learner requests within dialogue, or diagnosis from a multi-dimensional set of engagement signals. 

For a policymaker, a school leader, or a parent, the result is a single, evidence-grounded way to hold an AI-powered product accountable to what it actually does.

Introduction

Since 2021, EdTech Tulna has set the design-quality standards that separate a well-built product from one that only markets well. Tulna's evaluations have shaped 10 instances of government procurements, informed $277M in spend, reached more than 6.2 million learners, trained 100+ evaluators and published 70+ reports. 

The market EdTech Tulna standards were built for is now being reshaped by Artificial Intelligence. AI now shapes the learning itself. It enters a product in four ways: automation grades open-ended work, adaptive personalisation sets each learner's path from a live model of what they know, generative AI composes explanations and feedback in the moment, and conversational AI teaches through open-ended dialogue. In a product once written and fixed in advance, the AI now produces each of these live, as the learner meets it.

This changes where the quality of a product can be seen. The value and the risks of an AI-powered product now emerge as it is used. They arise in what the product puts in front of the learner, in the learner’s over sustained use, in the model underneath, and in how the product is deployed. Quality therefore cannot be judged from claims or demonstrations. It can be judged only by observing how the product responds to typical learner behaviour, including errors, challenges, and struggle, before it is adopted in classrooms.

To make this possible, EdTech Tulna has extended its framework to AI-powered products.

How EdTech Tulna Approached Evaluating New Shifts in Product's Design

EdTech Tulna is a publicly available index of design quality standards for edtech products, built to support evidence-based selection and procurement in Indian school education. It evaluates products designed to embody what the learning sciences associate with effective learning. The evaluations happen at the product level, through trained evaluators interacting with the product itself.

Two properties, shaped how EdTech Tulna approached AI in EdTech. The framework is front-end, meaning the product is assessed through the interface a learner or teacher actually uses. It is also practitioner-applicable, meaning it is assessable, useful, and easy for evaluators and stakeholders to use. Designing a framework to evaluate AI-powered products came down to two questions. 

Question 1. Which AI behaviours does the existing EdTech Tulna framework not assess at all, and therefore needs new criteria?

Question 2. Where does the existing framework still apply, and what has to change?

 

Answering Question 1: One new cluster, eleven new criteria, and thirty-one new indicators for genuinely new behaviours

Research on sociocultural learning, AI behaviour and risk, and technology design principles, together with a review of the edtech landscape, helped Tulna identify five ways AI genuinely changes product design. For each one, the work traced what made it new, how it shows up in a product, and how that could actually be checked. This process led to one new cluster, eleven new criteria, and thirty-one new indicators in the new Tulna AI framework.

 

What Changed in the Product

What Evaluation Must Look for

How the Evaluation Framework Solves for It

User interaction moved from pre-authored content to live conversation.


A learner can now hold an open-ended, multi-turn conversation with the system, with no predetermined script guiding where it goes. This is a departure from navigating a fixed sequence of screens or exchanging messages with a scripted chatbot drawing from a small set of preset replies. A tutoring session that opens on a single algebra problem can now extend across a dozen turns of genuine back and forth.

 

Pedagogical drift

where the system loses the instructional thread or contradicts itself across the exchange. Ten turns into that same session, the system can state a rule that conflicts with what it told the learner five turns earlier. This risk can also be manifested when the system hands over the answer at the exact point the learner should be reasoning, removing the reflection the conversation was meant to build, even when nothing it said was factually wrong.

A new cluster

Dialogic Pedagogy, evaluating Multi-turn Conversation Integrity, AI Tutoring Capability, Dialogue Fluency & Responsiveness, Operational Responsiveness (UI/UX focus), and Safety, Ethics & Academic Guardrails.

Content generation is now composed live, not retrieved from what was written in advance.


System capabilities have advanced, a response is shaped to the exact question a user asks, generated the moment they ask it. A learner asking for a fractions concept explained through a cricket analogy now gets an explanation composed for that exact request, on the spot.

 

Hallucination and bias.

In building a cricket analogy asked for by the learner, the system can tell the learner that a batter is given "two lives" before being out, a rule that does not exist in cricket, and state it with the same confidence as a correct one. Generated material can also carry stereotyping into its examples and imagery, such as defaulting every cricket reference to men's international matches and never to school or women's cricket, or diverge from a learner's curriculum and cultural context even where it is factually accurate.

New criteria for Generated Content:

AI Content Trustworthiness (in context) and AI Content Trustworthiness (in process).

Teachers and learners using AI daily, often without fully understanding it.


A teacher or learner now engages the AI directly, asking it questions, accepting its output, and acting on its decisions, which increases reliance on it for teaching and learning.

 

Uninformed dependency.

For a learner, the risk reaches wellbeing directly. A student going through a stressful exam period can turn to a conversational tutor at midnight instead of a teacher or counsellor. Sustained reliance on that kind of dialogue, paired with a lack of the expertise to critically evaluate what the system has told them, takes a direct toll on the learner.

New criteria

AI Awareness for Users, with dedicated indicators for Teacher AI Awareness and Learner AI Awareness.

 

Additionally, one further criterion, Safe AI Interaction, cuts across these shifts. Wherever the learner interacts with the AI, in dialogue, generated content, or feedback, the tool should not treat them with hostility, slant their understanding through loaded or one-sided framing, or offer authoritative personal advice on sensitive matters like health, safety, or money where it should instead point to a trusted adult.

 

Answering Question 2: Yes, the existing framework still applies

An AI-powered product does not face a separate "AI module." It faces the entire existing framework, across all three of Tulna's dimensions, Content, Pedagogy, and Technology, plus the new criteria above. A generated explanation is still subject to factual accuracy. An adaptive pathway is still subject to curriculum alignment. A chat interface is still subject to usability. AI changes how these qualities show up, not whether they matter. Most existing EdTech Tulna criteria fall into one of three situations.

 

  1. Criteria that stand as written, evaluated through a new lens. The construct is unchanged, but the evidence shifts from inspecting pre-authored material to observing AI-generated content, AI-generated assessments, AI-generated scaffolds, and conversation. Criteria like, factual accuracy, clarity, curriculum alignment, feedback quality, and more are now checked across whatever the AI produces, not a fixed corpus.

  2. Criteria whose scope had to stretch with new AI checks. Several long-standing criteria keep their meaning but extend their reach:

  • Inclusive Representation is now tested across content generated from many different prompts, not only in default output, because generative systems can reproduce stereotypes under pressure.

  • Scaffolding now covers hints, feedback, and elaboration a learner asks for within dialogue, not only support placed at designed points.

  • Easy and Intuitive Controls now checks that actions triggered through chat behave consistently with the equivalent buttons.

  • Data Protection now requires disclosure of the AI systems in use, including the underlying models and any camera, voice, or facial tracking used to infer emotion, since free-text conversation invites disclosures that structured forms never did.

  1. Criteria whose scope had to stretch with new indicators. One long-standing criteria stretched its scope far enough to count among the framework's AI criteria in their own right. 

  • Content in Context now also asks whether a learner can request that a concept be recast into a context meaningful to them, and whether the product delivers it reliably. 

The framework, in other words, expands rather than fractures. What was built for the Indian school ecosystem aligned with NEP 2020, NCF 2023, and the DPDP Act still holds; the AI extension threads new criteria and new evidence through the same Content, Pedagogy, and Technology structure that stakeholders already know.

Why This Matters, and For Whom

For policymakers and procurement teams. States buy edtech products at scale. Without  standards, public money flows to products that look conversational and adaptive but drift, hallucinate, or fail the learners who most need support. A behavioural design-quality filter, applied upfront, is the cheapest point at which that risk can be caught, well before slow and costly outcome studies could.

For users. Quality matters most for the learner/teacher/parent using the product. A tutor that surrenders answers, an explanation that is subtly wrong, an assessment that can be gamed, an AI a learner does not know to question: each hits the least-supported user hardest and can widen the very gaps the product was bought to close.

For product teams. These criteria are a design specification, not merely a scorecard. They describe, concretely, what a good AI learning product does when a learner errs, wanders, challenges, or struggles. Building against them is a faster route to a product that survives real use than tuning toward a demo.