AI Characters for Children

Branislava Lovre: Welcome to AImpactful. Today we’re exploring how children connect with AI characters and what that means for learning. Our guest is Sonia Tiwari. Her work explores the ethical design of AI characters for children and responsible AI in education. Welcome, Sonia.

Sonia Tiwari: Thank you for inviting me.

Branislava Lovre: As with every episode, I like to start from the beginning. You have an impressive career. You started in gaming and gradually became one of the leading voices in AI characters and children’s learning. What was the main motivation and how did you become so interested in this field?

Sonia Tiwari: I think every 10 to 15 years the media landscape shifts. So, you know, in the beginning there was broadcast television, TV shows would be on at a particular time. And so it was a shared experience like children all around the country would watch the same show at the same time, and they had only that one point of reference. The next day when they meet at school, they would talk about their favorite characters or story that they saw the day before.

Then, you know, streaming services came up. Everyone is watching different things. Now every child has a different kind of favorite character because everyone’s media diet is kind of different.

And so now we’re beginning to see a similar shift with AI, custom programming, custom content, custom character design and stories that are becoming more and more common.

And I’m not saying that, you know, pre-planned and well-designed stories are going to be completely replaced by generative things. I’m just saying that, you know, it’s an option now.

And so the way children engage with characters, from children’s books to TV shows and movies to all this streaming content, and now with AI, it’s just an evolution of how we experience media.

And for anyone who is in this field of storytelling, narratives, character design, it’s just a way of adapting to the needs of the time.

So I think, because AI is here, whether we like it or not, it’s better to understand it and to find ethical use cases.

Branislava Lovre: Do children relate differently to AI-powered characters compared to traditional animated or game characters?

Sonia Tiwari: Yes. So with AI characters, of course, the biggest difference is that these characters are now able to talk back more organically.

There was two-way interaction in gaming for a long time, but those were pre-programmed kind of conversations.

Designers and game writers would anticipate the kind of conversations that a character might have, and they would pre-write everything.

And then, you know, the algorithm would just… it was like a branching scenario, basically, that if someone asked this, you repeat this phrase, but now you can prompt the personality of a character, and then it acts completely organically. So the findings so far are different for different age groups. For example, in the news mostly we’ve heard about pre-teens and teens and the mental health impact they have from having developed parasocial relationships with AI characters. So that is more on the harmful side.

But on the positive side, you know, tutors are a great use case where there’s no emotional bonding. It’s more purpose-driven. There are a lot of Socratic tutors. That means they won’t give you the answer right away. They will lead you to the answer.

So if you’re doing a math problem, or if you’re coding something and you ask for help, it won’t directly say, “Oh, here’s the answer.” It will ask you a follow-up question like, “What do you think?” “What is something else that can be changed for a different outcome?” and eventually lead you to the answer, which is the characteristic of a human coach.

So it just depends on how we use it, there are both positives and negatives.

And for younger kids, it’s mostly smart toys. Again, they’ve been in the news for good and bad reasons. Within limit, you know, if it’s guided conversation for a few minutes, a little bit of interaction is not very harmful. It can be beneficial if used right.

But of course, if we leave children alone with an AI toy to talk for hours, then it can be… then it will be bad for them developmentally.

Branislava Lovre: You mentioned both the positive and the negative sides, and I think that’s exactly where a lot of teachers and parents find themselves. They see new possibilities, but they also feel overwhelmed or unsure how to approach this. What would be your basic advice for helping children use AI in an ethical way?

Sonia Tiwari: Right. So I think most educators already know this, that foundational skills are really important.

So it’s not like we are expecting educators to abandon all the good stuff that they were already doing and now replace everything with AI.

AI literacy, for example, in California, it’s mandated from early grades, like from kindergarten all the way to high school. But there’s no guideline as to what that exactly means in practice. So depending on who the teacher is, everyone is going online, finding some resources, trying their best.

So the best advice, I think, for families and teachers is: do not feel overwhelmed by the amount of information overload that exists around us. Everyone is trying to give you a framework, give you a new tool. So just slow down, focus on the foundational skills, and then try maybe one new tool every week or every other week so that you can really experiment and understand it before you move on to the next thing.

What happens is that everyone is kind of bombarded with these, like, “try this, try that.” And so if you’re trying too many tools at once, then there’s not enough energy left to focus on the foundational skills.

So in writing, that would mean actually sitting down to write. In art, that could mean actually sitting down to learn some basic color theory and different shapes and shadows and just observing nature. Those are basic creative skills that we still need to practice, and same goes for math, science and everything. So before we try intelligence augmentation, we need to build the baseline intelligence.

Branislava Lovre: That’s really helpful. Focusing on foundational skills first. But practically speaking, where do you even start?

Sonia Tiwari: I think we should start with the need. Like, what is an area where existing tools and systems are not proving to be helpful.

So for example, someone needs a math tutor. Now, the traditional classes like Kumon or Art of Problem Solving or Russian math, all these places can be expensive. Maybe there is a child who needs help, but their parents can’t afford this kind of additional support. So for them, having access to an AI tool  is very helpful because, you know, then they don’t have an alternative resource.

Or in terms of filmmaking, for example, I have supported many schools whose art programs have been cancelled because of lack of funding. So maybe they don’t have the budget for costumes. We can use video-to-video AI to kind of add costumes later through AI.

So it just depends on where the problem is, where the need is. So one way to just explore is open-ended exploration. There’s no need, there’s no goal. Just try out a bunch of different tools.

Another way is to start from that need: “Where is my problem and which AI tool can best support it?” And even just now with intelligence search options, you can even Google that: “This is the problem I have, which AI tool should I try?” And so you get pretty good recommendations that way as well.

I would also advise families and educators to try these tools on their own first before handing them off to kids, or even doing it together with kids because a lot of times it might say that this is educational or child-friendly, but it may not be because it’s very easy to kind of get around the laws to pretend to be a child-friendly tool. So I always advise to read the reviews and try it out yourself first and then try it out with the child together.

And then when you have enough trust and you see enough need and you’re able to monitor it, then if there is a need, then have the child interact with these educational tools by themselves.

Branislava Lovre: You developed a framework for evaluating AI characters for children. What are the key signals that a product is truly child-appropriate?

Sonia Tiwari: Yeah, so that framework is to design ethical AI characters.

And it’s meant more for designers to kind of evaluate what is already out there. Is it working? And if they’re designing something, then how to distill those insights towards design.

So it’s divided into three different buckets. One is: we start with the child.

What is developmentally appropriate for that particular child? Does the product say that this is for such-and-such age group and we are focusing on these specific skills? It’s an indicator whether they are actually thinking about the developmental profile of a child, or they’re just creating a generic open-ended tool that anyone can use for anything.

So just seeing that developmental alignment is of course important, and then setting up some kind of learning goal upfront.

I personally like, when I’m working with families or children in makerspaces and libraries, I tend to bring in tools that have a very specific and upfront goal. I don’t like those open-ended tools because I, as a facilitator, cannot guarantee the outcome if I don’t know what the goals are.

So I personally try to define a learning goal upfront, and then once we understand the needs of the child, then we start looking at the character design aspect. Because I was a character designer in the gaming industry, I apply those kind of lenses to my AI characters as well. Thinking about the visuals, the colors, the voice, the type of content or conversation patterns, the type of personality a character will have. Like, tutors usually have a friendly mentor persona that is approachable but also wise in a way it can offer advice. Sometimes it’s like Tamagotchi-type of characters where you need to learn about nurturing and caring behaviors by taking care of a character.

So it just depends on what the learning goals are, how old the child is, what’s their developmental context, and how might this character support that? And then once we have these character and child factors figured out, then we can start thinking of the interaction. So, my recommendation to designers is to think of an average session time, because I don’t think there’s any benefit in a conversation going on for hours. There are more like real-world things that children need to do. And then also, learning happens best when you absorb some kind of content which is dense in information, then you pause.

That gives you time to truly absorb it, reflect on it, practice it, apply it, then you come back and learn some more. So even if it’s like high-quality educational content, hours of conversations are not going to be helpful. So thinking about a session time, thinking about some kind of semi-structure to the conversation. If you have a goal upfront… let’s say in Buddy AI, for example, the goal is practicing to speak English. And so if a child is learning about shapes and colors, they’re learning about weather. If that goal is established upfront, that would be the scope of the conversation. Anything outside of weather or colors… that will be kind of…

usually you get a system response that: “I cannot answer that question” or “Let’s focus here first.” If it’s a very personal question, then, you know, “Why don’t you have this conversation with a trusted adult in your family?”, that kind of thing. And then most of these children’s products also have a parent-facing dashboard, so parents can keep an eye on the type of conversations that are happening. And then many of the good products, like Toybot for example, it also has prompts for joint engagement.

So the toy would serve more as a facilitator like, “Bring your family, let’s all play together,” as opposed to: “I’m your best friend and you can tell me anything and I’m here for you,” that kind of deception. So it depends on how we frame these tools and what purpose they’re serving.

Branislava Lovre: Now, looking ahead, what can we expect in the next year or two when it comes to AI characters? What should we prepare for and what should we be excited about?

Sonia Tiwari: Okay, so, on the good side, I feel like because the technology is getting really good at producing consistent results… you know, like last year we saw those six fingers, or faces looking different if the character turned sideways. All of those things are resolved now.

Because the production capabilities are good, an educator who does not have the production resources of a major studio can also produce high-quality educational content.

For content creators who have a background in education, this is good news: with AI, they can accomplish more.

On the downside, of course, I predict we’ll see more of these kinds of franchising deals between studios and companies. That can mean that for independent creators, it may get tricky to compete with all these big brands.

And then on the negative side, it’s also important to mention the environmental impact of these technologies, that’s continuing to grow.

I’m hopeful that if not this year, in the next 4 to 5 years, people will figure out a way to reduce energy consumption for a similar quality of results.

I’m hopeful about that, it’s just going to take time. But if it’s a miracle, maybe it will happen this year that, you know, someone figures out a way to be less resource-intensive.

Branislava Lovre: You mentioned environmental impact. I know you’re also doing important research in this space. You share so much valuable knowledge on LinkedIn. What’s the main question you’re exploring in your research right now?

Sonia Tiwari: I think I’m getting interested in this idea of creative augmentation, which is, in the human-computer interaction field, the idea of intelligence augmentation.

AI is just the technology, but intelligence augmentation is kind of the application.

So I’m curious about: where is that automation point in the creative process, so that until here this is human creativity? After this, if I outsource to AI, is it okay? Where is that point?

Because, you know, if from the start a child is kind of prompting: “Oh, this is a basic idea of a story that I have, can you turn it into a fully written story?” Would you consider it creative? Because the prompt is what the child came up with.

So it’s their creativity, but the finished story is what the AI did. So, like, where is the creative ownership, and where is the creative development? And when I say that focus on developing foundational skills, then at what point it’s okay to outsource it to AI, right?

So that is something I’ve been struggling with myself as an artist, because I went to school for this, so, I would like to hold on to the fact that I want to be able to handle the entire creative process by myself, and I take pride in that.

But I also acknowledge that, you know, maybe there are people who don’t have those skills. And is it not wonderful to give them this tool that will allow them to be creative? Or is it being lazy, that instead of trying to learn it, you’re just outsourcing everything?

So those are the questions on my mind, like, figuring out the right point of automation within the creative process.

Branislava Lovre: Where does human creativity end, and where does AI take over? Where do you think the line is? At what point should we not hand things over to AI in the creative process?

Sonia Tiwari: The databases that these LLMs are trained on, it’s the aggregate of all public information that’s documented.

In that sense, the way our creativity operates, it’s not built on the database of what everybody knows. It’s built on the database of what we have experienced, our lived experience.

And so for me, that’s my starting point, that what we bring to the table in the creative process are those weird little observations and lived experiences that we have had.

Because if you look at any painting or sculpture, or you talk to any artist and you ask them, “Where did this idea for a character come from?”, it’s usually like, “Oh, I had this uncle,” or “I knew this friend from my childhood that I haven’t spoken to in a while,” and they inspired me. And so there are these either heartbreaking or very happy stories from people’s lives that lead them to write stories and lead them to make characters.

It’s a very personal connection, and that is what carries through.

And we’ve seen in any movie where a popular character, after three or four or five series of movies… like Iron Man, for example, when Iron Man died, a lot of fans really felt that moment, because they had fallen in love with the character through the comic books and through all these series, and then you feel for it.

Whereas if it’s a character that you just prompted like 5 minutes ago, there’s not enough history, there’s not enough connection there.

And there’s a professional writing team behind those characters that bring them to life, who all went to film school or have degrees in English literature. They spent all their lives reading something before they became writers.

So you’re not just seeing the character. You’re seeing the character distilling from this history of writers who had been reading, and they had a full life and they brought all of their emotions and poured them into this one character. And so we see the end result. We don’t see this beautiful human process behind it.

And so that’s where my hope is, that that’s how we can separate creativity still.

Branislava Lovre: How do you approach the question of labeling AI-generated content, especially when children might see familiar characters in unexpected or even harmful situations?

Sonia Tiwari: The challenge is that there are so many other competitor products that are unregulated.

So if you look online, there are many videos and photos of Elmo, who is such an amazing ethical PBS character, with guns, Elmo doing all sorts of inappropriate things. So those videos and photos are already out there.

And unfortunately, until we get really, really strong with regulation, that will continue to happen.

The other way to balance this out is just through AI literacy for young kids.

Just having this conversation with them that, you know, maybe you love Bluey or you love Curious George or Daniel Tiger, whatever characters that you love, make sure that we’re watching them together and like define the locations, where… like on Disney Channel at home, in the living room or like on your iPad only in this app.

Those kinds of rules for young children, about where they should access the correct media, because it can be very deceiving if they go off-platform into some, you know, Instagram, YouTube or anywhere else and they see a character that they are familiar with.

There are many episodes of Peppa Pig on YouTube that are not the official ones, right? Like, it’s so bratty. And so if a child who is unaware of this comes across this, they’ll totally believe it.

So, yeah, just preparing kids for that scenario that even if you see someone who looks like your parents… because anyone can create a deepfake of a trusted adult for a child and then deceive them or, you know, say that: “Hey, your dad left this message for you, come with me.” That’s also possible. So we just need to prepare kids for what we say: “stranger danger.” Now, part of that is deepfakes and other ways of deception.

Branislava Lovre: Do you have a final message for our audience, parents, educators, anyone working in this space?

Sonia Tiwari: I think I’d just say: the power of characters is that they draw us in with a good story. All humans enjoy stories. It’s a biological thing. We enjoy stories.

So let’s not just depend on AI to scratch that biological itch of a good story.

Continue to read books, continue to watch television and films and real-life plays, and even make up your own characters.

So there are many ways to enjoy a good story and a good character, and make AI a very small part of it where you think it’s going to be a helpful use case for learning.

Branislava Lovre: Thank you so much for your time and for all the knowledge you shared with us.

Sonia Tiwari: Thank you.

Branislava Lovre: Thank you so much for joining us. This was AImpactful. See you next time