The AI-Assisted Design of Learning Experiences for Echoborgs: Practical Concerns & Speculative Considerations

Author: Amanda Tkaczyk, MMAI, CIPP/C, CIPT 

My Positionality

I have a master’s degree in Artificial Intelligence. I have advised, consulted, and built applied AI projects. I am deeply concerned about the environmental, social, emotional, and equity harms of AI, especially with poor regulation, enforcement, and oversight. It can be overwhelming to bring together each of these themes, so any oversight is not due to lack of care. I am enthusiastic about the potential of AI to improve healthcare and food security. I proceed with caution in my personal use of AI tools. I also fully recognize I cannot directly control or police how anyone else uses these tools.

Two Moments That Sparked My Project Motivation

In Spring 2023, I had experiences in the classroom where students chose to present AI-generated text, on AI-generated slides, with AI-generated data, and AI-generated  scripts. I felt defeated as an educator. While recognizing their autonomy and social pressures in deciding to rely on AI despite the academic integrity consequences and missed opportunities for learning, I began to think critically about the future of learning when all students have unfettered access to these tools. What might happen to the traditional classroom presentation assessment? Where’s the learning value of peer presentations or online discussion boards if students are regurgitating AI-generated text to one another in a continuous loop?

In Fall of 2023, while reading about one of my other interests, emotional AI & online dating, I came across a 2016 research paper about the concept of  “Echoborgs” – humans whose behaviour is determined in near-real time by artificial intelligence. When interacting with an Echborg who is being told what to say or what action to take, our interactions, emotional experiences, and learning experiences are no longer human-to-human but rather a subset of a hybrid human-AI interaction. Trying to explore this concept further my online searches showed no recent academic papers, no LinkedIn posts, and no news alerts other than promotions for a UK theatre experience. How was the popularity of AI tools taking over media across domains, but I could find little evidence of ongoing conversations about this concept online? For a moment, I questioned the credibility of the authors      but realized the lead author, Kevin Corti, is now a UX Researcher at Google. 

Combining my teaching experiences with the curiosity of the implications of Echoborgs in our society, I wondered: how might we fulfill the learning needs of Echoborgs? What tools and resources will be developed for the future of learning where all students are plugged into AI at all times?

In this practice-based exploration, I examine ethics in design from two angles:

1. Practical Concerns – The use of AI-assisted design for learning experiences
2. Speculative Considerations – The ethics of speculative AI teaching and learning technologies

Practical Concerns: AI-Assisted Design

To inform my perspective of the practical concerns of AI-assisted design and to immerse myself in the perspective and lived experiences of AI-dependent learners who interact as Echoborgs, I chose ChatGPT as my sole collaborator for this project. Rather than engage other humans, I was intentional about relying on generative AI as a brainstorming, evaluation, and feedback tool. This was both a frustrating and delightful experience. In moments where I would have celebrated insights, shared a laugh at the absurdity, or asked peers for their perspectives, I substituted prompts. The responses were validating, insightful, and beneficial. I sit conflicted with the convenience, valued contributions, and simulated social connection. I can now better empathize with those who have fully adopted generative AI into their everyday tasks and design practice. I do not know how I will balance my use of AI tools and human collaborators in the future. 

In this section, I describe three use-cases for AI-assisted design and reflection on the ethical tensions. 

Designing Personas 

When developing personas without the use of AI tools, designers gather information through ethnography and netography techniques such as interviews, content analysis, observations, and data mining. With access to AI tools, designers can prompt to generate highly detailed personas. 

By curating prompts that specify characteristics related to identify, demographics, psychographics, and lived-experiences, these personas illustrate generative simulated diversity.

Sample Prompt to Generate Personas with Diverse Characteristics

Given the critical design project above, let’s create a 3 x 3 matrix of personas for our design ideation.  I have the 3 x 3 titles and I’d like you to populate the relevant details of the persona.  Columns: AI Resistant, AI Assisted, AI Dependent  Row: Young Learner, Adult Employee, Educator  For each box Column x Row, please create the following:  Name, Age, Ethnicity, Interests, Likes, Dislikes, Wishes For the Future, Worries for the Future, How They Like to Learn, How They Respond to Change

 

Sample Generated Personas with Persona Individual Name, Persona Catchy Name, and ‘Most surprising characteristic about their AI-enabled learning capabilities”
	Resistant	Assistant	Dependent
Young Student	Emma the Traditionalist Despite resistance, shows curiosity towards interactive simulations when presented in a physical setting	Lucas the Explorer Quickly adapts to and masters new educational apps that gamify learning	Mia the Tech Whiz Can troubleshoot and improve AI learning tools herself, providing feedback on functionality
Adult Employee	John the Skeptic Becomes a strong advocate for AI tools after seeing their productivity benefits in a controlled pilot project	Sofia the Balancer Excels in creating hybrid workflows that optimize both AI tools and traditional methods	Ahmed the Innovator Utilizes AI to predict market trends and improve strategic decision-making
Educator	Mary the Preserver Develops a hybrid curriculum that gains popularity for integrating classical and modern teaching methods effectively	Carlos the Integrator Creates innovative, tech-infused music lessons that engage students deeply	Aisha the Innovator Pioneers ethical AI research and implementation in educational settings, influencing policy and practice

Designing Journey Maps

Without the use of AI, designers create journey maps informed by process mining, interviews, and sentiment analysis. When using AI tools, the structure, format, and description of emotional experiences can be generated. 

Sample Prompt to Generate Journey Map Data

Let’s create two sets of Journey Maps for the 3 student personas for an educational experience with the following characteristics.  Scenario: Learning how to do algebra for their grade level or job function.  JOURNEY MAPS  1. BEFORE AI – No access to AI tools or limited access to AI tools for some feedback  2. FULL AI – Unrestricted real time access to tools for monitoring emotion, learning, performance.  Assume that all of the personas have FULL access to AI tools that are continuously telling them what to do and providing feedback in real time. Journey Touchpoints: create 6 touchpoints related to a learning experience end-to-end  For each Persona, create a table  Columns – Journey Touchpoint Title  Rows – Emotional Experience Score, Emotional Explanation, Learning Experience Score, Learning Explanation, AI Experience Score, AI Experience Explanation

Designing The Design Feedback Process

In a traditional feedback process, designers share their work with others, conduct evaluations, and do experiments. When using AI tools for feedback, the approach and range of evaluation tools can be widely explored for simulated feedback and scoring. This feedback can be immediately compared and integrated into the design.

Human-AI Feedback Process – How Amanda used ChatGPT for revisions
1	2	3	4
Amanda created 5 ideas	ChatGPT revised 5 ideas for clarity and style	Amanda prompts ChatGPT for critique & to revise 5 ideas based on the feedback	ChatGPT revised 5 ideas
5	6	7	8
Amanda prompted ChatGPT for feedback on the co-design activity	ChatGPT provided feedback on the co-design	Based on the  feedback, Amanda provided additional information and asked ChatGPT to revise the 5 ideas and provide 2 novel suggestions	ChatGPT revised 5 ideas and generated 2 previously unexplored ideas

As described in Steps 3 and 4, I prompted for feedback using the following 5 frameworks or tools with a specific request to informally explore and demonstrate the range of capabilities:

Consequence Scanning – 3rd order consequences of climate change

1. Community Jury – a perspective of a victim of domestic violence
2. Google’s What-If Tool – generate 1 response from the evaluation framework
3. AI-Generated Fairness & Equity Evaluation Framework – generate a score for fairness & equity and generate 1 feedback response
4. OTESSA Attendee Interest & Adoption Scoring – given the context for the conference and my motivations, generate 1 feedback response 

As described in steps 5 through 8, I prompted ChatGPT to generate a reflection based on our human-AI collaborative design of the ideas. Based on this feedback, I added more details to the scenario and goals. With additional context, I prompted ChatGPT to revise the ideas again and provide an explanation for the improvement. 

Three Tensions

• Time vs. Environment

• While I saved time and money by using AI to generate artifacts rather than conducting interviews or analyzing data, there is an environmental cost to generative AI prompts. How do I balance the tradeoff of my timesaving with the harms to environment when performing queries to computationally intensive large language models?

• Simulated Diversity vs. Exclusion

• Generative simulated diversity is controversial because we know that ChatGPT can perpetuate harmful stereotypes and mischaracterizations. However, with limited resources or perspectives, designers risk excluding communities in their design. This could be due to lack of access or lack of consideration. While there is a risk of causing harm, there could be possible significant positive outcomes by including diverse characteristics in generated artifacts. How do we assess and manage the risk of simulated diversity compared to the harms of exclusion? 

• Balancing The Scope

• Through the three AI-assisted activities, I varied the level of ChatGPT instruction. For example, in the Journey Mapping activities, I did not prompt for specific touchpoints but rather requested ChatGPT “generate six touchpoints”. This contrasts the Feedback activities, where I prompted for specific evaluation frameworks. In traditional design, it is often impractical to do extensive qualitative analysis or broaden the scope of design to 100s of personas. Now that we have generative AI tools to assist in this, do we have a responsibility to do more? How do we balance the scope of the instructions with outputs? Lastly, how do we fairly determine the “autonomy” of AI decisions in our work by varying the level of instructions?

Speculative Considerations: Teaching & Learning Resources

With the design artifacts, as well as my own perspective on the gaps in educational AI tools available as of Spring 2024, I created briefs for 5 speculative tools. After drafting the 5 initial tool briefs, I engaged with ChatGPT for feedback to refine the ideas and generate descriptions of 2 additional tools as described in the previous section. Lastly, I used Canva’s AI tools to co-create the text for the resource descriptions and sample prototype artifacts.

These 7 speculative tools raise familiar ethical questions about ownership, transparency, trust, and the value of expected outcomes. 

Tool	Brief Description	Ethical Concerns & Open Questions
Reflection Rebel	Don’t waste your time crafting reflections for your students! Let the AI lead the way and sneak in some fibs for them to catch.	PARA-EMOTIONAL INTERACTIONS When educators use AI tools as substitutes for student feedback and interactions, how does this shape engagement and trust? Does this benefit or harm student learning?
Echoborg Interrupter	Students using AI to listen to your lectures? Slip in sneaky phrases to throw them off track and stir up some hilarious chaos!	USE OF ADVERSARIAL AI Is it appropriate to use adversarial tools to surveil or trick students in the classroom? If educators do not use these types of tools, how does that shift the power dynamic of the classroom?
AI Class Customizer	You can ditch the jokes and charm because now you’ve got real-time translation on your side!	AI-ENABLED PERSONALITY With access to technology where individuals can boost their likeability, does this enable new paths for social fairness?
Baseline Bob	Help your students nail those essays, hitting the grading bullseye without dipping into AI wizardry beyond their realm!	EVALUATION INFORMATION ASYMMETRY With educators using emerging tools to evaluate student work, what is the most fair way to share this with students to promote equity? In traditional classrooms, educators could anticipate information access students might have to promote integrity and fairness, with AI tools publicly available, does this change the practice of fairness for assessment? Without transparency, who gets caught and who gets punished? Who gets rewarded?
Emo-Guru	No more frowny faces or underachievers! Let AI be your emotional wizard!	EMOTIONAL AI With technology where we can develop emotional change campaigns, who gets access to these tools? Who decides when to use these tools and evaluates the outcomes?
Collaborator Coach	Teamwork just got a fancy upgrade with a coach on deck to referee squabbles, root for the team, and serve up top-tier tips!	AI-DIRECTED CONFLICT NAVIGATION Who designs, evaluates, and monitors tools we use for conflict? Who determines the values that they promote, the styles of conflict they tend towards, and the outcomes they facilitate?
Curiosity Catalyst	Are you tired of snooze-worthy teaching? Supercharge your students’ excitement and addiction to learning!	ADDICTIVE MECHANICS What techniques for habit forming behaviour could and should be used in learning situations? Who determines the appropriate techniques and the intended outcomes?

Full slides available on Canva.com