Doodling to the beat: Harmonizing technology and the Arts in the classroom
K-12 Education

Introduction: Beyond the binary
In an age where STEM dominates education headlines and AI is reshaping nearly every industry, the arts often find themselves pushed to the margins of innovation conversations. But what if we saw technology not as a competitor to creativity, but as its co-conspirator?
As an arts integration and educational technology specialist, I believe the future of learning lies in harmonizing what machines can analyze with what only humans can express. One compelling example of this harmony is the recently published research project DoodleTunes (Liu et al., 2024), which offers a vivid glimpse into how technology can illuminate the emotional and cognitive dimensions of children's artistic expression in the classroom.
Note: This research project is not to be confused with the 2016 interactive artwork of the same name by Danish creative coder Andreas Refsgaard. While both explore the relationship between drawing and music using AI, they are entirely distinct in their purpose, context, and implementation.
Music, Doodles, and Data: A new kind of canvas
Developed by researchers at East China Normal University, DoodleTunes is an interactive visual analysis platform that captures children's drawings created while listening to music and uses artificial intelligence to analyze their emotional and stylistic content.
More than 16,000 doodles were collected from young students as they responded to classical, romantic, and modernist musical pieces. The system tracked not only what they drew, but how the timing of their strokes, the colors they chose, and how those visual elements aligned with musical features like tempo and emotion.
This is where it gets fascinating: the AI could detect patterns linking a song’s rhythm to the sharpness of lines, or a melody’s emotional tone to the color palette of a child’s drawing.
This isn’t art for the sake of data. It’s data in service of deeper human understanding.
The symbiosis of technology and the arts
What DoodleTunes illuminates, and what many of us in arts integration already know, is that technology can amplify what the arts reveal.
Where the arts open emotional doors, technology provides the keys to walk through them. Where a student’s painting might hint at inner feelings, a well-designed AI system can help educators trace emotional patterns over time. Where a music and art lesson might once be dismissed as "nice to have," this research underscores its pedagogical power, especially when combined with meaningful data.
But here's the nuance: it’s not about quantifying creativity. It’s about qualifying it, giving educators tools to better understand, support, and nurture the creative process.
This is the harmony we need in education: the intuitive and the analytical, the expressive and the measurable, the heart and the algorithm.
Rethinking arts integration in a digital age
Let’s be honest, arts integration has often been siloed, episodic, or left to a single annual project. But with tools like DoodleTunes and the rise of creative AI platforms, we’re entering a new era, one where the arts are not just integrated, but interfaced.
Here’s what this means for the classroom:
🎵 Arts as inquiry, not just output
Instead of treating art and music as standalone outcomes, they become data-rich entry points into students’ cognitive and emotional development.
📊 Creative work as feedback
Doodles, paintings, and music compositions are not just artifacts; they are windows into engagement, affect, and imagination, elements that AI can now help us visualize and reflect on.
🧠 SEL meets AI
Social-emotional learning thrives in multimodal spaces. When students draw their feelings while listening to music, and educators can trace those changes over time, we are finally bridging emotional intelligence and technological insight.
From research to practice: What educators can do
You don't need to replicate a full AI lab to bring this philosophy to life. Here are a few starting points to blend arts and edtech meaningfully:
- 1- Use music as a creative prompt: Let students draw to instrumental music, then discuss what they felt, not just what they saw.
- 2- Capture the process, not just the product: Use simple tools (like time-lapse apps or digital drawing platforms) to record how students’ work evolves.
- 3- Visualize emotional growth: Ask students to revisit past work and reflect on how their emotions and artistic choices have changed.
- 4- Co-create with technology: Introduce creative AI tools (like generative art platforms or sentiment-based music apps) to show how tech can support, not replace, the human creative spirit.
A new kind of literacy
What DoodleTunes ultimately challenges us to consider is this: In a future defined by automation and analytics, how do we preserve and elevate what makes us human?
The answer lies in what I call creative data literacy, the ability to read, interpret, and design with both emotion and information in mind. As we introduce students to coding and computation, let’s also teach them how their drawings, songs, and stories carry meaning that even an algorithm can appreciate.
This is the next frontier of education. And it’s not about choosing between the arts and technology. It’s about creating an ecosystem where both thrive together.
Call to action: Let’s harmonize, not hierarchize
Educators, artists, technologists, we have a collective opportunity to reshape what learning can look like.
Let’s design classrooms where students don't just consume content but compose new meaning with colors, chords, code, and conversation. Let’s advocate for systems that value the emotional intelligence behind every brushstroke as much as the data visualization it can become.
And most of all, let’s remind our students that their creativity is not a soft skill it’s a superpower.
Because the future doesn’t just belong to those who can think. It belongs to those who can feel, imagine, and make meaning and to the educators bold enough to guide them there.
Reference
Liu, S., Bu, J., Ye, H., Chen, J., Jiang, S., Tao, M., Guo, L., Wang, C., & Li, C. (2024). DoodleTunes: Interactive visual analysis of music-inspired children doodles with automated feature annotation. Paper presented at the 1-19. https://doi.org/10.1145/3613904.3642346