Course Information

CS446 - Computer Graphics
Dr. Jesse Johnson
Final Project
December 2006

Project Overview

Musaic is an ongoing experiment with dynamic Internet based art. The application downloads and processes the lyrics for a song and extracts the most "interesting" keywords. These keywords are used to search for and download relevant images from sites such as Flickr or Google Image Search.

While the songs plays, the images are displayed in a three dimensional visualization. The images and lighting react to the intensity of the music. The program randomly selects music files from the user's library to visualize. Certain songs return more relevant images than others, such as songs with vivid descriptions. A good example is "Yellow Submarine" by the Beatles. This visualization displays images of seascapes, psychedelic imagery, and even a toy yellow submarine.

Features

• Displays a visualization of images based on a song's lyrics
• Plays and interacts with audio (MP3) files
• Images are deformed by the amplitude of the audio
• Simple beat detection controls lighting effects
• Allows users to move around the space with keyboard input
• Camera system tracks moving images and keeps them in frame

Technologies Used

• Written in C# and J# using the .NET environment
• OpenGL for graphics
• CsGL library, an OpenGL bridge for C#
• Bass.net framework for audio integration
• Stores lyric and image information in XML format
• Downloads and parses HTML from various Internet sites
• Makes heavy use of texturing, lighting, and display lists for graphics
• Uses multi-threaded code to perform background downloading

Reflection

This project was originally implemented in early 2006 as a web application written with Java and a Flash-based front end. I found that the Flash interface was too slow and clunky with large images to be an effective display. I realized during my course in computer graphics that I had gained the neccessary tools to create a very sleek interface for my old project. Luckily, I was able to reuse all of the Java code I had written by converting it to J#.

This was the most rewarding and fun project that I have worked on during my time at the university. I've been interested in aggregation of information since I started working on the University's web portal. I found that this application required me to use a lot of my artistic creativity to create a display that was visually pleasing and interesting to watch. At the same time the application was a huge programming challenge, especially dealing with the math and algorithms that went into the image deformation and lighting effects.

I have continued working on this project after I turned it in for credit. Currently, I am in the stages of redesigning a new interface - one that is less random and more "shiny" - and I hope to soon release the source and code eventually so that others can contribute to the project.