The Dynamic Word Cloud
Prototyping this idea seemed to be an interesting internship side project to work on whenever I had the time.
How it Works
The resulting text is separated into individual words. Those words are then sifted through to sort out conjunctions, interjections, pronouns and essentially every other word that isn’t carrying any significant meaning on it’s own or cannot convey meaning due to the lack of context in the randomized visualization it is going to end up in. Usually speakers tend to use words they’d like to emphasize on more frequently. The same thing applies to essential information like the name of the topic for example. This is why the the occurrences of each remaining word are counted in the next step. The resulting occurrence-count is then mapped onto the size and color of the respective word. Finally all the words are randomly placed on the available surface to achieve a cloud-like visualization.
Technical Setup
First I needed to find a sufficiently functional speech to text converter that I could run in the browser. After attempting to utilize many different Chrome browser extension, trying to use the built in speech recognition of Windows and among other things installing Amazons Alexa on a Raspberry Pi amounted to pretty much nothing I ultimately ended up setting up a Node.js server running some JavaScript to access Googles Web Speech API.

With this solution I could continuously put in audio via microphone and get a clean string back. Node.js in combination with Socket.IO also allowed me to effortlessly pass on the results to vvvv via OSC.
For most of the programming the existing vvvv nodes where sufficient but I wasn’t able to find nodes to prevent the overlapping of words which would inevitably occur if iI just spread them out randomly. Overlapping greatly diminished the readability so I also had to write a custom node for vvvv in C# to prevent it from happening.

Examples
Here are additional examples for word clouds generated from some very different speeches, talks and presentations.
President Donald Trump Speech at CPAC 2017, Friday, February 24, 2017
Apple WWDC Keynote 2017
Elon Musk talks Climate Change and Carbon Tax at the Sorbone 2015
Martin Luther King Jr. “I Have a Dream”, 28 August 1963, Lincoln Memorial, Washington D.C.
With the guidance of my patient colleagues I went through the process of generating an idea and developing it into a functional prototype. This internship project was a valuable experience for me and and the advancement of my humble programming skills along the way certainly didn’t hurt either.
vf
KOMMENTARE