‘Next Big Fish?’ is an installation prototype that tries to connect the real with the virtual world by making the phrase “fishing for contacts” a tangible experience.
Inspired by the idea to create and design the connection between realities we wanted to find a new, playful way to do so. To demonstrate our way of working, we created a prototype for an interactive exhibit that uses Mixed Reality to blur the border between both worlds: By giving a haptic input, the user can interact with the virtual environment and get a physical output in the end.
Designing the exhibit prototype we could visualize our studies, play around with a possible solution, and test the experience on various users at Push Conference. As a use case that‘s suitable for this occasion, we chose to create a networking game, where visitors can literally fish for contacts.
Game Concept
Using Magic leap and its hand controller as a fishing rod, the user can catch contacts that are displayed by fish in a pond. To create a fish, they can enter their contact data at the log-in desk. Based on their profession, the fish type is varying: students, recruiters, managers, developers or creatives each get a fish that has an individual shape and color.
By wearing the Magic Leap glasses, the users can enter the pond environment and throw a line. If they have caught a fish, they can grab it with their hand, view the contact and decide, whether they want to keep it or not. If they want to save it, they can drop it into the bucket, so the contact gets physically printed in realtime and can be taken home afterward.
Technical explanation
The installationʼs hardware consists of several devices: the Magic Leap One glasses forms the heart of the development and use. A Lolin NodeMCU microcontroller is attached to the Magic Leap Controller in a self modeled and 3D printed housing. On a separate PC, we run the SQL Server, the back-end control software as well as the front-end software. Also, we connected the Epson PoS printer to it.
The software for Magic Leap One is running on the PC and is developed with the Lumin SDK in the Unity game engine. Additional to this software, there are further small applications. The back-end software allows to test and configure the OSC connection to the glasses, to install the database connection, and to take different printer configurations. The back-end control software, as the main junction between the applications, is coded in C#. It allows choosing different configurations for the OSC- and the database connection, as well as the printing output. Furthermore, the glassesʼ setup mode can be activated, which enables you to place objects like the pond, the waterfall and the bucket individually to create the virtual environment.
The front-end software that allows the visitor to transmit their contact data to the database is built in JavaScript. The software on the microcontroller reads out the connected rotary encoder and sends the direction of the rotation via WiFi to Magic Leap. It is written in Arduino IDE.
As soon as a visitor enters his data via the contact form, a new database entry is generated, and the front-end software sends a command to Magic Leap via OSC. When a visitor catches a fish, Magic Leap reads the ID and the data record and shows the information on the fish‘s info card. Once the visitor throws the fish into the bucket, the glasses send an OSC command, containing the fishʼs ID to the back-end software. This way, the back-end software can read out the correct data set and sends a print command to the printer.
Software: Unity 2019.2, Visual Studio 2017, Magic Leap Package Manager & Zero Iteration Services, Blender 2.8, Arduino IDE, OSC Data Monitor, Adobe Illustrator, Adobe Audition, Sketch, Microsoft SQL Server Management Studio
Programming languages: C#, JavaScript, C (Arduino)
KOMMENTARE