Keyboard Hero

9 min readAug 26, 2021

Why Keyboard Hero

Learning an instrument is a very valuable skill that offers a good amount of benefits along the way. It will help increase your cognitive abilities, improve memory, reduce stress, and could even lead to a wider friend group. Learning an instrument is beneficial, but it’s not an opportunity that’s available to everyone, for that reason we decided to work on our project, Keyboard Hero. Keyboard Hero is an online collaborative music player designed to be enjoyed by one or multiple users. It offers a range of musical instruments, multiple key configurations, and is a great way for beginners to learn. The inspiration for Keyboard Hero came from Google’s own Shared-Piano which allows users to join online rooms with their friends and use a collaborative keyboard to make music.

Technologies

Keyboard Hero has many functionalities occurring above and below the hood. The application allows users to either create or join rooms so that they can collaborate on one piano. Users choose a color or have a random color assigned to them. This color will be used throughout their experience and will serve as a highlight color when they interact with the keyboard and collaborate with others. Once in a room, a hashed key code will be provided which will serve as a password for their current room that can be shared with others. Along with the online keyboard being presented, you will be given its configuration options. These are for choosing what instrument you are playing, how many octaves are being shown, which octave your keyboard is mapped to, and a volume slider. We created this experience to be completely customizable for the user to have complete control and allow for a more natural musical experience.

Implementation

Tone.js

A key part of this project is the ability to play sound from the browser which is achieved with Tone.js. Tone.js is a Web Audio framework for creating interactive music in the browser. This framework makes working with the Web Audio API simpler because it handles most of the tedious inner workings that come along with it. It makes loading the samples of each instrument simpler with the utilization of a sampler tool and an onload callback function which gets called when the samples are loaded. We utilized this callback to give users the ability to change the volume of the instrument being played by using the volume method on the sampler. Using this with the loaded method improved user experience by providing a check for when the samples are fully loaded. The loaded method prevents the app from crashing when the samples are not done loading.

Socket.IO

One of our main focuses of this project was the collaboration between users — without this, it would be just a normal keyboard application. Since we needed a way to sync multiple user interactions within a single room, we decided to utilize Socket.IO. Socket.IO is a library that enables real-time, bidirectional, and event-based communication between the browser and the server. We created a server that listens for events that occur in the browser, so when the server receives these, it uses the data sent from the event to either update necessary information or perform actions on the other clients’ browser. This all happens very fast so there is little to no lag between events, which is essential from a user’s point of view.

Initializing a Socket

When a client first connects, we assign them a socket that connects them to our server. This will allow the server to listen to any event triggers that their socket sends. Each socket has a unique ID assigned to them, which makes it easier to identify each user. We utilized this socket ID in our data objects to keep tabs on different users and their assigned rooms.

Creating a Room

Once a user’s socket is connected, they can either create a room or join an existing one— the process is similar but they do have some minor differences which we will discuss later.

User objects consist of a “name”, “color”, and “room” code. The “name” and “color” are gathered from user inputs while the “room” code is generated from a hash function to ensure all the room codes are different.

We store these values in local storage and route the user to the Piano page, where a “join” event is waiting to be emitted. This event sends the user object to the server so we can keep track of it in our users array. The server sends back a filtered version of the users array with all the users that have the same room code. This will be mapped over to create the “User List” on the UI.

Upon initial creation of a room, only the user that created the room will be listed. Along with displaying the “User List”, we are also displaying the room code that’s held in local storage.

Joining a Room

In the event there is a room already made that a user wants to join, we follow almost the same process but those differences I mentioned before come up now.

This time we are checking if the room code a user enters exists and that their name isn’t already associated with said room code. We added this check to ensure there wouldn’t be more than one user with the same name in the same room. Once these checks are passed and the user chooses their color then they will go through the same process as before. They get sent to the piano page with their values inside local storage, they hit our “join” event, they get added to our users array, and the server will send back the filtered users list. This time there would be the user and anyone else who was already in the room before the time of joining. Since this is the same emit event as before it will also send out the updated and filtered array to all others in the same room.

The playing of notes

For the playing of notes, we’re also utilizing socket events. Every user should see and hear all notes that are played by users in their specific room.

Whenever a key on the piano is clicked it triggers a function that creates an object that holds the note, the sound, and the buttons class.

Once this object is created, it is sent to the server. The server will know what socket it came from because it can look up the socket ID in our user array and give us back the corresponding user object. Having this user object allows us to know what user sent it and what room they’re in.

We can then send another event from the server to every user in that room which triggers the playing of the sound on their browser. It might help to think of this process as a relay race— when a key gets clicked, we send a cue from the browser to the server which then sends a cue to all correct users which trigger the sound on their side.

Leaving a Room

The process of leaving a room is the opposite of joining a room. When a user clicks the leave room button, we send them back to the home page and also trigger the “leave room” event.

The server hears this and will once again use the ID of the socket to find the corresponding user object in our array. Once we know where it is, we simply splice it out. Then the server sends the updated user list back to everyone in the room, which updates their “User list”.

The only scenario where the user won’t immediately be removed is if their socket disconnects. This could happen by the user closing the browser before leaving their room. In this instance, we listen for a “disconnect” event. We try to find the user by looking them up by their ID in the user array. If they’re found then we go through the same process of removing them.

Options

Shared piano includes many different features like changing instruments, adjusting the volume, changing the displayed octave, and switching which octave the user wants the keyboard to map to. A dropdown state is used to control which instrument sample is loaded to the sampler and mapped to the keys on the keyboard. Rendering each key by octave, we can map the keyboard to each index in an octave array. When the dropdown display is changed, the state that controls how many octaves are rendered is changed, altering the number of octaves the user can see/has access to.

Challenges

Scalability

An issue that could arise over time is the scaling of our users. At the time of writing this, we used an array to keep track of all the users. As users are created, we add their data into this array, and when they disconnect, we remove them. The issue that occurs is the more users that are created, the more the array grows, which increases the likelihood of the app slowing down. A way around this would be to implement a database to hold and keep track of users. A database is great because it holds more data more efficiently, keeps data consistent and accurate, and allows faster access to the data.

Deployment Procedure

Shared-Piano was deployed on Heroku using the Heroku-CLI. The server and react application were deployed separately. The server was deployed as a backend to Heroku; the process was similar to the front-end. We used the Heroku-CLI to deploy the node server. Deploying the react front-end was tricky due to the use of a proxy to make the Socket.IO connection. We used an npm package called http-proxy-middleware to redirect every request for /socket.io to the backend that was deployed on Heroku.

Future Features

Chat

In the future, we would like to add a chat system. Since we’re already using Socket.IO for communication between browsers, adding in a chat should not be difficult. It would be the same flow as when we were sending played notes between relevant users. We’d make message objects just like our user objects to hold message data like user and text, we would send every user in a room the message, and we could even have a private message where only a specific user would see a message. To make this persist, we store all sent message objects in an array and filter them by their room value, and send them to the corresponding room.

Keyboard Hero is a fun and interactive application that could encourage people to learn a new skill, teach others new skills, or just hang out with each other. We hope you enjoy it as much as we did making it. Have fun!