Pcomp Week 7 (mid-term): Halloween Crystal Ball

Last modified date


Emily and I cooperated with the Graduate Musical Theatre Writing Program in making this project. Our goal is to provide an interactive device fulfilling their needs in the Halloween show. We came up with a crystal ball for the Fortune Teller room, in which an NPC would play Tarot card prediction while triggering the crystal ball to spoil clues for the guest.



The theme of the project was defined as “Halloween”. In our early brainstorming, Emily sketched some of her ideas about devices related to Halloween.

We discussed more but found that we do want real audiences to play with the physical device. Therefore, having attended an info session of the Graduate Musical Theatre Writing Program’s students and faculties, we decided to collaborate with the GMTWP Halloween show by providing them with technical support.

The show would be mainly about creating a 1950-styled town using multiple rooms at GMTWP department with wired things going on. There are some clues buried in each room, leading to a final story behind it. With a preference for combining graphical programming with physical computing and fabrication, we found a proposal in the wishlist:

Fortune Teller:Though unfailingly accurate, Twyp’s Resident Fortune Teller is notoriously disorganized and so there is no guarantee the fortune you’re told will be yours and not someone else’s. Sessions are liable to be interrupted by rude spirits or random visions, thanks to a crystal ball in the corner that is perpetually on the fritz.

Then we contacted the room owner, Wilson, who also acts as the NPC in the Fortune Teller room. After a short meeting with Wilson and Robert, the faculty of GMTWP and manager of the show, we finally figured out the goal and main requirements of the project.

Basically, the fortune teller room needs a controlled crystal ball, with images constantly changing and ‘floating’ inside. The basic technique to make images look like ‘floating’ is called “Pepper’s Ghost“, as Tom says.

It’s a plastic sheet set at 45 degrees in an acrylic globe, which sits on top of a screen. The contents of the screen are reflected on the sheet so that if you’re standing in front of the globe it looks like there is an image floating in it. —-Robert

Luckily enough, we borrowed the ball from Robert, together with a plastic board in it.

As for the contents, the room owner wants to have two main states: normal states and triggered states. In normal states, there should be 50’s-style TV clips playing in chaos. In triggered states, there should be some “alien calls” to disrupt the normal process and gave clues towards the whole story happening in the town.


Room Setting Conditions

Normally, the room could hold 2-3 people at a time. One of them would be the main guest, while the other one or two would stand at the door to watch. Wilson will do Tarot Card interpretation in front of the piano, upon which the crystal ball should lie. Between the main guest and Wilson, there would be a small table to hold Tarot Cards and electrical devices.


Task break-down

(1) A webpage showing on the iPad, with a “normal mode” and a “triggered mode”.
In the normal mode, the iPad would play some old-fashioned wired clips (maybe in B&W) for former TV shows or other sources. In the triggered mode, the iPad would start to play more abstract images about the clues of the city story. This part could be more like symbols, silhouettes, and icons. Emily worked on gathering and editing these assets.
To assure the information that we all want to convey would not be cut off from the ball, we may also work on HTML and use masks on the webpage to make the image as the same size as the bottom of the crystal ball (around 5.2 inches in diameter, as we measured). I was working on this.
(2)A tailed black cloth to cover the bottom part of the ball and a paper box serving as the basement.
Emily worked on laser-cutting the base part later.
(3)Using the finger hitting motion as a switch to trigger the “triggered mode” of the crystal ball.
We wanted to put a touch sensor beneath the tablecloth (with table fabrication to cover it and hide it) and code it so that when Wilson uses a finger to tap it according to some rules, the crystal ball behind him would be triggered and show the clues towards the myths. I also focused on this.
Soldering components
At first, without a working touch sensor switch, I tried to use a force-sensing resistor to achieve the trigger switch effect. Thus, after physical testing, I soldered a socket and attached it to a cardboard.
Coding part
(1) Arduino Side
I used analogRead() to read the FSR values and detect the finger hitting by setting a threshold of the values. Then, with serial communication and hand-shaking methods to reduce possible buffer overflows, I wrote the first version of Arduino program.
I set the “trigger cipher” to 2, which means that when Wilson hit the sensor for twice, it will change the triggerState to “1” and then reset the switchCount value to 0. Similarly, a “normal cipher” of 3 means that when he hit the sensor for another three times, it will change the triggerState to “0” and then reset the switchCount value to 0.
Pitfall: I suffered a little bit with the “hand-shaking” method. I put “checkButton()” function into the if statement of handshaking and found that it doesn’t work well. Later on I realized that maybe the handshaking should only be used to control the serial communication part. Any other things about Arduino itself and the logic on hardware part should not be included in that “if” statement (Otherwise it may not be processed, if there’s no signal coming in?).
(2) p5.js part
The first challenge I met was dealing with the “circle mask” of those images, and how to make the elements on the webpage automatically resizing themselves. In this way, when the page was loaded on an iPad, it should suit the size of the window. By calculating the diameter of our crystal ball, it could be made to suit the best.
The second challenge was to making multiple gifs constantly changing, like slides, with a fixed interval. I finally succeed in adding some loops and using the frameCount to monitor the time and interval. All of the images are loaded in the preLoad() function, but they are immediately hidden. Then, when the slider turned to a specific photo, it will be set as “visible”.
Pitfalls: I tried to use the “loadIMG()” function in p5.js to add gifs, but it doesn’t work because this method would only support still images. Then I turned to HTML img objects. However, it could be more tricky to make them auto-adjust the window size. It could be even harder to create a circle mask on HTML img elements. Finally, after 5-6 hours of trials and failures, I used CSS to successfully added the mask and used p5.js library to make them automatically resizing themselves.
The third challenge would be synchronizing a specific gif and a sound file. After our initial meeting, Emily created a piece of lovely animation for the “triggered mode” to uncover the clues of the women scientist’s death and the power of alien god. It was originally accompanied by a sound track. However, when I got the gif and the sound separately, I couldn’t find a best way to combine them. Especially, the gif is looping with the “hidden” mode in the background, which made it even harder to adjust the sound file.
I tried several ways, such as setting “restart()” function of the sound object with controlling the interval of playing. Nevertheless, it still won’t go well with the gif itself as the gif was not triggered at its beginning each time.
Finally, with analyzing the flow of the program, I decided to use the preLoad() function to load the sound and make it looping in the setUp() function, with the volume set to 0. When the sound should be playing, I set the volume back to normal. In this way, it could loop together with the gif in background.
Real testing and adjusting
We conducted a real test in GMTWP on Monday and Tuesday. The effect was quite good with the room light off.
However, Python HTTP Server seemed less stable and it will take a long while to refresh things every time we make changes on the code.
Besides, the FSR seems to be less stable than a touch sensor switch. The former one was using analog input while the latter one was using digital inputs with less threshold concerns. Thus, with the newly bought touch sensor switch, I soldered a new circuit and upgraded the physical setting:
It looks more hidden. Moreover, it is much more dependable.
Emily made a wood case with precise slots for the iPad and a hole at the top, allowing projections into the crystal ball. See her blog here about animation making, clip editing and fabrication.
Final Version


[lack of final video]

Room setting

[wating for tomorrow images]


(1) Arduino part

const int switchPin = 2;

boolean triggerState = false;

const int triggerCipher = 2; //touch twice to toggle the state to “triggered”
const int normalCipher = 3; //touch three times to toggle the state to “normal”

int switchCount = 0; //recording the number of times people touch the switch
int prevSwitchValue = 0;

void setup() {
pinMode(switchPin, INPUT);

// Starting the “hand-shaking”
while(Serial.available() <= 0){
Serial.println(“Waiting Data”);
delay(300); //wait 1/3 second

void loop() {
if (Serial.available()>0){
int inByte = Serial.read();
//send a new message to p5.js if it has been triggered

void checkButton(){
//read the touch sensor value
int switchValue = digitalRead(2);
//if the switch has changed,
if (switchValue != prevSwitchValue){
//debounce the switch;
//and that the switch is touched
if (switchValue == HIGH){
prevSwitchValue = switchValue;

void checkTriggerState(){
if (triggerState==false){
if (switchCount >=triggerCipher){
triggerState = true;
switchCount = 0; //reset the swithCount to 0.
if (triggerState==true){
if (switchCount >=normalCipher){
triggerState = false;
switchCount = 0; //reset the switchCount to 0.


(2) p5.js part

Holloween CrystalBall
P5.js side for Physical Computing mid-term
Made by Emily Lin and Chunhan Chen
Oct 22 2018
NOTE: You have to use “python -m SimpleHTTPServer” in the terminal.
Then open a tab and type “localhost:8000” for the address to get access to the music file.
If you are viewing from another device, use “ID address:8000” instead. Make sure the device share the same local internet with the laptop.
Also you should keep p5.serialcontrol application open while running the program.
Reference: https://codepen.io/dclappert/pen/mJeYye?editors=1100
const normalJpegNum = 0;
const normalGifNum = 5;
const triggeredJpegNum = 0;
const triggeredGifNum = 1;
var normalAsset = [];
var triggeredAsset = [];
var triggeredSound;
var normalInterval = 10; // Interval to switch images of normal conditions
var triggeredInterval = 7.2; // Interval to switch images of triggered conditions
var triggerState = 0; //KEY VALUE! To record the triggered (1) or normal (0) state.
var prevTriggerState = 0;
var triggerFrame; // to record the frame that triggering happens
var nIndex = 0;
var tIndex = 0;
var serial; // instance of the serial port library.
var portName;
function preload() {
if (normalJpegNum>0) {
for (leti=1; i<=normalJpegNum; i++) {
img.position(windowWidth/2-0.415*windowWidth, windowHeight/2-0.415*windowWidth);
img.size(0.83*windowWidth, 0.83*windowWidth);
img.style(“visibility”, “hidden”);
if (normalGifNum>0) {
for (leti=1; i<=normalGifNum; i++) {
img.position(windowWidth/2-0.415*windowWidth, windowHeight/2-0.415*windowWidth);
img.size(0.83*windowWidth, 0.83*windowWidth);
img.style(“visibility”, “hidden”);
if (triggeredJpegNum>0) {
for (leti=1; i<=triggeredJpegNum; i++) {
img.position(windowWidth/2-0.415*windowWidth, windowHeight/2-0.415*windowWidth);
img.size(0.83*windowWidth, 0.83*windowWidth);
img.style(“visibility”, “hidden”);
if (triggeredGifNum>0) {
for (leti=1; i<=triggeredGifNum; i++) {
img.position(windowWidth/2-0.415*windowWidth, windowHeight/2-0.415*windowWidth);
img.size(0.83*windowWidth, 0.83*windowWidth);
img.style(“visibility”, “hidden”);
//mySound = new sound(‘triggered_state_assets/triggered_1.mp3’);
function setup() {
serial.on(‘list’, printList); // ‘list’ is an event, printList is a call back function
serial.on(‘connected’, serverConnected);
serial.on(‘open’, portOpen);
serial.on(‘data’, serialEvent);
serial.on(‘error’, serialError);
serial.on(‘close’, portClose);
function windowResized() {
//To automatically adjust the size of images, based on physical mesasurements of the crystal globe and the iPad pro screen.
for (letthisImgofnormalAsset){
thisImg.size(0.83*windowWidth, 0.83*windowWidth);
thisImg.position(windowWidth/2-0.415*windowWidth, windowHeight/2-0.415*windowWidth);
for (letthisImgoftriggeredAsset){
thisImg.size(0.83*windowWidth, 0.83*windowWidth);
thisImg.position(windowWidth/2-0.415*windowWidth, windowHeight/2-0.415*windowWidth);
function draw() {
if (triggerState==0){
triggeredSound.amp(0); // set the volume of the sound to 0.
for (thisImgoftriggeredAsset){
thisImg.style(“visibility”, “hidden”); // hide all of the triggered state images
if (nIndex>=normalAsset.length){
normalAsset[nIndex-1].style(“visibility”, “hidden”);
normalAsset[nIndex].style(“visibility”, “visible”);
if (frameCount%floor(nIntervalCount) ==0){
normalAsset[nIndex].style(“visibility”, “hidden”);
if (triggerState==1){
for (thisImgofnormalAsset){
thisImg.style(“visibility”, “hidden”);// hide all of the normal state images
if((frameCount-triggerFrame) %round(tIntervalCount) ==0){
triggeredSound.amp(1);// set the volume of the sound to normal.
// triggeredSound.playMode(‘restart’);
// triggeredSound.play();
triggeredAsset[0].style(“visibility”, “visible”);
//Below are serial communication parts.
function printList(portList) {
// portList is an array of serial port names
for (vari=0; i<portList.length; i++) {
// Display the list the console:
console.log(i+” “+portList[i]);
//automatic choose port as Arduino port:
if (portList[i].indexOf(‘usbmodem’) >=0) {
console.log(‘–Using ‘+portName+’ as serial port, probs Arduino’);
serial.open(portName, {
function serverConnected() {
console.log(‘connected to server.’);
function portOpen() {
console.log(‘the serial port opened.’)
function serialEvent() {
if (inString.length>0) {
if (inString!=”Waiting Data”) {
function serialError(err) {
console.log(‘Something went wrong with the serial port. ‘+err);
function portClose() {
console.log(‘The serial port closed.’);
(3) CSS part
html, body {
margin: 0;
padding: 0;
background-color: black;
canvas {
display: block;
Questions and prospects
  1. How to control the exact start point of a gif loop in javascript? In otherwise, how to make a gif and a sound file start at the same time?
  2. The Python Simple HTTP Server seemed to generate some time lag between triggering in Arduino and receiving while displaying on the iPad screen. Is there a better way to “push” the contents on a PC screen to another device, like iPads, more easily and faster?
  3.  We are only sending 0 or 1 through serial communication, but I used strings (Serial.println() in Arduino) to convey messages. It could slow down the communication a little bit. Maybe we’ll improve that later in process.


Leave a Reply

Your email address will not be published. Required fields are marked *

Post comment