top of page

Project Gestrua

Nov - Dec 2023

Team: Kyrie Yang, Alan Ren, Kefan Lyu, Muqing Wang

Team Website: https://gestura.super.site/

 

Gestura is a wearable device that revolutionizes the performing arts by empowering artists with cerebral palsy and limited speech abilities. Gestura represents a transformative approach to stage performance and real-time expression, transcending traditional boundaries of communication. It redefines the intersection of technology and art, extending its reach to artists across disciplines. As a gesture-based system, Gestura enables performers to manipulate the nuances of sounds and visuals through the simple movement of their fingers or arms. This innovative technology not only expands the realm of possibility for those with speech limitations but also offers a new dimension of expression for the artistically adventurous. Each movement becomes a brush stroke in a stage canvas, allowing the seamless integration of sound and visuals with physical expression. Gestura's user-centric design is the cornerstone of its inclusivity, inviting individuals to customize its functions to their unique expressive body language. It is a testament to the power of adaptive technology to not just overcome barriers but to create an entirely new lexicon of artistic vocabulary. With Gestura, the body speaks in tones, and movement resonates with the voice of creativity.

Technical Details
  • Hand-made leather glove and arm strap that fits most people.

  • Arduino Nano 33 IoT

    • Flex Sensors

    • Soft Potentiometer

    • Accelerometer

    • Gyroscope

    • Wifi

    • 9V Battery

  • TouchDesigner - Visual

  • Max/MSP/Jitte - Audio

Week 1

Screen Shot 2023-09-18 at 11.37.05 PM.png
Screen Shot 2023-09-18 at 11.36.56 PM.png

This week we collected all the components for circuits with Arduino, and learned the circuits. 

Week 2

This week we learned about microcontrollers. They are interesting as a connection between outside information and computers. We did some easy and small experiments on how to use an Arduino Nano 33 IOT, Digital Input & Output, Analog Input, and Sensor Charge Detection. 

This week's most challenging part is to understand the function of each 1) component, 2)code, and 3)circuit. There are a lot to remember. Copying the code from the website does not help me understand the mechanism behind it, which makes me feel anxious. 

Understanding Arduino Nano 33 IOT

Screen Shot 2023-10-02 at 5.39.28 PM.png

Understanding Coding language (Based on C++) (https://www.arduino.cc/reference/en/)

Programming & Playing with Digital Input / Output, (Pic 5 is debugging)

Programming & Playing with Analog Input. LEDs stay dim but light sometimes. 

conclusion: 

void setup: setting, run for one time

void loop: run for many times

input: button / order

output: sound / light / etc

Digital Input/Output: 

void setup: 

set pinMode (pin number, Input/Output)

void loop: 

if: one condition // else: the other condition

set digitalRead = pin input

set digitalWrite = pin output

pin HIGH + ; pin LOW -

Analog Input: 
const=#define

int:integer(整数?)

Serial: Used for communication between the Arduino board and a computer or other devices. 

Week 7

Asynchronous Serial Communications