Do Re Mi
For my p5 sketch I chose to work with poseNet and create a simple piano keyboard to play music using your right hand. I first started exploring poseNet by making a sketch where you can draw with you body (creating visual art through dance). The sketch drew from your left and right wrists and ankles key points. That sketch lagged terribly. My laptop is kind of on the slow end so I expected it but it was hard for me to try out the sketch at all so I had to change my idea.
I made another sketch where you can only draw using your nose, and that worked much much faster but was definitely less exciting so I decided to play around with music.
I built on a keyboard piano sketch that I found online. The draw key points function records the position of my right hand when there is high accuracy and if that position is a piano key the notes are played using p5.js Oscillator constructor.
As the world’s least musically inclined person, I wasn’t really able to play anything nice but I am sure someone out there can :)
I struggled a lot getting the sound to stop after I remove my hand. It ended up sounding like a really long glitch due to how fast the confidence score changes.
What questions do you still have about the model and the associated data? Are there elements you would propose including in the biography?
- How does this model fare with bodies that are differently abled?
- I was recently reading an article on how some government use machine learning to watch who is acting suspiciously in certain areas known for public gathering, successfully predicting and squashing protests and any form of dissent before happening. What sort of things are in place to make sure this doesn’t make the world a more dystopian place?
- Were the data subjects asked for consent?
- Where there people chosen to ensure the diversity of the data set?
How does understanding the provenance of the model and its data inform your creative process?
I would say understanding what the model is capable of helps guide my creative process. More importantly, understanding the ethical implications of the model informs whether I should be creating in the first place.
Resources:
- https://editor.p5js.org/monniqian/sketches/TwckaaeGl , piano sketch
