• Your presentation
  Online Link:
  https://docs.google.com/presentation/d/1hC8NEC84CYO7_m8aihOJW1nSprMXHYKTWJMagTQmW_o/edit?usp=sharing or
  PDF link:
  Pcomp Twinkle Stare_2
  
• Concept + Goals.
  My concept was to create an IoT device that reconnects me in New York and my dog in Taiwan to be able to feel her presence and recreate a moment we share together. My goal was to be able to feel my dog’s presence over long distance and to spend more time with her somehow. This project is very close to me and I decided to an long distance device because of my dog who is actually sick with cancer back in Taiwan. So I want to build something that will actually work for us.
  
• Intended audience.
  This project is mainly for myself so I can spend more time with my dog with her presence over long distance. However, I feel like this device for be for other dog owners who are in long-distance relationship with their dog and would like more presence of their dog.
• Precedents.
  Pillow Talk is a device that lets you hear the real time heartbeat of your loved one over long distance by Little Riot, which really inspired having the presence sense in my device. SoftBank from Japan also created a series of devices called called Personal Innovation Act, Analog Innovation that helps connect the older generation to the younger generation by translating the new technology we use into older forms such as printing your social media updates in the mailbox as newspapers every morning so your grandma can read updates about you.
• Description of the project.
  After confirming my concept, I begin to think about the technical aspects of my project. My device’s interaction has two parts with different interactions. One part will be my dog’s end with her doll embedded with a pressure sensor on which she will lay on it and a speaker. The other part is the model of my dog in my room in New York which will have a face tracking camera, a button, and a pressure sensor built into it. As I began searching online for the details of how to make these interactions work, I was also introduced to two tools that could help with the long distance IoT connection to work. One tool is the MESH sensors which consists of 7 block sensors which each has a built-in function such as tilt, led, button, motion, and more functions to make prototyping and building project easy for the Internet of Things. The other is the IFTTT the free web-based service to create chains of simple conditional statements, called applets, to connect with different applications and services over the internet. Both tools are extremely helpful for my IoT device, however, I want to make all my interactions work properly first offline. First of all, I decided to figure out how to get the face detection to work on camera. I used an Arduino controlling to a servo motor and connecting the servo motor to the OpenCV face detection on a program called Processing to track faces when it moves from left to right on the screen. Thankfully, I spent a few days studying the open source code example online and made face tracking work with no problem. Next was to get the pressure sensor connected to a trigger to open the face tracking camera to start tracking face detected when it is pressed. This was a part where I was stumped and frustrated because I could not get the code to work with my own ability. With some help from my peers, I was able to get the pressure sensors to work to open up the camera to start face tracking when it is pressed and another pressure sensor to turn in off. However, there was another one problem I encountered with my code which was that it can only be run once. If I want to try to press the pressure sensor to turn it on again, the sensor cannot read my pressures values anymore. I would have to rerun both the Arduino and Processing sketch for it to work again. After some debugging, I discovered that it was part of the Arduino code that made the instructions appeared to be stuck in a loop. After modifying my code, my code was working fine, though, at times unstable, I decided that with the time I had, I would not make the IoT connection happen but just focus on the interactions and the physical doll of my device. My whole framework to make this IoT device work is displayed in the diagram below. What I’m focusing on is the interactions on the left. Ideally, I would incorporate this with the MESH sensors and by using the IFTTT webhooks service to send to communicate with my MESH sketch.
  
• Video documentation
  

  
  

• Materials list
  

  For the face tracking part: 

  Software Required

  • Arduino
  • Processing
  • OpenCV Framework (Windows, Mac, Linux)
  • OpenCV Processing Library

  Firmware Required

  • OpenCV Examples
  • SerialServoControl Sketch
  • Face Tracking Processing App

  Hardware Required

  • Arduino Uno (or other 5V Arduino Compatible board)
  • Pan/Tilt Servo Bracket
  • Webcam
  • USB Cable
  • 9V DC Power Adapter for Arduino
  • Breadboard
  • Jumper Wires
  • Male Header Pins (2x 3 pin lengths)

  For the physical making of the doll –

  • Fluffy Socks – http://amzn.to/1FwCmTk, http://amzn.to/1NWUkhD
  • Polyester stuffing – http://amzn.to/1Ke62Wy
  • Sewing Kit – Amazon
  • Process + Prototypes.
    After making the technical part of my device work, I decided to quickly move on to the physical enclosure. The physical enclosure of my device consists of two parts. One is the doll that my dog will lay on and the other is the model doll that sits in front of my desk. I completely underestimated how hard it is to put anything physical together, even if it is something cute and furry such as a doll. Somehow, the thought of making a doll to me seems simple to me. Not until I started a few tries with making a doll did I realize that I have a lot of practice to do. I thought of taking apart an actual doll but I also wanted my doll to be customizable so I decided to make one myself. To start off, I used fluffy socks as my main material and stuffed it with polyester fiberfill. I made a koala doll as the doll that my dog lays on. This was easier to make as it is such a small doll with no body. Then, I moved on making the model doll of my dog. With my limited experience, it was hard to me to make a model of the dog that looks exactly like my dog. One of my first versions of my doll was unable to stand up properly so I had to make a stand stuck onto an acrylic board and put it inside the doll so the model doll could stand by itself by itself. I then glued the servo onto the board and cover it with polyester fiberfill and put the sock fabric for the model’s head over it to make the model’s head. After getting the body to sit properly on a flat surface, I moved onto making the head. I put a web camera into the head and cut a small hole for the web camera to be able to peak out through the sock fabric. The web camera, however, did not work well hiding inside. One problem is that the web camera is very sensitive to the lighting, the distance, and the height of where you stand. When testing with the web camera inside the head of the doll model, the camera had a hard time detecting face and would jump from different shadows in the screen which causes spasm of quick movements. Another factor that was contributing to this unstable web camera screen was the fur from the sock material I used. This seemed to disrupt the clarity of the screen with a few furs sticking out along the side of the hole I cut. Therefore, I made a hard decision to connect my device to my computer’s camera to ensure the most stable and accurate face tracking. In the end, I was able to put together a functional model doll of my dog. The face tracking camera can be triggered by another doll when you press on it (when my dog lays on it) and you can turn it off by pressing on the model’s head. You can also press a button which is using one of the MESH button sensors that will turn on music which will play in the speaker inside the doll.Prototypes-
         
    Playtesting:
    
    Challenges:
    • Pressure sensor did not work as will as I thought
    • Complexity of the code
    • Webcam did not work well behind my fabric + distance issues
    • Time Management
    • Aesthetics doll’s with many wires sticking outFuture Iterations:
    • Refinement on the design/look of the doll with no wires with webcam built inside the doll
    • Making it work over local distance and wireless with bluetooth
      
• Circuit diagram.
  

Continue reading →