Emily Lin

Xiaotong and I worked together on this project! The goal was to create a collaborative live drawing app using ITP’s Axi-Draw Machine and RESTful APIs.

The functions we aimed for: 

• Co-Draw allows people to contribute to a drawing using a web interface
• Control the Axidraw’s position based on the user’s mouse x and y position on the website.
• Use socket.io to allow many people to draw at the same time.

When coding this project, we came across many challenges and we ended up with 2 versions:

• version 1: tweaked example code for the Axi draw and added sockets to it (as backup)
• version 2 (final): uses our own RESTful API that communicates from client → server → serially communicates to machine

Final code without sockets

Final Version

This version uses our own RESTful API, which was done with a lot of help from Professor Tom Igoe. We were able to get as far as sending commands from the client side –> server side –> (communicate serially) –> to axi draw machine.

Systems Diagram

Server side: 

a.) GET and POST endpoints

// here are all your endpoints. The pattern is:
// GET the current value, or
// POST the new value as a request param:

server.get('/mouse', getMouseState);

server.get('/command/:command', runRemoteCommand);

server.post('/mouse/:mouse_x/:mouse_y', moveMouse);

server.post('/mouse/:mouse_y', handlePostRequest);

server.get('/state', handleGetRequest);

server.post('/state/:state', handlePostRequest);

b.) moveMouse() function that gets request from client and puts into a string to send to Serial

function moveMouse(request, response) {

// request is /mouse/:mouse_x/:mouse_y'
// get the position from the request
//SM,1000,-250,766\r

let result='SM,'+'1000,'+request.params.mouse_x+","+request.params.mouse_y;

// send it to the serial port as a command
sendSerialData(result);

// wait for confirmation from the serial port
// send a response back to the user

response.send(result); //send for string

}

c.) Send command from mouseMove() function to Serial 

function sendSerialData(command) {

   console.log(command+'\r');

   myPort.write(command+'\r');

   console.log("Sending something out the serial port");

}

—

Client Side

a.) setMouse() sends POST request to server using httpDo function of p5js

function setMouse(x, y) {

    var path = '/mouse/' + x + '/' + y; // assemble the full URL

    var content = '';

    console.log('path: ' + path);

    //httpDo( path, 'GET', content, 'text', getResponse); //HTTP PUT the change

    httpDo(path, 'POST', content, 'text', responseHandler); //HTTP PUT the change

}

b.) setMouse() function is called on in index

<!--top left-->

<buttononclick="setMouse('-550', '0'); moveTopLeft()">top left</button>

<!--top-->

<buttononclick="setMouse('-1000', '550'); moveTop()">top</button>

<!--top right-->

<buttononclick="setMouse('0', '550'); moveTopRight()">top right</button>

<!--bottom left-->

<buttononclick="setMouse('0', '-550'); moveButtomLeft()">bottom left</button>

<!--bottom-->

<buttononclick="setMouse('550', '-550'); moveButtom()">bottom</button>

<!--bottom right-->

<buttononclick="setMouse('1000', '500'); moveButtomRight()">bottom right</button>

// Function for sending to the socket

function sendmouse(xpos, ypos) {

// We are sending!

console.log("sendmouse: "+xpos+" "+ypos);

// Make a little object with and y

vardata= {

x:xpos,

y:ypos

};

// Send that object to the socket

socket.emit('mouse', data);

}

b.) Socket code on server side for listening for data, then emitting

// When this user emits, client side: socket.emit('otherevent',some data);

socket.on('mouse',

function (data) {

// Data comes in as whatever was sent, including objects

console.log("Received: 'mouse' "+data.x+" "+data.y);

// Send it to all other clients

socket.broadcast.emit('mouse', data);

// This is a way to send to everyone including sender

// io.sockets.emit('message', "this goes to everyone");

}

);

Version 2 (sockets added to AxiDraw Example Code)

For this version, we used the cncserver code as our example: https://github.com/techninja/cncserver

Run cncserver.js

install Node.js

install npm

node cncserver.js

References we used: 

• SerialOut: https://github.com/tigoe/NodeExamples/blob/master/SerialIntro/serialOut.js
  • Set up serial port
  • Readline parser
  • myPort
• httpDo (for p5): https://p5js.org/reference/#/p5/httpDo

• EBB Command Set
  • EBB command documentation about using serial communication APIs: http://evil-mad.github.io/EggBot/ebb.html#SM
  • To use serial in terminal: https://www.youtube.com/watch?v=vpZDfii5n3A
• RESTful API
  •  https://github.com/tigoe/NodeExamples/blob/master/RestTest/server.js
  • We tried to get the mouse position
  • Go to the IP address
  • Enter the endpoint (mouse/mouse_x)
  • Get the data (20, 50)
• AxiDraw app code
  • https://github.com/techninja/cncserver
• Node with p5js Coding Tutorials + Source code
  • https://www.youtube.com/watch?v=bjULmG8fqc8

Process:

a.) Our backup plan of using the AxiDraw code and just adding sockets to it (when our own code wasn’t working.)

b.) Practicing sockets to disconnect a user after 30 seconds. 

c.) inital start: simply communicating to Axi Draw using serialPort

So Many Thanks:

We bugged way too many people and professors for this project. I’m so grateful to everyone who took the time to help and listen. The biggest thank you to Professor Tom Igoe for being patient and spending so much time to help us understand how to write RESTful APIs and how to communicate with the AxiDraw using serial.

Thank you so much to Professor Shawn for providing guidance and troubleshooting help. Thank you Professor Mimi for helping us last minute with the socket pairing.

Thank you Dana and Noah for asking critical questions and giving good feedback during the start/ideation of this project. Thank you Jackie and Andrew for guiding us through the logic.

Live Drawing Show!

Goal:

•  User will draw on the website’s draw page. The Axi Draw will update based on user’s drawing. There will be a camera that is recording. The recording will be sent to the website’s live stream page so others can see the live stream of the drawing. 
  
  

—

Functionality:

• Co-Draw allows people to contribute to a drawing using a web interface
• Control the Axidraw’s position based on the user’s mouse x and y position on the website.
• Include a camera that video records the Live Drawing show. The live stream will be posted on a separate page on the website.
• Ideally: use socket.io to allow 2 people to draw at the same time.

—

REST API:

1.) Identify Axi Draw Machine

• Address: /identify_axi
• Method: ?

2. ) Find current position of the user’s mouse x and mouse y on the website

Address: /pen_position

Method: post

Body:

{
   “mouse_x”: 2344,
   “mouse_y”: 281,
   “state” : 1  //pen state is from 0 to 1 (down/on)
}

Response:

“coordinates”: [
        {
        “success”: 
            {
             “type”: “200”, //ok
             “mouse_x”: “2344”,
             “mouse_y”: “281”,
             “state”: “1”
            },
         “error”: 
             {
               “type”: “400”, //error
               “details”: “invalid inputs”
             }
         }
     ]
}

3. ) Set to original position when user presses Reset Button

Address: /original
Method: POST
Body: 

{
   “mouse_x”: 0,
   “mouse_y”: 0,
   “state” : 0 // off
}

Response: 

{
    “original_position”:[
        “success”: 
        {
            “mouse_x”: 0,
            “mouse_y”: 0,
            “state” : 0 // off
         },

   “error”: 
        {
             “type”: “400”,
             “details”: “invalid inputs”
         }
      ] 
}

—

Video for live stream

4.) Get channel 

Address: /channel
Method: GET
Body: 

Response: 

{
    “channel”:[
        “success”: 
        {
            “recording”: on, //turn channel on or off
            “channel_id”: {channel_id}, //integer
            “width": 1920px,
            "height": 1080px
         },

   “error”: 
        {
             “type”: “400”,
             “details”: “invalid inputs”
         }
      ] 
}

5.) Post channel data to client

Address: /channel
Method: POST

{
 “channel”:
      “recording”: on, //turn channel on or off
      “channel_id”: {channel_id}, //integer
      “width": 1920px,
      "height": 1080px
}

—

Reference:

• https://www.dacast.com/video-api-documentation/
• https://developers.meethue.com/
• https://github.com/techninja/cncserver

Assignment:
“Run Linux host for several days, with a firewall in place and a public IP address. Make a table of all the IP addresses that attempt to connect to your host. Found out where they are located? What organizations are the associated with? What service providers are providing their IP address?”

a.) Step 1: Using my Digital Ocean Linux Host, I installed and configured the ufw firewall. I enabled the TCP connections on  the ports below. I did this by following this super helpful tutorial from the ITP Network Site.

—

b.) Step 2: I collected a table of all the IP addresses that tried to connect to my host for Nov. 2, Nov.3, and Nov. 4. I was planning to combine all 3 day’s worth of data into 1 set, but that was just too large and made it hard to manage in Excel.

I decided to just focus on the Nov. 4 data, which was more than enough data points to work with (5,640 unique hits).  The below image shows a section of the full dataset.

—

c.) Step 3: Using Excel I was able to find: total hits from all the sources, number of unique sources, top 10 most frequent src (including their ip addresses). The image below shows the results for this.

Other Data (Nov 4): 

• Total Hits from All the Sources: 5,640
• Number of Unique Sources: 1,111

—

d.) Step 4: Using ipinfo.io, I tried to find where each of these 10 ip addresses are coming from and the organizations they are associated with.

• #1 hits: 185.156.73.52
  • Moscow, RU
  • OOO Patent-Media

• #2 hits: 185.176.27.254
  • Moscow, RU
  • Balkan Internet Exchange Ltd

• #3 hits: 185.156.73.52
  • Moscow, RU
  • Balkan Internet Exchange Ltd

• #4 hits: 80.82.64.73
  • Amsterdam, NL
  • Incrediserve.net

* Side note: Incrediserve.net is an IP Trading company! This is a dumb question, but do they mean ip addresses or ip as in intellectual property? 

• #5 hits: 96.250.119.232
  • New York City, NY
  • MCI Communications Services, Inc. Verizon Business

—

e.) Step 5: Reflection.

UM, why is someone from Russia constantly hitting my IP address?! And so many time in 1 day! Is this maybe tied with the hacking of our elections? Tried to look into this a little and found this forum. Seems like some people on the forum say it is “normal” to get probes from many countries like Russia, China, Vietnam, etc. Basically, I shouldn’t let my biases get in the way, even though my top 3 places that hit me are from Russia.

—

Thank You: 

• So much gratitude to Rashida for answering my many questions about this!
• Thank you for the straightforward guide to setting up the firewall and ip tables, Professor Tom! By initially trying to do this assignment without the guide, I have realized how confusing the process of setting up a firewall and configuring the ip tables could have been. The internet has so many resources, but not all structured so well!

—

Helpful Resources for Understanding UFW ( initially, I forgot about the itp tutorial above, so found some forums/helpful links online for trying to understand what to do) :

• https://www.digitalocean.com/docs/networking/firewalls/resources/troubleshooting/
• https://help.ubuntu.com/community/UFW
• https://ubuntuforums.org/showthread.php?t=801351

—

Lines to remember:

sudo dmesg | grep '\[UFW'
grep UFW /var/log/syslog
/var/log
sudo tail ufw.log

remember:
csp root@.....:/var/log/ufw.log .