Cross-device collaborative applications


1) Pickit: a decision-making activity for high school students

The figure on top left is a decision-making learning activity which I designed and developed for high school students. In this activity, a group of three students act as a consult team, their mission is to help a agriculture company to choose an appropriate location for establishing factory. They need to take concern of the factory's requirements and also think about using sustainable resources.

To achieve their mission, students need to use the combination of large interactive display with personal tablets. The shared display shows the context of the decision-making. It supports group collaboration, such as discussion, negotiation and taking decision. The personal devices are used for individual exploration, such as analysing data of each location, sumbitting comments and providing evaluation. To get information of each location, students need to tap their own avatar inside the pop-up box of the marker on the shared display. The work flow contains three steps: individual browsing, collaborative discussion and collective decision-making.

For more details of this study, welcome to check our paper here.

This application is also hosed on Heroku server: shared surface (resolution: 1920x1080), tablet. Codes are here.

2) GoNY: a trip planning activity

GoNY is a trip planning activity I designed and developed to study how the orientation of a large display (horizontal vs. vertical) used alongside tablets impacts participants' collaboration. In this activity, users need to plan their trip itinerary for New York within a limited budget. The shared screen displayed a map with markers for tourist attractions and hotels. Participants could push detailed information on their tablet by tapping their avatar on a marker. Information provided for each location included: description, price, rating, and feedback from other tourists. Using their tablets, participants could individually add locations to their favorites and take notes. A card per location showed its favorites and notes on the shared screen. Four filter buttons on the shared screen were used to show/hide attractions, hotels, favorite locations, location cards.

Our results show that using a horizontal surface will lead to better equity of interaction and more cohesive activities. On the other hand, group coordination is more structured and is organized around a main interactor when a vertical display is used.For more information about this study, welcome to check the video here. This study is accepted by INTERACT conference 2017 (to be updated).

The application is available on-line: shared surface (1920x1080), tablet. Codes are here.

General design & develop methods & tools:
  • Interviewed and discussed with teachers to learn their requirements (Pickit)
  • Collaborated with a research team for designing UI and workflow
  • Developed applications: AngularJS, NodeJS, Mapbox (customizing map), Bootstrap, CouchDB (database), Heroku (hosting), Hammer (touch gesture), and Websockets (communication among devices)
  • Applied agile design methodology in the design and develop process
  • Conducted usability tests with students and participants using qualitative & quantitative analysis methods

InsectPuzzle: a mobile orienteering learning game


In this study, we designed and tested an orienteering mobile learning game to better understand how device use shaped collaboration in highly mobile conditions. The study involved four groups of three students all equipped with tablets. Each students in a group was assigned with a specific role. We focused our analysis on the relationship between participants’ arrangements (F-formations), their device usage and coordination mechanisms (i.e. awareness, regulation, information sharing, and discussion).

Our results emphasize the importance of considering the transitions between arrangements more than F-formations per se. We discuss the implications of these findings for the design and analysis of mobile collaborative activities.

For more details of this study, welcome to check our paper here.

Design & develop methods & tools:
  • Co-designed the game with teachers using storyboarding and Balsamiq Mockups
  • Managed a student developer for creating the Android application for tablets
  • Conducted usability testing with students to study how the mobile device and game design impact their collaboration and task performance using qualitative & quantitative analysis methods

An infrastructure that allows dynamic reconfiguration of the interactive spaces


It is an infrastructure which allows the dynamic reconfiguration of interactive spaces by grouping different devices together. I applied a simple web-based game onto this infrastructure. The game world could be automatially extended when we bring in new devices. User can touch the interactive surface to control a little player to jump through devices to collect stars.

To see how it works, please check the video here.

For more information of this work, welcome to check our paper here.

Develop methods & tools:
  • Developed a device detection system using Arduino and magnet switches
  • Developed multi-touch and tangible interaction recognition using CCV
  • Developed the application using jQuery, TUIO (multi-touch), NodeJS, Hammer, Phaser (game framework), and Websockets

Former work


This is part of the work for my master thesis which the title is "PMSM motor controller software and hardware system design based on TC1767". TC1767 is a high-performance 32-bit micro-controller from Infineon designed for automotive applications. In this work, we built a model for the permanent magnet synchronous motor (PMSM) drive system under the Matlab/Simulink environment, designed the hardware system using TC1767, and tested our PMSM vector control model on it. We proved that our model and hardware system worked well for motor drive control.

The figures on left are the peripheral test circuit I designed for our experiment.