Posts Tagged ‘mobile computing’

Watching smartwatches

April 26, 2016 13 comments

Smartwatches provide easy access to personal data in a wearable device. Modern devices sparking the latest wave of use include Pebble, Android Wear, Apple Watch. An important aspect of the popularity of these platforms is their open programming and app distribution platforms. For little or no cost, anyone with programming knowledge can develop and distribute an app. However, excitement about the platform and availability of a programming platform does not necessarily translate to useful and usable apps.

Two big hurdles exist that are particularly relevant for app designers: domains of use and continued use. First, it’s not yet clear what the domain for the smartwatch “killer app” will be—the apps that are so necessary and desired that people will pay for the technology necessary to use them.  Candidate areas for the killer app include health and fitness, highly accessible notifications for email and messaging, and social media. Second, an unanswered question is whether people will use them long term–there’s lots of attrition for even the most popular hardware.

We set out to understand these questions in my CS 3714 mobile software design class. An assignment asked that students perform an analytic evaluation of a smartwatch over the course of at least 5 days. Pebble, Android Wear, and Apple Watch smartwatches were available for checkout. Students were asked to identify at least three smartwatch apps to use prior to the 5-day period, then use the smartwatch and apps over the course of the 5 days for several hours each day. It was asked that at least one of the apps be a health- or fitness-related app, and at least one of the apps (perhaps the same one) was to have a companion app for the smartphone.

Students completed a form indicating whether they generally wore a watch (standard or smartwatch), which smartwatch they chose to wear for the assignment, how long they wore the smartwatch for the assignment, and which apps they used. The students were asked to craft a narrative to describe the experience with your selected hardware. The narrative covered display and interaction experiences as well as experiences with each of at least three different apps. It is expected that the narrative cover about 800-1000 words.

Students tended to complete this assignment with a higher completion rate than the other (programming) assignments for the class–68 out of 71 students submitted it. 24 students used the Pebble, 38 used an Android Wear watch, and 6 used the Apple Watch. Most used the smartwatch for longer than the requested 5 days; the median usage time was 7 days and the average was 8.9 days. Only 40% of students reported that they regularly wear any sort of watch, and only 10% reported having worn a smartwatch regularly.

Students tended to use more than the 3 apps that the assignment asked them to use. Most students used fitness apps that came with the smartwatch (e.g., Android Fit, Apple Activity). Others used run tracking apps, and a few tracked other diet or exercise. Map alerts and other notifications were popular, as were games. Surprisingly, only a few people reported using social media in a meaningful way (i.e., beyond receiving text messages); perhaps that is because of the short usage time.

Comments from student narratives reflected a general interest in the technology. They found the smartwatch “pleasant”, “nice and convenient”, and “very handy”.  Notifications seemed to be an advantage, with the smartwatch “a great way to read and dismiss notifications” (though others found notifications annoying or “glorified”). However, few people seemed poised to purchase or use the technology based on their experiences. The most common complaints were that the hardware was “ugly”, “awkward”, “incredibly silly”, and “not aesthetically pleasing”. Others found the technology hard to use, with comments like “my finger takes up half the screen”, “small buttons”, and “no way for users to type”. Lots of students admitted that they were “just not a watch person” or that they “disliked watches”, and there was nothing about the smartwatch that they wore to change their minds.

An important side effect of the smartwatch watching assignment is that students better understood the capabilities of smartwatches. In prior semesters when students did not have the experience of wearing a smartwatch, designs tended to be unrealistic or impossible to implement. Students in this semester seemed to have a better understanding of how a smartwatch would be used, and as such their homeworks and projects were targeted more appropriately for the smartwatch. There’s a danger that their experiences may stifle their creativity by highlighting what has been done, but that seemed outweighed by a realistic understanding of capabilities and scenarios of use.

There’s an interesting history for smartwatches, from the Dick Tracy vision to the poorly-received models from Seiko, IBM, and others through the 1980s and 1990s. The new wave of smartwatches seems to be booming, but it’s unclear whether that boom is here to stay. My research group has been exploring smartwatch use in the classroom as reported in a SIGCSE paper, demo, and poster in 2015.  And we put together an app set to look at reactions to smartwatches in an elementary school outreach experience.  A previous in-class activity comparing games across platforms (smartwatch, smartphone, and laptop/web). It seems likely that young people will help define whether and how smartwatches will be used (or whether the movement will fizzle, or appeal only to niche groups) in upcoming years.

NCA&T Mobile Computing Faculty Development Workshop 2015

July 30, 2015 1 comment

Last week I attended a faculty development workshop on mobile computing at North Carolina A&T State University (NCA&T). ncatgroup2015The workshop was funded by the NSF HBCU-UP program as part of a 3-year grant (with one year remaining).  A goal of the grant is to assemble modules and materials that could be adopted or adapted for use in undergraduate courses. The modules, which were core to the workshop, are described at Attendees came from universities, 4-year and 2-year colleges, community colleges, and one K-12 specialist!

I was struck by the breadth of ways in which mobile computing is taught: freshman-level courses, multi-course tracks, upper-level courses, module-based topic-centered modules.  I was invited because I’ve taught a junior-level mobile design class for a number of years.  I talked with one of the organizers at SIGCSE earlier this year, and he encouraged me to apply.  Some of the modules were spot-on, really hitting on topics that I should have been including in my course all along–particularly those related to security and performance. Some were topics that I already covered (maps, sensors) and others were better suited for more introductory courses. But overall it was worthwhile to hear about the modules.

Even more valuable than the modules were the discussions.  There was a great interactive session in which we brainstormed implications of the differences in mobile (sensors, multiple cameras, multiple changing networks, touchscreens, security at download) vs desktop (virtual memory, peripherals, multi-user support, runtime security) and how that impacts teaching.  The introductory session, the breaks, and the reception gave opportunities to talk with other attendees about their teaching approaches.  And the workshop wrap-up session gave the subset of us who could stick around a chance to brainstorm ideas for how to organize the modules and materials, explore ways that an EDURange-style approach could be used for dissemination, and possibilities for a SIGCSE paper that details successful teaching modules.  With the grant continuing, I look forward to taking part in follow-up efforts.0721151331b

The NCA&T campus is lovely, tucked in near downtown Greensboro right across from (the even more beautiful?) women’s college Bennett College.  (Alas, as with many places they choose the summer when students area away to do their campus improvements, so some key landmarks were being repaired.) NCA&T is a historically-black university with strength in computing security and information assurance. I’d been to NCA&T before as part of another grant, and I grew up in Greensboro so I’m certainly familiar with the school and area, but it was great to go back and visit again.


March 26, 2015 1 comment

Earlier this month a large group of Virginia Tech faculty and grad students attended the ACM SIGCSE Conference in Kansas City, MO—the flagship conference in computer science education. It’s an interesting conference, full of people at lots of levels of CS education: K-12, small colleges, big universities, and companies and book publishers that support them. The conference was is Kansas City, known for its BBQ, downtown plaza, long walks, and BBQ. VT faculty Steve Edwards, Cliff Shaffer, Manuel Perez, Dwight Barnette, and I all attended, along with a large number of grad students. Within my research group, Andrey Esakia, Shuo Niu, and Mohammed Seyam joined us on the trip to SIGCSE. I connected with lots of VT alums, along with with colleagues from Georgia Tech, UNC, NCWIT, Colorado, and elsewhere.

SIGCSE 2015 Pebble demo

our SIGCSE 2015 Pebble demo session with Shuo and Andrey

Our highlight of the conference was a full paper talk, focusing on our use of Pebbles in CS 3714. As far as I can tell, we were the first to use smartwatches in the classroom, and we had a well-attended talk. We touched on the assignments and activities from spring 2014 and summer 2014 sessions. We covered the core lessons related to smartwatches—including multi-device connectivity issues, wrist-mounted accelerometer use, and limitations in graphical and processing resources. At the end of the talk, Andrey demoed VT undergrad Jared Deane’s music synthesizer app—a class project in one of our classes—which was a big hit among those in attendance. There were lots of good and on-point questions at the end, showing that the audience was plugged in to our talk. Most of the questions focused on use cases for smartwatches that we were considering, though there was one on security issues with regard to smartwatch-phone pairings that merits future consideration.

We also had a demo in which we showed off the many apps that VT students have developed, including Jared’s and several by VT students Sonika Singh and Shuo Niu (favorites were Pebble-Paper-Scissors and Selfie Watch). That was a fast and furious hands-on session with some good discussions that hopefully inspired interest in using smartwatches in the classroom–as well as future outreach efforts like our Pebbles and kids program at a local elementary school. In addition, we had several posters as well. Seyam’s work on Pair Programming in the classroom was chosen as a finalist in the Microsoft Student Research Competition–yeah!  Andrey had a forward-looking poster on Android Wear in CS 3714.  These can both serve as stepping stones to bigger and better things.

VT folks on the SIGCSE road trip

VT folks on the SIGCSE road trip

Big thanks to everyone for helping make this trip a big success. Particular thanks to everyone for getting the apps working and available on the online store, and thanks to those who gave feedback on talks and posters and such.  And it was great to get to know the VT crowd on our massive road trip. CS Education is a big deal at VT, and it’s great that we were able to contribute to a big VT presence–over 20 faculty, students, and alums.  Next year’s event is in Memphis–a bit closer–so I’m hopeful we’ll have an even bigger presence there.

Pebbles and kids: Smartwatches in the elementary school

March 20, 2015 3 comments

Virginia Tech undergrads took Pebbles into a local elementary school computer club to show off the capabilities of smartwatches. 2015-02-10 18.07.07The Pebble smartwatch is a low-cost (and low-functionality) device with a low-resolution black-and-white screen, an accelerometer, haptic output, and Bluetooth connectivity. Thanks to a generous donation from Pebble, Virginia Tech received 100 of the classic Pebble smartwatches, presenting to us a great opportunity to use them in outreach events. At the elementary school, we set up stations with three types of apps: a graphics station, an accelerometer station, and a games station.

At the graphics station, students learned about the difference between the high-resolution color displays on computers and mobile devices and the low-resolution black-and-white displays on a Pebble smartwatch. Pebble’s 144×168 display may seem large (24,192 pixels!), but not so much when compared to the more than 2 million pixels on tablets and laptops.  Similarly, you can’t do as much with 2 colors (black and white) on a Pebble as with the 10,000+ available elsewhere. We highlighted this difference with the SelfieWatch app (that is at the heart of a Virginia Tech course-wide programming assignment crafted by grad student Shuo Niu). 2015-02-27 15.52.54 (2)This app allowed the kids to take a full-color picture of themselves with a smartphone and transfer a black-and-white version to the Pebble as a watch face—highlighting the difference in quality on the two screens, and letting the kids explore how faces, posters, words on the blackboard, and more don’t always look the same in reduced form.

At the accelerometer station, the students got a simple lesson in 3-degree-of-freedom accelerometers, learning how changes in x, y, z position can be used to measure steps, gestures, shakes, and more. The kids got to try out a simple step counter created by Virginia Tech undergrad Sonika Singh, as well as a more complex one that differentiated sitting, walking, and running. The kids tested out how many steps it took to walk around the school, and they saw how much the step count varied from person to person. They also had fun trying to “fool” the step counting algorithms by waving the Pebbles around.

At the games station, the students tried out a bunch of Pebble-specific games, both those by Virginia Tech students but also some available on the Pebble store. Most were accelerometer-based, including a version of the popular 2048 game by Sonika Singh in which kids had to tilt the watch to slide the blocks. Another popular app is a Pebble-based synthesizer written by Virginia Tech undergrad Jared Deane, in which a user can control features of music by tilting the Pebble.

This was one of the many wonderful and worthwhile activities sponsored by Virginia Tech’s Computer Science Community Service (CS Squared) student organization. They teach about programming, web apps, and mobile apps at AHarding Avenue Elementary school and other schools in the area; they visit our community center and a senior center to teach local residents about web apps like e-mail, Pinterest, Google Maps; and they participate in a variety of outreach activities around campus. (Full disclosure: I’m the faculty advisor for this club, though the vast majority of the work is done by the students.)

This event is just one of the many things we’ve been doing lately with smartwatches—in elementary schools, in Virginia Tech classes, and in our research efforts. Many were highlighted at the SIGCSE Conference in March 2015, a conference dedicated to computer science education. Links to papers, posters, smartwatch apps, and other resources can be found at

PIC-UP Mobile: For mobile by mobile

September 14, 2011 1 comment

Much of the ongoing research in my group focuses on usability tools for application developers, centered on knowledge transfer and decision-making among teams of designers. At the heart of my approach is the notion of a claim as a unit for knowledge capture, sharing, and negotiation–claims provide a falsifiable, designer-digestible chunk of knowledge that encapsulates an interface feature, its upsides, and its downsides. The small, hypothetical nature of a claim provides designers with opportunities to debate and evolve ideas to meet the needs of novel problems. Recently, Shahtab Wahid led a quality series of papers at Interact, DIS, and CHI (available from my pubs page) that summarizes our progress and describes PIC-UP for those interested in learning more about it.

An emerging focus is on tools for diverse teams, particularly teams of domain experts with little or no expertise in usability engineering. Since it’s tough for many companies to hire a large team of usability professionals to oversee interface development efforts, a suite of tools has promise to assist with the capture, sharing, and deliberation that must happen when addressing the needs of large stakeholder groups. It certainly doesn’t remove the need for a usability expert, but it can help magnify the power of experts, allowing a smaller number of experts to contribute to a larger number of projects.

As a next step, we are working to transition PIC-UP to a tool we call PIC-UP Mobile that will be useful in industrial and educational settings–specifically, to assist in the early-project development of mobile interface designs. We want the tool to be embedded with a small but high-quality set of claims that can be used in design activities. Designers will be able to browse, rate, group, and evolve claims during the design process. Design sessions can create scenarios or storyboards that incorporate key claims, with This will enable a large and diverse set of designers to have meaningful roles in the design of appropriate user interfaces for human experts–particularly domain experts with little or no knowledge about user interface design whose opinions all too often are often ignored.

Based on an established context, PIC-UP Mobile should help to answer questions like these:
– How can an interface best show multimodal information to a user within a unique context?
– How can an interface enable appropriate interaction.
– What are the tradeoffs in choosing one design technique over another?
– Where is more research needed to determine an appropriate interface approach?

There are tons of application domains where this seems well-suited: medicine, education/training, command-and-control, gas-and-oil discovery and processing. All of these domains integrate diverse situations in which stakeholders with differing backgrounds must reach decisions about acceptable (if not optimal!) approaches to addressing a problem. We expect that PIC-UP Mobile will help in reaching these decisions toward defining appropriate user interfaces.

Lessons from mobile interfaces class

August 16, 2011 Leave a comment

I just turned in grades for my 5-week mobile interfaces class–CSCI 4830/7000–a cross-listed undergrad/grad course offered during the “Summer B” term at UC-Boulder. It counted as a full 3-credit class, but it was crammed into 3 hours a day, 3 days a week–whew! Some thoughts I want to capture:

– Big thanks to Google for supplying the hardware we used in the class through their University Consortium program. Google is following the path to success that Microsoft followed in the 1980s and 1990s–lots of free stuff for universities to aid in teaching, with the payoff of knowledge and experience with their products and software when students graduate. More broadly, they are doing better than Microsoft of old in that they are providing lots of forums and development environments for free. Ironically, Microsoft has turned into Apple (and Apple has remained Apple-like) when it comes to their mobile platform, with a more closed platform and few/no inexpensive hardware options…and it doesn’t seem to be turning out well for them.
– Not sure if any course is a great fit for such a compressed time period, but mobile interfaces (with a programming emphasis) is better than most: lots of small projects that are individually useful but (if the prof does a good job) collectively meaningful. That’s a great side effect of the current state of things in mobile platforms: small standalone apps are in demand, so you can do something meaningful in a short period of time.
– Projects are nice, but big group projects in a summer class don’t make much sense. Doing it in the future in a month or less? Ditch the group project, or make it an extended/expanded homework.
– I like encouraging/requiring the students to publish to the Marketplace, though again 5 weeks is a tight turnaround. But my current thought is to keep it. It was great to look back and see the success that DeMarcus had with his tip calculator app–1000+ downloads, most still active–and we’ll see if anyone from the summer group can replicate that success.
– I liked having an assignment for which students presented on their favorite platforms. It was good to learn about Blackberry and iPad–but the Magic Cap presentation provided an interesting historical perspective on the development of mobile interfaces and allowed us to compare past and future. I’d be tempted to do more platform presentations (perhaps requiring every group to do a historically important platform and a modern one).
– Amateurs don’t write reusable claims about the apps they develop…and why should they? It’s hard to capture the key design lessons in a way that others will be able to learn from it. It’s a classic CSCW problem: there’s not much benefit for the person writing the claim, even if (big if) there is benefit for people reading them down the road. We need a claims goddess to polish them up and make them presentable!
– It was good to have the grad/undergrad mix, though there needs to be more mixing. The grad students should do some programming, maybe with App Inventor or Droid Draw, to understand better the challenges. The undergrads should do more writing, to encourage them to reflect on their programming.
– Avoid professional paper format for written submissions–the papers won’t be published in anything resembling the form in which they are turned in. And professional paper formats are tough to read, IMO.

Next up: a Maymester offering? Or a cross-listed semester-long offering at VT?

Google Faculty Summit recap

July 29, 2011 Leave a comment

In July 2011 I attended the Google Faculty Summit at Google NYC. The Google offices contained everything that I’d heard about: open offices, tons of meeting rooms, high-quality free food and drink every 50 feet or so, scooters to take you from place to place, old video game console library where you can “test out” old games, a giant Lego play area where people create Google logos and Sergey pictures and such, a rec room with foosball and ping pong and medicine balls and yoga classes, the by-appointment massage area, great meeting rooms with views of the city, and all the other perks I’m forgetting. It was a smart decision to have the meeting at the headquarters–definitely a positive experience.

The meeting started with a series of presentations, with Vint Cerf (yes, that Vint Cerf, now a Google employee) leading things off. Other speakers included John Wilkes (Omega), Andrew Chatham (self-driving car), Johan Schalkwyk (mobile voice search), Stuart Feldman (Google around the world, especially the East Coast sites). Most of these things you can learn about online–though it was good to put faces with the projects and be able to connect with the speakers afterward.

Seems like meetings of this sort have one big “half-baked idea”–for the Google Faculty Summit it was the Library Wall–a wall-sized display of all of the Google Books. You could swipe around with it, open books, read parts of them, add them to your personal collection (no wonder Borders went under). It was great fun…not sure what will become of it.

The afternoon of the meeting was pretty free-form, with lots of breakout groups, which was a bit frustrating. I ended up in a meeting that started off great but went in some directions not at all interesting to me–and it wasn’t really possible to switch to anything else. (Other rooms were full, or discussions were well under way.) It was good to get to talk to Johan about mobile voice search–though 2/3 of the people in there were algorithms folks trying to get him and his team to adopt their latest algorithm. The rest of us tried to push the Google team to broaden what the API can do (e.g., use more limited vocabularies, return correctness probabilities) but no promises!

Breaks and dinner provided a great opportunity to connect with people (my UC Boulder Department Head Jim Miller, NCSU/UVA colleague Mark Sherriff) and re-connect with people (Jason Hong and his pending baby and tenure case, Ben Bederson and the new UMD professional MS program). More time for that sort of interaction would have been great.

One interesting aspect of the second day was a rehashing of Google Health by its creator, Alfred Spector. Google had recently announced that they were shuttering Google Health at the end of the year. Most people assumed that it was the security and privacy issues that led to the closure, but Alfred’s take was that they never had enough data to really run into those issues (kind of scary that they were collecting data without a model of how it would be used/shared). Alfred pointed to two issues that were more relevant to GH’s failure: buy-in and objectivity. First, nobody was joining–not enough people to make it worthwhile for the types of tools Google wanted to develop. Second, it was difficult to establish objectivity; e.g., if liver function is poor but improving, it’s hard to say whether it should be classified as “good” or “improving” or “disastrous”…all of which may be true (or false).

Alfred also explained the Google research model–to focus on things that are useful now, to have researchers working side-by-side with developers, to seek out flexible knowledge dissemination paths (i.e., not papers). Not sure how ideas for the long-term will emerge from that, but it seems to be working for the short term.

Alfred also pitched all of the university programs–grants, focused awards, named fellowships, internships, visiting faculty, postdocs, summer of code, CS4HS–all great. In talking with the Google folks afterward, it’s clearly valuable (essential?) to have a high-up contact at Google to make this happen. Guess that should be my next step…Google Boulder, here I come!