Conference on Teaching Large Classes 2016

July 28, 2016 Leave a comment

In preparation for teaching a junior-level undergrad introductory human-computer interaction class with up to 150 people, I attended the Conference on Teaching Large Classes at Virginia Tech. There were over 100 people in attendance, with about half from VT and the rest from other universities. Peter Doolittle served as master of ceremonies, and a huge group of other people worked hard to make it a success. There was a lineup of educators from a wide variety of disciplines who shared approaches that they feel work for large classes.

My main interests were to understand how to connect with the students in meaningful ways that get them to actively practice the techniques that are important in the field of human-computer interaction. This introductory class doesn’t focus on programming but on methods, so if the students aren’t understanding when and how to use the methods, they won’t get it–and that takes practice. In past years I taught this class with 30-40 people, maybe up to 70 or 80, where I could assign in-class activities to teams of 3-4 with some expectation that I would be able to interact with all the groups. With a group of 150, that no longer seemed possible.

The conference made it clear that the notion of “large” is self-defined; it’s put forth as a size at which one feels unable to use familiar methods in teaching the class. That seemed to be a great match for situations like mine, though sometimes it meant that a speaker might be talking about teaching to a large group of 50, or a large group of 400, and the method wouldn’t be relevant to me. It would have been helpful if the breakout talks were somehow labeled with an indication of what size range “large” meant to the speaker or leader.

Martha Olney from UC Berkeley gave the opening keynote. She talked about iClickers (or free mobile phone equivalents) that encourage participation throughout a class period. She also noted that she often has a rapid-fire handful of quizzes early on to get people in their seats and engaged on time. She noted that students are more engaged and there were fewer D and F grades when even just a few quizzes were given each day. She also talked about other electronics in class and admitted she’s gone back and forth on a policy. A survey of the audience revealed that 58% of attendees allow any electronic device to be used in classes, 16% ban mobile phones, 9% ban all electronics.

Much of the conference focused on breakout sessions, where attendees could choose a session that matched their needs and interests. My favorite presentation was by Gary Green from UGA gave a great and engaging talk that pushed the effectiveness of humor and the need to call people out and keep them off balance. One example he gave was a “what’s in the bag” technique, where good answers (or questions) would give someone the opportunity to pull an object from a bag. The first time he did it, the “bag” was his lunch, and the activity arose because people wondered what was in there. He later created a bag of old kid toys, company giveaways, and the like. He also talked about ways to balance low, medium, and high stakes assignments (worth <3%, 3-9%, or 10+% of the total grade, respectively) to maintain people’s commitment and attention levels. He also surveyed students about their preferred means of communication–in order student preferences are texting, before/after class, mobile phone, learning commons, email, office phone, office hours. Audience members were stunned that he gave out his mobile number to large groups of students, but he noted that he peaked out at 6 messages an hour and most could be answered in a single word (“yes”, “no”, “5pm”, “tomorrow”). I’m not sure I’m going to try that last one!

There were tons of other pointers that were provided that need follow-up as I prepare to teach my class. Michelle Soledad talked about online homework resources like chegg, WileyPLUS, and informal forums. Deborah Good and her colleagues presented an online resource for getting students engaged. Mary Marchant and Kim Morgan talked about lessons from CIDER’s big courses workshop, detailing ways to engage people through projects, activities, and other graded activities. One theme that emerged in multiple presentations noted that many interactive techniques don’t “move the bar” in terms of number of As, Bs, or Cs, but they do decrease the number of Ds and Fs—helping the poor or struggling students. I feel certain I’ll be going through the online proposals and additional readings from the web site as my planning efforts continue.

Greg Justice from VT’s Theater and Cinema Department gave the closing keynote. He acted out lots of positive and negative examples of how lessons from theater can be used in large classes. A smattering of examples: Move from back to center-front, or move from left to right to draw attention (it’s the direction we read), but introduce conflict by moving from right to left. 50% of effective communication comes from what you communicate physically, 40% is vocal, and 10% is from the words that are used. Warm up before you “perform”, and have your students do it too. Lean forward if sitting. Get out from behind the podium. Avoid teen speak, where each sentence falls off at the end. Talk from your heart, talk from your passions.

Overall, a worthwhile experience. It was great to see other folks from my department there, including Dwight Barnette and Anamary Leal. More to come on this topic as my preparation for the class moves forward. And perhaps next year I’ll be presenting here!

Categories: Uncategorized

NCWIT 2016

June 22, 2016 Leave a comment

ncwit2016The National Center for Women & Information Technology (NCWIT) held their 2016 annual summit last month in Las Vegas. The big news is that Virginia Tech received a NCWIT NEXT Award for our work on recruiting and retaining women in computer science (CS) and related areas. I’m particularly proud of my own work in reaching out to minority-serving institutions and in helping to craft CS-related minors (hopefully to be augmented with an HCI minor soon!), but this was definitely a team effort that included efforts by Barbara Ryder, Libby Bradford, Greg Farris, Deborah Tatar, Margaret Ellis, Bev Watford, and many others at Virginia Tech, plus a long list of NCWIT folks highlighted by our consultant Cathy Brawner, the Extension Services team, and the Pacesetters team.

NCWIT is a collection of companies, academic institutions, government agencies, and other groups working to increase women’s participation in computing-related fields through recruitment, retention, and advancement. As usual, the summit was an impressive event, packed with notables from academia and industry with keynotes and meet-and-greet events featuring exciting themes. Particularly motivating was the plenary by Melissa Harris-Perry from Wake Forest, who talked about getting more black women engaged in computing, particularly as professors. She called our Virginia Tech as a leader in this regard, particularly given the relatively large number of black women who have received PhD’s from here. But there’s certainly a need for more concerted efforts toward crafting welcoming environments for people in underrepresented groups.

Breakout groups help focus on topics of interest and importance to schools and groups with needs similar to our own. I attended meetings for the Academic Alliance and Extension Services, and workshops focused on diversity with respect to makerspaces, growth, pedogogy, and evaluation. One theme repeated at multiple venues that really resonated with me was the need for peer mentorship. We do a good job with this, but other ideas worth considering involve credit-based opportunities and other rewards for participation that enable and encourage a breadth of participation. This breadth can encourage diversity in the mentorship pool, and corresponding diversity in our student population. UC Irvine and the University of Wisconsin both have credit-based programs in place that reportedly are working well for them, and others have been considering adding them.

So who should attend the NCWIT Annual Summit?  It’s great to keep a foot in the door and make sure some people from your institution attend every year. But it’s also important to invite a few different people each year—we had myself, Barbara Ryder, and Libby Bradford there as regulars, but also our Associate Dean for Academic Affairs and Director of the College Center for the Enhancement of Engineering Diversity, Bevlee Watford, for just the second time.  I’m hopeful that we’ll get some repeat attendees again next year, but it’s also good when there are new faces as well. Our departmental Diversity Committee will be under new leadership starting in the fall, so hopefully the new chairs will attend!

Reflecting on McCrumb books

May 12, 2016 Leave a comment

On May 10 I attended the premiere of Sharyn McCrumb’s latest book, Prayers the Devil Answers, at the Salem Historical Museum in Salem VA. Sharyn is a great storyteller, both in person and in her writings, and I’ve enjoyed hearing her talk about how her books come into being over a long period of time. Last night’s premiere was no exception…and it gives me an excuse to revisit her work and talk about some of my favorites.2016-05-10 20.21.42

Sharyn McCrumb is an Appalachian author in every sense of the word. She’s born and bred in the region, with family roots that go way back.  Plus, she really does her homework with the books she writes, and it shows. She’s best known for the multiple stories she interweaves in her books, but she also does a great job crafting interesting, believable, and flawed characters with whom a reader is made to feel a connection. In recent years her characters increasingly have been based on historical figures, including NC Civil War Governor Zebulon Vance. She also has some recurring characters whom she’s revisited at various life stages: a collection of law enforcement officers including Sheriff Spencer Arrowood, and possessors of “the sight” Nora Bonesteel and Rattler.

I can’t yet comment on this latest book, but I’ve read many of her other books at least once, and some multiple times. Here are some of my favorites, roughly in order (though that changes).

  1. She Walks These Hills: A series of interwoven tales about a woman who escaped from Shawnee captors in 1779 and walked hundreds of miles home, a grad student who seeks to retrace her path, and an escaped convict who happens upon the same trail–in the end all coming together in an interesting way. I identify with the characters even more than many of her other books–some days I’m grad student Jeremy Lamb wandering through the woods, other days I’m Harm Sorley…and maybe even a bit of radio announcer Hank the Yank or police deputy Martha.
  2. Rosewood Casket: A story about adult children who come home to say goodbye (and, in the end, bury) their elderly father.  An interesting and emotional book–I might appreciate this book even more when it becomes more real for me.
  3. The Ballad of Tom Dooley: A more recent novel, it really shows how far she’s come in her ability to research the heck out of a topic, then tie together the things she learned in an engaging tale set in Civil War times.  It seems she really found a unique and believable angle for an assortment of characters–a diseased and godless young woman, her beautiful but lazy cousin, the apathetic and faceless husband, the idle but handsome lover.
  4. The Songcatcher: Highlighted in this book is a father-daughter relationship, where they are seeking to reunite during the father’s dying days. There are parallel stories tracing the McCourry family history and a pair of small plane crashes separated by several decades. (Note: this book is not related to the 2000 movie, except in the songcatcher theme)
  5. Ghost Riders: Another book that weaves several stories together, rich in Civil War themes. It highlights how Appalachian people–both then and now–engage with the Civil War…the anger of stolen land and food and people by both sides, and a desire to be left alone.  The book features Civil War reenactors, a recurring character named Rattler, NC Governor Zebulon Vance, and Appalachian newlyweds Keith and Melinda Blalock just trying to make their way in life.
  6. Foggy Mountain Breakdown: It’s hard to compare a collection of stories to a novel.  There are some stories in here that are among her best, and there are some that I skip.  Most have some phrase or paragraph that I really love, though.  And some have phrases and themes that appear in novels.  I tag them when I read them, favs include Telling the Bees, Precious Jewel, A Predatory Woman, Happiness is a Dead Poet, and the title story Foggy Mountain Breakdown.

Most of these books are categorized as in her “ballad” series, that includes many other good books as well. You may also come across her short stories from time to time; some of the best are in the Foggy Mountain Breakdown collection described above. She also wrote a set of “MacPherson mysteries” (e.g., If I Killed Him When I Met Him) that are in the classic whodunit style.  And her first books had Virginia Tech and sci-fi connections to them that may appeal to some people.

Categories: Book reviews

Watching smartwatches

April 26, 2016 13 comments

Smartwatches provide easy access to personal data in a wearable device. Modern devices sparking the latest wave of use include Pebble, Android Wear, Apple Watch. An important aspect of the popularity of these platforms is their open programming and app distribution platforms. For little or no cost, anyone with programming knowledge can develop and distribute an app. However, excitement about the platform and availability of a programming platform does not necessarily translate to useful and usable apps.

Two big hurdles exist that are particularly relevant for app designers: domains of use and continued use. First, it’s not yet clear what the domain for the smartwatch “killer app” will be—the apps that are so necessary and desired that people will pay for the technology necessary to use them.  Candidate areas for the killer app include health and fitness, highly accessible notifications for email and messaging, and social media. Second, an unanswered question is whether people will use them long term–there’s lots of attrition for even the most popular hardware.

We set out to understand these questions in my CS 3714 mobile software design class. An assignment asked that students perform an analytic evaluation of a smartwatch over the course of at least 5 days. Pebble, Android Wear, and Apple Watch smartwatches were available for checkout. Students were asked to identify at least three smartwatch apps to use prior to the 5-day period, then use the smartwatch and apps over the course of the 5 days for several hours each day. It was asked that at least one of the apps be a health- or fitness-related app, and at least one of the apps (perhaps the same one) was to have a companion app for the smartphone.

Students completed a form indicating whether they generally wore a watch (standard or smartwatch), which smartwatch they chose to wear for the assignment, how long they wore the smartwatch for the assignment, and which apps they used. The students were asked to craft a narrative to describe the experience with your selected hardware. The narrative covered display and interaction experiences as well as experiences with each of at least three different apps. It is expected that the narrative cover about 800-1000 words.

Students tended to complete this assignment with a higher completion rate than the other (programming) assignments for the class–68 out of 71 students submitted it. 24 students used the Pebble, 38 used an Android Wear watch, and 6 used the Apple Watch. Most used the smartwatch for longer than the requested 5 days; the median usage time was 7 days and the average was 8.9 days. Only 40% of students reported that they regularly wear any sort of watch, and only 10% reported having worn a smartwatch regularly.

Students tended to use more than the 3 apps that the assignment asked them to use. Most students used fitness apps that came with the smartwatch (e.g., Android Fit, Apple Activity). Others used run tracking apps, and a few tracked other diet or exercise. Map alerts and other notifications were popular, as were games. Surprisingly, only a few people reported using social media in a meaningful way (i.e., beyond receiving text messages); perhaps that is because of the short usage time.

Comments from student narratives reflected a general interest in the technology. They found the smartwatch “pleasant”, “nice and convenient”, and “very handy”.  Notifications seemed to be an advantage, with the smartwatch “a great way to read and dismiss notifications” (though others found notifications annoying or “glorified”). However, few people seemed poised to purchase or use the technology based on their experiences. The most common complaints were that the hardware was “ugly”, “awkward”, “incredibly silly”, and “not aesthetically pleasing”. Others found the technology hard to use, with comments like “my finger takes up half the screen”, “small buttons”, and “no way for users to type”. Lots of students admitted that they were “just not a watch person” or that they “disliked watches”, and there was nothing about the smartwatch that they wore to change their minds.

An important side effect of the smartwatch watching assignment is that students better understood the capabilities of smartwatches. In prior semesters when students did not have the experience of wearing a smartwatch, designs tended to be unrealistic or impossible to implement. Students in this semester seemed to have a better understanding of how a smartwatch would be used, and as such their homeworks and projects were targeted more appropriately for the smartwatch. There’s a danger that their experiences may stifle their creativity by highlighting what has been done, but that seemed outweighed by a realistic understanding of capabilities and scenarios of use.

There’s an interesting history for smartwatches, from the Dick Tracy vision to the poorly-received models from Seiko, IBM, and others through the 1980s and 1990s. The new wave of smartwatches seems to be booming, but it’s unclear whether that boom is here to stay. My research group has been exploring smartwatch use in the classroom as reported in a SIGCSE paper, demo, and poster in 2015.  And we put together an app set to look at reactions to smartwatches in an elementary school outreach experience.  A previous in-class activity comparing games across platforms (smartwatch, smartphone, and laptop/web). It seems likely that young people will help define whether and how smartwatches will be used (or whether the movement will fizzle, or appeal only to niche groups) in upcoming years.

SIGCSE 2016

April 6, 2016 Leave a comment
2016-03-05 14.04.23

Virginia Tech students and alums at SIGCSE 2016

SIGCSE 2016, the flagship conference on computer science education, took place in Memphis TN in March, with a big collection of Virginia Tech students, faculty, and alumni taking on a variety of important roles. My grad student Mohammed Seyam and I presented a paper on teaching mobile software development with Pair Programming. Cliff Shaffer and his students and alums had multiple papers and exhibits. Greg Kulczycki served on a panel.  And, most notably, Steve Edwards was program co-chair this year!

Mohammed Seyam’s paper and talk focused on Teaching Mobile Development with Pair Programming. It explored his investigation of Pair Programming (PP) when teaching mobile software design in an upper level CS course. PP has been shown to be useful in some teaching situations, but Mohammed is the first to look at it in teaching mobile. He also had an entry in the graduate Student Research Competition that took a broader look at the balance between PP, hands-on activities, and traditional lectures when teaching mobile software design, for which he was named a finalist.

As always, SIGCSE featured interesting and engaging keynotes. John Sweller talked about the impacts of cognitive load theory on CS education. Barbara Boucher Owens and Jan Cuny received service awards from SIGCSE and gave keynotes that reflected their life experiences. It was particularly good to see Jan Cuny receive an award given her contributions to diversity in leading broadening participation in computing programs at the NSF. Karen Lee Ashcraft talked about breaking the glass slipper, and how organizations historically (and continually) have crafted jobs and workplaces that encourage stereotypes. This was a bolder and more developed version of a talk she gave at NCWIT 2015.

One of my favorite emerging things at SIGCSE is the Common Reads initiative, which returned for its second year. It’s an effort to encourage SIGCSE attendees to read a common set of CS-related materials. There are stickers for conference badges that are handed out at registration to highlight who’s read what, thus providing another avenue to start conversations. And there’s a conference session one evening to discuss the readings, how they relate to CS, and how they can be used with students. This year’s books were all science fiction: The Diamond Age by Neal Stephenson, Ancillary Justice by Ann Leckie, A Logic Named Joe by Will F. Jenkins, and Seven Years from Home by Naomi Novik. These books and stories touch on core CS themes like AI, parallel computing, fault tolerance. While thee themes are certainly relevant to CS, it seems important to me to promote topics other than just science fiction to support a breadth of interests.  As such, for SIGCSE 2017the most intriguing common read to me is The Thrilling Adventures of Lovelace & Babbage: The (Mostly) True Story of the First Computer by Sydney Padua. It’s a comic-style reimagining of CS heroes Ada Lovelace and Charles Babbage, exploring a world in which they collaborated closely to build and use a computer. There are a couple of other sci-fi entries included as well, Andy Weir’s The Martian (yes, the book that the movie is based on) and Isaac Asimov’s The Last Question short story.

It was fun to connect with the VT crowd on the LONG van ride across Tennessee to Memphis. The Memphis area is a little depressed, but there seem to be efforts at renovation, and the food and music were a great indulgence. It was fun to be just a few feet from the Mississippi River during the conference, and we were able to duck across the border to neighboring Arkansas and Mississippi on our drive.  We also had quick visits to Nashville and Kingston going to and from the conference. Next year’s SIGCSE will be in Seattle, so it’s unlikely we’ll drive to that venue!

Several others put together writeups about this event as well. CS@VT blogged about VT’s participation in SIGCSE (excerpts from this post), and Georgia Tech put forth a press release about the event. Mark Guzdial from Georgia Tech has several blog posts about Jan Cuny’s SIGCSE Outstanding Contribution award and a description of one of his posters replicating his earlier work. It was enlightening to read about the frustrations in publishing replicated work. There’s real value there but so many venues put much more value on innovation rather than replication. Janet Davis blogged about her experiences at SIGCSE from her perspective as a faculty member starting a new CS department. Georgia Tech and NCWIT had groups there too, and it was great to connect with them. And I’m sure there’s much more writeups about SIGCSE that I missed–feel free to include other relevant links in the comments.

Being smart about games for smartwatches: Preliminary evaluations and feedback

November 18, 2015 1 comment

Smartwatches like Pebble, Android Wear, and Apple Watch provide user interface challenges given their small size and limited input means. At Virginia Tech, we’ve taken part in development and outreach efforts that have explored the use of smartwatches through app development, K-5 outreach, and, most recently assessment. This post presents findings from an in-class investigation in an undergrad human-computer interaction class.

The class explored performance on a game that is common across multiple platforms: Minesweeper. We configured the game to use an 8×8 grid (64 total squares) with mines hidden at 10 of the locations. The game requires players to identify the 10 mines and uncover 54 squares with no mines. A player can select a square to reveal how many mines are in adjacent squares, or mark a square as having a mine. The game ends when all squares are reveled or marked, or when the player reveals (as opposed to marking) a square with a mine. Score is reflected in number of mines found (so 10 is the worst score and 0 the best) and time taken to complete a game (lower times are faster/better).

We looked at Minesweeper on three platforms: laptop (Web), smartphone (mainly Android, some iOS), and smartwatch (original Pebble with 3-button input). We endeavored to find Minesweeper apps that were similar in appearance, though the limited graphics of the Pebble resulted in a less visually appealing game for that platform. Also, we note that the interaction styles are very different across platforms: laptop users used the touchpad and buttons to highlight mines, smartphone users performed a press to reveal and long-press to tag, and watch users scrolled with the top and bottom buttons, revealed with the middle button press, and tagged with a long press.

We explored whether platform has an effect on performance and enjoyment, with the thought that a more interaction-rich platform like the laptop and smartphone would be easier to use. Students in the class played the game for 12 minutes, with encouragement to complete as many games as possible within the 12 minutes but seek to be successful in playing the game. Students divided into groups of 4, with one person recording data and the other three playing the game on the three platforms.

Major disclaimer: this study was conducted in a classroom setting by students with minimal training in running user studies (i.e., a single lecture). Some students didn’t understand the activity, collected the wrong data, and entered data incorrectly. In some situations, the data seemed highly questionable and was eliminated from consideration. I would expect that the results would be of interest toward crafting future studies rather than in and of themselves. This activity was primarily done as a learning experience for the students, but it was interesting to see the results that students generated as part of the activity. Certainly this doesn’t belong in a peer-reviewed venue, but I’m hopeful it will serve as a launching point for future investigations of smartwatch interfaces.

Some key results:

  • Participants attained better scores with the Web (4.9) and smartphone (4.4) than with the Pebble (7.5), p=0.000001. The difference between Web and smartphone is not significant, p=0.21.
  • Participants found the Web (4.3) and smartphone (4.2)  versions of the game easier than the Pebble (0.3) based on a 0-5 point scale, p<3×10-30. Similarly, they found the Web (4.1) and smartphone (4.4) versions more fun than the Pebble (0.5) on a similar scale, p<5×10-31. There was no difference between Web and smartphone for either measure.
  • There were no differences in time spent on each game between Web (40.1), smartphone (45.2) and smartwatch (51.2), p=0.31.

Students expected that Pebble would result in the worst performance and would be most disliked, but they found it surprising that there was not a significant difference between Web and smartphone. People speculated that a more in-depth questionnaire and deeper examination of the tasks (slips, errors, strategies) would reveal more actionable differences. I was glad they saw this as the start of understanding smartwatches and other platforms.

This was the first time that I had the students in my class run a class-wide study of this type, and it resulted in mildly-controlled chaos. Usually I encourage them to take part in user studies run by grad students to get a sense of how studies work, but I feel there was a lot more participation and understanding by having them take part in all aspects of a study.

Games is one of the six categories of apps for Pebble, with 44 games that have over 100 downloads (compared to 29 for notifications, 44 for health and fitness, 66 for tools and utilities, 34 for remotes, and 56 for daily use as of 11/18/2015). Some of the games leverage definitive characteristics of smartwatches (e.g., Maze, Pong, and Ledge Jumper’s use of the accelerometer), but many are ports of common games (e.g., Tetris, chess, Flappy Bird, Minesweeper) that aren’t good matches for the Pebble display and interface capabilities.

We focused on games in this investigation because of the breadth of knowledge of our participants—everyone was familiar with Minesweeper and understood basic strategies. I would consider undertaking such a study at the start of a mobile computing class, to get across to people that apps should be targeted wisely for their platform. However, our research efforts continue to focus on health and wellness apps, leveraging lessons about appropriateness for the platform but considering how the smartwatch can provide unique value to the user.

Categories: Uncategorized

NSF Graduate Research Fellowships: Maximizing Chances for Success

September 10, 2015 Leave a comment

The U.S. National Science Foundation (NSF) offers Graduate Research Fellowships (GRF) to applicants who are beginning or about to begin a Ph.D. I’ve advised a student who has written a successful one, I’ve reviewed applications internally for people in my department, and I’ve become intimately familiar with the current review process for the NSF. There’s no magical formula for getting one that I’ve discovered, but there are definitely things you should and shouldn’t do to maximize your chances. This post seeks to capture my experiences and advice—of particular relevance to those in computer science and human-computer interaction but perhaps applicable in other fields as well.

My grad student Greg Wilson received an NSF GRF in his first year at Virginia Tech. His proposal discussed solid and interesting ideas related to mobile and ubiquitous computing, but what really appealed to the reviewers was his outreach efforts. He has a passion for K-12 education, and his application discussed that in detail. He described prior outreach efforts in his personal statement, thus demonstrating an interest and ability in similar efforts in his graduate work. Receiving this fellowship allowed Greg to pursue his own ideas and really make a difference with his work. He completed his MS at Virginia Tech and went on to a Ph.D. in education at the University of Georgia.

The Virginia Tech Computer Science Department hosts an internal review process for national and international graduate scholarships and fellowships like the NSF GRF. It is organized by faculty member T.M. Murali and includes work sessions, early reviews by fellow grad students, and reviews by faculty in the department (including myself some years). It’s a great way to get feedback both from peers and from potential committee members, and I feel like it really made a positive difference for my student Greg. If you don’t have this available to you, find a way to get feedback from a breadth of other people.

I am very familiar with the reviewing process for NSF applications. For the last couple of years, it has taken place via teleconference, in which reviewers read and comment on applications prior to a pair of online meetings. The meetings present a listing of ratings, then ask for champions of lower-rated proposals that seem particularly worthy. The 20+ person online panel breaks into smaller 3 person groups to discuss moving proposals up (or down) the ranking if a proposal’s champion makes a compelling case for why it should be moved. If you can attract a champion, you’re greatly improving your chances. The final listing serves as a recommendation to NSF program officers and other personnel, who make the final determination as to who receives an award.

A few summary thoughts and recommendations that can help with a successful submission:

  • Follow the guidelines. Yes, there are lots of them, and I’m sure you have great ideas that you might feel should carry your proposal even if you don’t pull together your application just right.  But failing to follow the guidelines can obfuscate your expected contributions. You risk annoying the reviewers and the program managers by making them dig for (or guess at) certain elements of your proposal.
  • Provide a roadmap for your proposal. Keep in mind that reviewers will be looking at lots of proposals, and secondary reviewers and program managers will be looking at even more—sometimes for very short periods of time. As such, make sure the key points of your proposal can be found at a glance. Label sections and subsections, highlight key terms, craft figures and tables that are both descriptive and easy to understand. And don’t use a tiny font just to squeeze more in—find a way to say what you want to say concisely. Of course, none of this matters if the content isn’t good, but good content that can’t be understood easily can also sink a proposal.
  • Think about intellectual merit. The NSF cares a lot about this (and the next bullet, broader impacts). Read the full description on the NSF site and specifically address ways in which your work will have intellectual merit. Even if you feel your entire proposal is all about intellectual merit, make sure to explicitly highlight your expected contributions.
  • Think about broader impacts. This one is even harder, but as with my student it really matters. It’s important to show how your work will make a difference, keeping in mind that reviewers will be generally knowledgeable about your field but not necessarily deeply knowledgeable about your topic. As such, don’t just make a laundry list; e.g., stating that your work will lead to improved interfaces for scientists, bricklayers, moms, bartenders, etc. Instead really draw the path to the future utility of your work—and if you can show yourself guiding the research down the path, all the better.
  • Get good letters. This one, to some degree, is out of your hands—but that doesn’t mean you can’t make choices that maximize your chances for good letters. The best letters are from people who BOTH know you AND know how to write good letters. A letter from someone who knows you very well but doesn’t understand NSF GRFs might be a poor choice, just as a letter from a highly regarded individual who clearly knows nothing about you and has little to say about you likely will be unhelpful. Seek to approach people who’ve been part of successful NSF GRFs in the past, and from people who will help you toward your proposed goals. But make sure these are people who can either say good things about your prior work and/or good things about your proposed work—people who have been a meaningful and integrative part of your research life.

Finally, keep in mind that, for better or worse (usually better), the NSF regularly changes the guidelines and procedures for fellowships, so make sure to verify that your submission matches the way things are done. There’s lots of other advice out there, so seek to find it and identify the path that is most promising to you. There’s always a bit of randomness to the procedure, but there are steps you can take that can increase your chances of receiving an award. Most of all, pursue interesting and important ideas that appeal to you and your collaborators. Good luck!

Follow

Get every new post delivered to your Inbox.