Panel Summary #1 Proposal Number: 0845351 Panel Summary: Panel Summary I. A brief statement of what the proposal is about The PI proposes two interesting social computing projects -- one addressing author motivation in open content systems (an extension of his SuggestBot work), the other examining the role that social media could play in reminiscence. II. Intellectual merit - Strengths The panel was impressed with the framing of the proposed work, noting that it was "sophisticated," "intellectually rich" using "nuanced argument" and filled with "detailed, concrete examples." They noted the excellent track record of the PI to date on these topics, and appreciated the strategy of pursuing a "safer" project in the first few years and pursuing a potentially transformative "high risk" project in the later years. The panel thought that the PI provided a solid critique of the memory as data metaphor and provided some interesting alternatives. As the study progresses, it will be interesting to see how these will play out in the design space. The social computing sites, Wikipedia (multi-language sites) and Facebook are particulary approporiate platforms for each study and the PI has a demonstrated track record of working well with each. - Weaknesses Reviewers were concerned that some key aspects of the study design were underspecified, in particular, the design and use of cultural probes. Providing some examples would have been very helpful. The partnerships with other faculty members (e.g., Sengers) who are experts in these methods alleviated these concerns. Some panelists commented that the two projects are largely unrelated and independent, although together they do represent an energetic five year plan. III. Educational activities, including integration of research and education - Strengths In his short time at Cornell, the PI has had a significant impact on the HCC curriculum. Reviewers were impressed with his "deep thinking" leading to a reimagining of the introduction to HCC as an introduction to a full HCC program to develop cross-disciplinary students. - Weaknesses The panel hoped that the PI would expand his integration of research and education beyond the standard curriculum for traditional students at his own institution. IV. Other broader impacts, including enhancing diversity There are clear societal benefits to both projects, continuing to improve the content of social media sites and rethinking the role that they may play in enriching our daily lives. V. Panel recommendation and justification, including key strengths and critical weaknesses _X_ Highly Competitive ___ Competitive ___ Not recommended for funding by panel This project clearly excited the panel with its potential to fundamentaly transform our relationship with user-generated social media. The summary was read by the panel, and the panel concurred that the summary accurately reflects the panel discussion. Panel Recommendation: Highly Competitive ==================== REVIEW: What is the intellectual merit of the proposed activity? This work has important consequences for improving the state of general knowledge about (1) human motivations that contribute to distributed content creation and (2) the use of distributed personal content to develop psychologically beneficial consequences using the psychology of reminiscence. What are the broader impacts of the proposed activity? The project will create useful working applications that have practical value. One application (SuggestBot2) will improve distributed content creation and management on Wikipedia, while another area of research will produce working applications that assist in the use of computing systems (such as social networking sites) as a kind of prosthetic memory. The project also includes a novel re-imagining of the introduction to human-computer interaction course, and proposes to develop a new breed of cross-disciplinary graduate researcher able to exploit both social and technical approaches to computing. Summary Statement This project is well-written and engaging and I found it to be a delightful read. This is a thoughtful and sophisticated research plan that integrates theory and research across a very wide variety of intellectual domains. It brings this wide breadth to bear on a single application area: distributed collaboration. The proposer is uniquely positioned to make substantial contributions in this area, due to the success of an earlier application (SuggestBot) on Wikipedia and a deep preexisting network of relationships among Wikipedia contributors. A very wide range of possible research activities is proposed here (3.2). For me, the most creative and interesting part of this proposal was the novel use of reminiscence protocols. However, this research project is not well-integrated with the proposals about Wikipedia. Although clearly both areas of research are somewhat related (through the hook of distributed content creation), it seems relatively straightforward to reposition the reminiscence research as also being about motivation in a collaborative environment like Wikipedia. This would make the overall program of research more coherent. That is, the proposer identifies the individual-level psychological benefits in learning, community-building, and affect that come from reminiscence. As Wikipedia is a community and the first part of the proposal tries to identify motivating factors to increase attachment and contributions, can't we see the reminiscence project as a possible motivator? Don't Wikipedia contributors like to reminisce? Perhaps only a very small portion of regularly contributing writers on Wikipedia would enjoy reminiscing about other writing tasks (I don't know), but even so this link would allow the overall research plan to be presented in a more coherent way. The proposal attempts to construct cross-disciplinary competency in students in a very novel and promising way. It combines information science, history, communication, psychology, and human-computer interaction in surprising ways. It proposes (in part) to use the results of this research to help redesign introductory curricula in human-computer interaction -- an important programmatic goal. Overall I am delighted by the proposal and am eagerly looking forward to the proposed research and its results. --------------- REVIEW: What is the intellectual merit of the proposed activity? This is an outstanding proposal along many dimensions: 1. The topic of study is intellectually very rich, while also having direct relevance. (More on that under "Broader Impacts".) 2. The PI has done an outstanding job of outlining not just a short-term research project, but a 5-year line of investigation, as is appropriate for a CAREER proposal. He has an crisp overarching theme, for which he has identified a set of well-defined initial research efforts; at the same time, he provides compelling evidence that there are a range of research questions that will develop from the first few years of work. 3. He has completed initial studies that give credence to his ability to pull off this larger effort and that demonstrate his mastery of the background literature. 4. The proposal is beautifully written. These are all high-level comments, but for the most part more detailed ones aren't needed: the proposal speaks for itself. I have just one small quibble, which is that I think he should take more seriously the economic literature on motivation and incentive in social computing; see for example the work of J. Mackie-Mason and of R. Sami. What are the broader impacts of the proposed activity? The proposal is transformative. It's obvious that the use of the web has had an enormous impact on all kinds of aspects of daily activity. The PI proposes to undertake research that will, on the one hand, enhance the quality of information available on the web, and on the other, will make use of web-based activity that people perform anyway, to improve their lives (by supporting "reiminscing"). These goals in themselves have broad impact. In addition, the PI's use of "hard-core" computational methods to tackle "softer" issue such as human motivation is also transformative and can have broad impact by adding to the growing body of literature that shows that such an approach is feasible. Finally, the proposal contains well considered, detailed plans for outreach and education that can change the way in which students approach problems, enabling them, like the PI, to think about how to use computational approaches for psycho-social studies--and to truly think in interdisciplinary ways. Summary Statement As is apparent from my comments in each of the "Intellectual Merit" and "Broader Impacts" category, I find this to be a compelling proposal to do transformative work, and I very strongly suggest that the PI be awarded a CAREER grant. --------------- REVIEW: What is the intellectual merit of the proposed activity? Provides a compelling description of the motivation issues in online social knowledge communities. Does a great job setting up the research story. Great related work coverage throughout the proposal. The four research activities in section 3.2 are interesting, impactful, and build on each other nicely (e.g. using the modeling of the turning points in the lifecycle of a Wikipedia contributor in the agent-modeling work for modeling contribution quality). The reminiscence argument is compelling, though I wished for more discussions of the prototypes (cultural probes) you'll try out. You do a good job pointing out critiques of the memory as data view, life logging approaches, and the use of special purpose devices to capture memories. But this leaves the reader hanging with no concept of what you might build. The Facebook reminiscence study is more well defined, though the last part of this thread also ends with the construction of one or more undefined prototypes. While I understand the need to keep your prototype design open to the qualitative data, I would like to see at least a hand-wavy description of the potential design space for such systems - the easiest thing would be to give examples of prior systems you've built that support reminiscence and describe strengths and weaknesses, or describe other people's systems (that you think work, and why). There's too much "trust me" in the reminiscence part of the proposal. What are the broader impacts of the proposed activity? Understanding and harnessing the motivation mechanisms in online communities can have strong impact in understanding the motivational mechanisms behind successful projects such as Wikipedia and harnessing those mechanisms for other projects. Providing technical support for reminiscence and reflection explores new ways for technologies to touch the messy, human parts of our lives. Well-thought out education plan - goes above and beyond the "default" education plan for CAREER proposals. Summary Statement Intellectually interesting, high-impact proposal. Would be even stronger if the wikipedia motivation and reminiscence stories were integrated and the reminiscence argument had more concrete examples of the technical interventions. ------------- REVIEW: What is the intellectual merit of the proposed activity? The PI proposes two empirical social computing research projects, one on author motivation in user-generated content communities and one on the use of distributed social media in supporting reminiscence. The first project is largely an incremental extension of the PIs already well received research. The second is an innovative, potentially transformative new direction. The primary weaknesses of this proposal are how poorly these two projects hang together and the unnecessarily strong rhetoric of incrementalism. The first study asks good questions and is a reasonable extension of SuggestBot. Minor quibbles were with the overgeneralizations about the similarity of multi-language Wikipedia sites, but the backup plan of examining similar English sites assuaged this concern. I am confident that the PI could continue to do good work in this area that would make progress on these problems of motivation. The second study, apparently completely unrelated to the first, is what captured my imagination - I would love to explore that personalized fortune cookie generator. The PI argues convincingly for the value of such types of systems and provides preliminary results of rudimentary pilot work in the area. The design rationale is superb, but there is little concrete detail about the fieldwork and evaluation (e.g., population sampling is key to this, but not discussed). What are the broader impacts of the proposed activity? The PI was thorough in his engagement of the CAREER objective of integrating research and education. While there was nothing particularly novel in his proposed approach, all of the activities are practical and valuable. During his time at Cornell he has clearly demonstrated leadership in education in social computing, so I am confident that this practical plan can actually be accomplished. Summary Statement In summary, the first project promises continued progress in an important domain, while the second offers a truly visionary rethinking of the role of social media in our daily lives.