Harvard "New Directions in Analyzing Text as Data", 2012
http://projects.iq.harvard.edu/ptr/event/new-directions-analyzing-text-data
Copy of page: http://www.cs.cornell.edu/home/llee/extra/new-directions-analyzing-text-data-2012.html

FRIDAY

1) SECRETS AND LIES (Friday, 9:00-10:30AM)
“How Censorship in China Allows Government Criticism but Silences Collective Expression,”  Gary King, Molly Roberts*, and Jennifer Pan* (Harvard) (paper)

* = presenter 

"largest selective suppression of human expression in history" (by hand, and so is "content filtering", estimated by talking to bloggers, officials, censors: about 200K censors, including about 1K at individual sites like sina.com.  internet polecat, 20K-50K

prevailing wisdom: it's to stop govt criticism.  But they show: allows plenty of criticism, and we understand it incorrectly: it's about stopping collective action and clipping social ties.  (although they also censor porn, criticism of censoring).

In China: soc media is fractured over 1400 sites (but long tail, of course).  

collected 3.6M posts in 85 topics over 6 months, and then random samples for 127K and 11M posts.  Looked for diffs in availability (you get an error page.  Sometimes have a cartoon internet police character Jingjing on the page: so there's no ambiguity about govt intervention.)

Example: posts re: shanghai subway crash.  censoring: 0-1 days after appearing: over 200; for the rest, fewer than 25 removed after that 0-1 day period.  

In contrast, for certain topics (porn, criticizing censorship), censoring seems to be constant (ongoing), in the sense of "published" and "censored" histograms match.

Of course, for most topics, censorship (+ posts) are bursty.So they look for volume bursts, and then go back and try to align with an actual event.

Hypothesis of govt's procedure: consistent monitoring of topic areas; detect volume bursts; check if seem to be related to collective action (perhaps in future); remove all such posts if so.

Policy (e.g., speculation about reversing one-child policy at a national committee level, or govt increasing power prices) and news posts tend to not be particularly censored.  (Although the rush to buy salt was definitely censored)

Automatically compute proportion of posts that are supportive, critical, or irrelevant (since this is what Bayes rule needs to compute P(censored | category)).  Turns out that positive/negative was not a strong factor for predicting probability of censorship.

Ai Weiwei case seems to show: increase in censorship re: topic seems to correlate with an action about to happen (i.e. percentage censored re: Ai Wei Wei increased in the few days before Ai Weiwei's arrest).  This seems ironic seems censorship is supposed to be about suppressing information. Fig 8 shows that posts that support the govt on Ai Weiwei and inner Mongolia are more likely to be censored.

The project makes use of a secret massive-scale real-time blog-post harvesting engine ("presumably even computer scientists would be impressed")

“Finding Deceptive Opinion Spam by Any Stretch of the Imagination,"  Myle Ott, Yejin Choi, Claire Cardie, Jeffrey Hancock (Cornell) (paper)

Thanked Arthur for coming up with such inventive session titles

build on sentiment/subjectivity, learn more about the lang. of deception, ultimately for political speeches/blogs, debates.  (news clip: "Do Americans really care whether they are being lied to")

(Checked review of Harvard Square Hotel (where we're staying) on tripadvisor, and the review came out classified as truthful)

opinion spam: disruptive (in contrast to "obvious" spam, like ads, or not relevant, which are not really that problematic for people in the sense that people can easily tell that they are irrelevant) vs. deceptive (original example: Belkin paid people to write positive reviews: see "Laurie B. Cook"'s review of a certain USB hub on Amazon.  The person who initiated this program at Belkin was fired.)  People are not good at accurately determining which are deceptive from the text.

Since people aren't good at ID'ing deceptive vs. truthful, can't ask people to create gold standard, so created their own fake "realistic and positive" reviews via Amazon Mechanical Turk (at $1/review, which is very high for AMT; took 2 weeks to get 400 such reviews; the Turkers were told that this was for research purposes (after they realized from the first round that Turkers were worried about the purpose); may not have said that it was deception research).  Turkers had to have over 90% approval rating.  

For the real reviews, had to be 5-star reviews and not first-time reviewers, and used the 20 most-reviewed Chicago hotels (since in such cases, there's probably less incentive for people to write fake reviews), and picked reviews so that the lengths were distributed similarly to those of the deceptive reviews.

In most previous work, it's been hard to beat human performance at deception detection.  For them, 3 undergraduate judges (given 80 true, 80 deceptive) accuracies were 53.1, 56.9, 61.9% (only the latter was distinguishable from chance).  Note that the people were probably operating under a truth bias, as can be seen in the recall for deceptive being 18.8, 36.3, 36.3%.  So, again: people are not so good.  (In follow-up work, used hotel.com for real reviews, because for hotel.com you have to have booked a room to submit a review, and so much more likely to be truthful).  (In follow-up work, subjects were told that there's a 50/50 class split, and the people clearly still had a truth bias (they don't count to check their class distribution), although a little milder.  In general, results did not change in the follow-up experiments).

used a linear SVM because it's easier to analyze the feature performance.  Features: inspired by genre identification (imaginative text vs. informative text; 48 part-of-speech features: 73..0% accuracy.  There are various things that don't match the previous findings, particularly, superlatives are associated with deceptive reviews (but aren't supposed to be imaginative); of course, for reviews, superlatives are important), psycholinguistic deception detection work (LIWC, 76.8% accuracy), standard textcat (88.4 - 89.8% accuracy, depending on which features, best including LIWC).

Interesting features: seem to be able to interpret some of the features as spatial difficulties (Vrij et al 2009) and as NOT psychological distancing (cf. Newman et al 2003) (probably the contrast is because of the task, pretending you've been at the hotel)

looking at prevalence of opinions, negative opinions, other domains (doctors), other genres (bluff the viewer (?), NPR)

http://reviewskeptic.com

Concluded with a fake review: Marion Crane for the Bates Motel.

Discussant: Kenneth Benoit (LSE)
"Now it's time for truth"
Commonalities
Pie charts are kind of old school (Pacman; charts that are crap/almost crap/King et al)
"secret methods problem"
Can you be sure you are catching the sites before the censors got there?
Why only 3 subjects, and why are they so inconsistent, and why only POS (not the only thing for genre ID), ditto that LIWC features not synonymous with psycholinguistics
Why are issues and not specific posts ID'ed as collective action.  Why is buying salt "collective" or power price being "news"? Reply: has to be, to help with the coders/censors to do the job, they need instructions.  Reply: buying salt did cause a collective action, so that's how we decided it was collective action; and maybe they want to hear criticisms
Theory seems thin - could have a censorship budget reflecting original preferences on what to censor (so didn't get around to criticism of the government).  And pornography doesn't fit into collective action, by the way.  Reply: we have a reason to believe that the govt wants to encourage an impression of openness.  We are wondering if there's this 50-cent society of people that the chinese govt pays to write positive posts. The more we look, the more we're sure we're right.
About the deception paper again: it is all standard text classification.  The deceptive reviews are not naturally occurring

QUESTIONS from audience: 
Is pornography perhaps political also?  Reply: most govts seem obsessed with porn.  Porn seems more about attracting people to see sites, not porn distributed by individuals
What about self-censorship?  Reply: don't think there's the issue. Well, and also the bursts seem to be contra-indicative.
What about whether things are in a thread or not?
Are there (hotel) reviews that appear and disappear, like the censorship happening on the Chinese social-media sites?  
What are the instructions given to the censors?  Reply: text-analytic methods in the US are definitely used in China
2) AMERICA! (11:00-12:30PM)

“How Words and Money Cultivate a Personal Vote: The Effect of Legislator Credit Claiming on Constituent Credit Allocation,”  Justin Grimmer (Stanford), Solomon Messing, Sean Westwood (paper)

"The impression of influence". Voters don't necessarily know when $ is spent in their neighborhood or the context of the spending.  So how does a candidate show constituents that they are responsible for getting the money?  cutting ribbons, breaking ground, Facebook and tweeting, press releases - credit claiming -> constituent credit allocation -> personal vote.

Political economics (I think): particularistic spending; Representatives (I think): actively engaging constituents and affecting evaluation; Political behavior: constituents evaluate actions, not $

Press release: show that they exercise discretion; use deceptive and selective language.  Will show how credit claiming affects support - it's not just visibility, and more messages are more effective at getting support than the increasing the amount claimed.

Previous theories: credit is related to money spent (contrasting hypothesis: number of new projects/programs).   But people aren't real good at evaluating such things.

People, it is believed, can tell the big differences in messages, but not necessarily amounts.  So they'll be happy to assess actions rather than evaluate expenditures.  Which implies more messages is better than bigger monetary claims.

Experiment: vary type of messages (fluff vs non-fluff), frequency, amount claimed.  Use actual representative's names, and run the experiment with representatives who tend to use social media to claim credit regularly.

Facebook app - participants told it will connect them with their rep.  Or email newsletters.  Run treatment for 5 days (1 msg per day), then get survey results.  Recruitment through AMT (so geographical and political diversity) (and included trap questions to ensure Turkers are paying attention).  High completion rate (like 90%, and unrelated to the covariates).  BTW: could measure click-through rates, but some of the clicks are perhaps b/c of the friends, so click-through rates probably aren't accurate.  QUESTIONS: Could look at the comments attracted, did it diffuse/spread?  (concerns about fake news getting out).  ANSWER: none of this went viral.

Experiment 1 : (a) credit-claiming re: money; (b) ads that don't claim credit about money, but raise awareness about the legislator ("he says he's excited that somebody won an arts award"); this is actually a lot of what they actually talk about, art contests are "huge"; (c) no message sent.  

Check: could they identify their own rep?  In (a) and (b), about 85-90%, control percentage was something like 55%. 
Check: how effective did they think their rep was at delivering $?  Big jump between control and ad, and between ad and credit.
Check: also similar effect for feeling towards the rep.
QUESTION: priming effect in q order?  ANSWER: the randomization probably took care of that.

Experiment 2: 2x2 design: 1 msg or 5 msgs received; at same time, small award ($15K) vs. large award ($1.5M).  (for five days:  $15K, $19K, $86K, $21K, … etc. vs. large magnitude diff.)

Check: select legislator (I think this means, could name/pick their representative): large-high > small-high > large-low > small-low freq;
check: effective at getting $: large-high > small-high > (note/recall 10x differences on amount) large-low > small-low
check: ditto for feeling towards rep, except not stat-sig for large-high and small-high.  
"90 times greater per-$ return for repeated messages"
Check: how much $ do they think was brought to the district?  large-high-freq > large-low > small-high > small-low.

QUESTION: shouldn't you have done this with a pie-chart?

“Textual predictors of bill survival in Congressional committees,"  Tae Yano, Noah Smith (Carnegie Mellon), and John Wilkerson (University of Washington) (paper)

About 4K-6K proposed bills per Congress, about 500-700 (13%) survive to be voted on after getting through congressional committees (which have no set procedures, no roll call votes, basically very little tracking)

Control for metadata: sponsor, committee members, timing, party affiliations, etc.

Related work on relating text to action: Thomas/Pang/Lee 2006, Gerrish/Blei  2011

model: logistic regression with L1 regularization.

data: thomas.loc.gov for bill records and content, committee membership from http://congresionalbills.org .  Aligned the data themselves.

eval: prediction on held-out Congress. Also look at how much each feature has "impact" on the final prediction. (its weight and its frequency)

metadata-only model: 3731 binary features.  Does some reduction in error over majority-class baseline.  Big impact: sponsor in majority party (good), sponsor is Ron Paul (a bit negative)

text model 1, functional categories (trivial, recurring, important): want to approx. the function info seems it's expensive to annotate;  have bit of training data, and trained a text classifier for it as the labels.  Get an improvement when in combination with metadata (chart doesn't show results of function category alone)

text model 2: text similarity to past bills: members should vote the same as they did on similar bills.  Variant: only look at the chair's (proxy) vote.  Variant: only look at the majority members' (proxy) vote.  Adding these features in reduces error again.  Most heavy feature is chair proxy vote.

text model 3: words as features. When added to meta, best gain.  (Just unigrams and bigrams is worse than using metadata alone (I think, didn't quite catch it)).  Only 98 of 24K features really affect the results.  All features together are the best.  (See fig on number of "effective" features vs. the number of features "actually" used)

Note: some bills appear to "die" because they get bundled into bigger bills.  Potentially there are bills sponsored on principle, not because they can survive. 

Data available.

Discussant:  Nicholas Beauchamp (NYU)
The two papers contrast in interesting ways, with black-box analysis in both cases.  CS one is bottom-up: try all possible factors, and select the significant ones; the classification is the goal and substantive implications generated but not independently tested.  Sort of the opposite in the Grimmer et al paper.
How about more topics (created manually or induced automatically) (ANSWER: we tried, decreased accuracy (I think that's what was said)).  Maybe it would be interesting to look at prediction for specific years. ANOTHER ANSWER:  people had looked at fine-grained categories, as well, but people hadn't really answered the question of whether the text matters.  AND: thinking about adding bill lineage in.)  And, can we get the "theories" that arise organized by granularity, or hierarchy, to get a theory?  Also, tried with a different type of feature-selection methodology/algs: found diff sets, and the top 20 variables reduced error by 3 points.(Found post-office-related cluster, btw)
Grimmer didn't mention that centrists do more credit-claiming.  Could it be that centrists can't talk so much about ideology, so they have to engage in credit-claiming instead?  Be interesting to now do as future work more subtle changes (dollar signs vs "I" vs. "build")
Can we marry bottom-up and top-down?
QUESTION: isn't this supposed to be about voter turnout?  And you have their real names and stuff, right?  And maybe the effect will really decline over time.  
ANSWER re more subtle features:  attaching civics lesson like explaining how grants get given; also some other variant: "I secured funding" vs. "announcement of funding".  P Resnik said someone did something about focus being a factor in being less susceptible.
QUESTION: do credit claims get divvied up so that the rep can claim multiple achievements out of one?  ANSWER: yes, there are lots of small pots in order to get an actual amount.  They do piecemeal it: "requested this", "got this", 
3) ON TOPIC(S) (1:30-3:00PM)
“Vanilla Models for Measurement of Topic and Topicky Concepts in Text,"  Margaret H. Ariotti, Muhammed Idris, Burt Monroe*, and Eitan Tzelgov (Penn State)

with examples form "Losers' politics:  parliamentary …."

table of manifest vs latent variables; dimension reduction from continuous to categorial.  Then trying to fit LSI and PLSI into these tables.

If you add "compositional (values add up to 1, i.e., proportions)" to the continuous and categorical set, you get a 3x3 table; and O'Connor/Bamman/Smith called hierarchical admixture models the highest on the scale.  But what about admixture models in  the continuous-continuous square?  Such a thing should exist…

The question: can we build a topic-modeling approach based on factor analysis? (to help better understand LDA)?  When are we getting topics and when are we getting things aren't topics but are topic-y?

In political science, we invested so much in discussing whether there are two-factor roll-call votes, but don't blink at 100-factor topic models.  What's up with that?

Now, factor analysis assumes normals, not the discrete (multinomial-y) counts.

A "factor analytic topicky" (FAT) model. After pre-processing, run the usual factor analysis on the resultant matrix, then rotate to max posit simple structure on loadings (want loadings and scores positively skewed). Note you have a residual topic in the FAT case.

Example w/ lda.  Noting that there seem to be both topic and doc-length factors behind a particular topic that was found.  Some of the LDA and FAT topics correlate well.   

"Topics" should give us this projected skew (it's topic 1, it's topic 2, or nothing), so when you get projections that don't look like "everything's on the x or y axis", that "topic" isn't very topicky at all.  There are even examples of sine-wave-y things (where x  = time)

A theory on trying to emphasize frames where the opponent looks divided and weak.  Frames (values) can be like topics, perhaps, unevenly distributed.  

In discussions, can see frames and response to frames over time.

“Topic-Specific Communication Patterns from Email Data,” Bruce Desmarais, Peter Krafft*, Hanna Wallach, James ben-Aaron, and Juston Moore  (UMass-Amherst) (paper)

Talk about data visualization to get patterns, in data where there can be multiple interactions (e.g., email) btwn parties.  How shall we picture these?

Subnetworks, e.g. Mitt Romney has "governor (blue speech to MA)" vs "President (red speech to US)" spaces, and these are communication patterns.

topic: latent space; email text described by LDA; recipients chosen according to mixtures of latent spaces (e.g., I'm talking about parks, but I also always cc: my manager).

Note: recipients are assigned to topics based on the topics that are exhibited in the mail.  

In NC, emails of managers in govt are publicly accessible.

Showed how agents fell in the 2-d space for a given topic, with size of label being amount of belonging or communication there. (I think)  THere was clustering among groups close in the org chart.

Discussant: Scott Moser (Texas)

Connecting latent class models to latent space models.  Maybe linking these can help with defining topics.
Docs are or live in the links, here.
Tversky on whether similarity is really symmetric ("cuba is like ussr; ussr is like cuba")

QUESTION: thinking about the fact that one gets negative/positive/zero, vs. LDA needing to be 0 or non-zero.  So you can "hide" two topics in one in a way that's visible.  So does the comparison against LDA number of topics for FAT same number of topics, is it fair?  ANSWER: not trying to say that one was better than the other, anyway.  

QUESTION: why emphasis recipient vs sender in model? ANSWER: author influences choice of topic

QUESTION: no thread structure?  Like initiating a thread is different and might influence topic.   QUESTION: recipient could be both To: and cc: . QUESTION: what is the content of these public emails?  ANSWER: some is frivolous "what kind of tree is that", event invitations.  QUESTION: do you see any cases where the people indicate, "let's take this offline" or affect what they talk about ANSWER: probably affects; every participant in this dataset knows the email is publicly available.

ANSWER: can get both email and meeting minutes (so can see what's in the meeting conversations (i.e., is offline).   

Q: what's the difference btw a topic and a frame?  ANSWER: conceptually: frame is aspect of topic that you want to have emphasized vs. other aspects.  And some appear across topics, like "cost". Q: so maybe values occur across topics.  Like liberty/fraternity/equality. AFTER MORE DISCUSSION: ANSWER: the number of topics I've seen in the literature range from 3 to 600.  Q: mustn't frames be independent of the speaker, to be just in the text. Q: how about: frames as lexical (death tax vs estate tax), vs framing as second-level agenda setting (so sub-topics in the second-level can line up with frames)

4) ARGUABLY... (3:30-5:00PM)
“The Exaggerated Life of Death Panels: The Limits of Framing Effects on Health Care Attitudes", Dan Hopkins (Georgetown) (paper)

Identify elite frames in health care, vs public word choices.  Elite frames are far more punctuated (bursty) than public opinion.  This implies an issue about talking about framing as being critical.  Elite framing has limited but discernible effects on subsequent public word choice. 

Issue frame: yes, people have been talking about this for a long time; 1993 article on framing a fractured perspective (or something).  Let's call it a rhetorical structure calling attention to a subset of relevant considerations (Kinder 1998,  Chong and Druckman 2010), and so is indeed like a sub-topic. And, distinctive vocabulary items are tell-tale signs.  

Let's move from the traditional survey experiments to external validity (so, don't assume that elites have complete freedom in choice of frame, for example; also, take into account fact that people are trying to get stuff that resonates with the public, and so may choose frames from public discourse)

Decided to avoid open-ended responses b/c of sparseness and colloquial language; instead stick to elite lang, with press releases. 1488 on health care by senators in 2009, 2010.  Held-out data: press appearances.  

Can see, for example, usage of the (LDA) topic "medicare" over time (divided by republicans (procedure, tax increase, jan '10 medicare spike) vs. democrats (public option wanes, emphasize extended coverage)) having behavior correlating with our understanding of public events.  Other clusters, as usual, seem like amalgamations of things, like procedural lang.

Then, for open-ended responses in 7 healthcare-reform surveys july 2009 to nov 2011 (so, some of this is before they'e heard public figures talking about it a lot).  And can we see if reasons change, even if opinions' "polarity" don't - so, plot supporters and opposers separately?

Comparing public vs open-ended.  For 100 stems that occur in both, calculate distance btwn press release language, and language in prior open-ended responses; or against subsequent open-ended responses.  There are cases where the Democrats introduced language tha was adopted by the subsequent public responses.  In general, some evidence, despite the fact that public lang seems pretty constant, that there is some uptake from elites.  "death panels" is almost never used by members of congress or the public, so influence of (the phrase) "death panels" and of framing may have been exaggerated.

1965 Donovan Ward, AMA Prez: said things about the world is taking steps toward socialized medicine: govt control, other tropes that seem to be used again now.

“ ‘I Want to Talk About, Again, My Record On Energy...’:  Modeling Control of the Topic in Political Debates and Other Conversations,”  Philip Resnik with Jordan Boyd-Graber and Viet-An Nguyen* (University of Maryland) (paper)

Who is setting the agenda? Cohen 1963: press is successful in telling readers what to think about, although not what to think.  Meraz 2011: social media is weakening traditional media's power.  

Here, agenda will be identified with topics.

Look at topic flow from turn to turn in conversations.  In model: Mix of topics from turn before can be preserved for the next turn, depending on probabilistic choice, who's next speaker ,and what's their tendency to change the topic.

Data: ICSI meetings (topic annotations), 2008 (topic annotations) and 2012 debates, Crossfire (influencer annotations), and Wikipedia discussions (influencer annotations).

First, let's evaluate w.r.t segmentation, as a reality check.  This work allows conversations to share hierarchical topics, learn the number of topics.  Cf. Purver et al 2006, Fox et al 2008.

In the Crossfire data, any videos are treated as another speaker.  It looked like the top switchers who aren't otherwise explainable (e.g, hosting) seem to be moderates.

october 3 debate.  Romney: "Let's get back to Medicare." Lehrer: — before we leave the economy —.  Romney: "Let's get back to Medicare".  Lehrer: No, no, no, no ---.

Boydstun et al in prep: Changing the topic in a debate is an outcome of a cost-benefit analysis, where you weigh "looking like you dodged the question" against "talking about what you want to talk about".  So, crowd reaction is important.  Resnik and Boydstun collaborated on ReactLabs for 3800 students participants (across the country) for the Oct. 3 debate.

Discussant: Justin Gross (UNC-Chapel Hill)
shared themes: temporal issues, framing.  

Hopkins addresses the causal challenge, and doing this outside of the lab, and in looking at shifts in language (and this could be a newer notion/conceptualization  of influence)  Could this be extended to a broader ecosystem of actors?  Could one have a gatekeeping theory (stuff getting filtered by the elite media, vs. elite media trying to influence politicians, and then the addition of social opinion media … so are we looking at the death of gatekeeping)?  Is LDA overkill for short docs … like sentences?

can using conversational dynamics be used to "denoise" the data to remove the sort of procedural stuff (like banter, or stuff we just say that isn't relevant).  The fact that speaker ID influences topic evolution: can that be extended to actor clusters or media types?  

QUESTION: maybe the congresspeople are also really downstream, too, so they aren't actually the influencers, but it's Karl Rove that everyone's picking up.

Q: is there enough data for public opinion ANSWER: text is the only record we have for historical opinion.

How about informative priors, or interactive LDA (reference to a Boyd-Graber paper)

Q: could the fact that the elite media is volunteered, whereas public info was elicited, be a factor?  So get public info that's volunteered?  A: got blog data, but are bloggers elite or public?

5) RISE OF THE MACHINES (Saturday, 9:30-11:00AM)
 “Automated Coding of Political Events from a Very Large Text Base,”  Phil Schrodt* and Jay Yonamine* (Penn State) (paper)

People who might be interested in event data: academics and policy analysts (interested in prediction); economists/financial analysts (since there can be economic effects to events)

Coded for material conflict, verbal conflict, and material/verbal cooperation

Challenges: previously, didn't have geographical info at finer granularity than the state. (Gene Thorpe visualization of civil way casualties, with manual analysis) ("do rivers impede the diffusion of violence?")

“Crowd-sourced data coding for the social sciences: massive non-expert coding of political texts,”  Kenneth Benoit (LSE), Drew Conway (NYU), Michael Laver (NYU), and Slava Mikhaylov (UCL) (paper)

"97.36% of political science relies on made-up (coded) data", based on a single-point estimate from the author.
Crowdflower for managing the coders1

The experts don't agree that often on individual sentences.

pushed the job out at 4pm US time, which may have changed the demographics of who responded

crowd-sourced results don't use the extreme labels as much as the experts do

This work is more like  "science" because the data-generation process is replicable

Discussant: Amber Boydstun (UC Davis)
These are excellent groups/papers, and both are also great models of "how-to" papers.  

Categories of frames: can be issues frames (human obligations vs states' rights vs loggers' jobs, which all seem like sub-topics, second-level agenda setting),  global (moral vs legal vs economic -- categories that transcend particular topics), syntactic (logging endangers owls vs owls are endangered (which doesn't imply causality), lexical framing (pro-life, death tax), episodic vs. thematic (what is happening to a specific family, vs. thematic way), equivalency (2/3 owls will be saved vs 1/3 owls will be killed) 

Many of these can morph across media and time

ANSWER: every time d-day rolls around, because of movie reviews and commemorations and stuff it looks the US has just gone to war
ANSWER: "get-another-label"
ANSWER: paying more gets your data faster, but not better data.  "quasi-sentence" : part of sentence said to express an independent/different thing - segmentation into quasi-sentences is hard, and they thought natural sentences got you the same labels anyway.

6) HOW TO WIN FRIENDS AND INFLUENCE PEOPLE (11:30AM-1:00PM)
	1.	“Echoes of power:  Language effects and power differences in social interaction,” Cristian  Danescu-Niculescu-Mizil, Lillian Lee, Bo Pang, and Jon Kleinberg. (paper)
	2.	“Using Friends as Sensors to Detect Planetary-Scale Contagious Outbreaks,”  Manuel Garcia-Herranz, Estaban Moro Egido, Manuel Cebrian, Nicholas A. Christakis, James H. Fowler (UC San Diego) (abstract)
Discussant: Will Lowe (University of Mannheim)
No notes (I was otherwise occupied :) .  But here's a link to my slides.

 
7) METHODS ROUNDTABLE (2:00-3:30PM)
“A RegEx Machine: a tool for Humanists”  Peter Bol, Shih-pei Chen, and Elif Yamangil (Harvard) (paper)

1 year for 8 editors to process 5000 short Chinese biographies. Users can graphically design patterns, and the system has a user-friendly interface.

Giving term lists is intuitive.  (e.g., list of reign period titles, Chinese numbers) and also delimiters and non-delimiters are easy to express

“Knowing Where and How Criminal Organizations Operate Using Web Content,”  Michele Coscia and Viridiana Rios (Harvard) (paper)

Focus on South/Central America.  Query-based approach.

“Summarizing topical content with word frequency and exclusivity,"  Jon Bischof* and Edo Airoldi (Harvard) (paper)

interpretability is often viewed as an accidental byproduct of dimension reduction.  And frequency and "evocativeness" are not necessarily linked (i.e., relative frequency may not be sufficient summary of topical content).  Airoldi et al '06 use "word exclusivity" for authorship, like whilst vs. while.  So, want words that are common in topic of interest and not common in other topics.

Claim: SAGE (Eisenstein et al 2011) is biased towards rare words because differential use is not regularized

Start with data that has been labeled (Reuters) to check real topics and their word distributions. 

Hierarchical Poisson Convolution (HPC) model: model usage rate of word f in topic k, log usage rates for word across topics coming from Gaussian diffusion down known topic hierarchy

usage rate in child topic is parent rate with error

FREX score is harmonic avg of rankings in exclusivity space and frequency space.  Acts like a minimum

“Improving Robustness in Qualitative and Quantitative Content Analysis: A three stage model,”  Aude Bicquelet*  and Katja Mirwaldt(LSE) (paper)

Analysis of parliamentary debates about the use of referendums.  

Why should citizens be denied the right to vote on political matters of crucial importance?  (Budge 1996, Barber 1989) advocate more public participation.  Bowler et al 2007 survey: people would like to have more opportunities to take part.

Data: 10 parliamentary debates about the use of the referendum in House of Commons

Claims: very little variations in the framing, content or frequency of arguments from 1974-2010 --- so people are always saying the same thing.  # stages:  identification and quantification of words (software: Alceste), latent content analysis (QDA miner) , triangulation
 (apply coding scheme developed in stage 1 and 2 to new corpus)

case against:  citizens are ignorant (Schumpeter, 1943); referendums are inconvenient (Dahl 82) and dangerous (Sartori 1985); jeopardize parliamentary sovereignty (Burke, 1778)

responses: 
(1) undermines legitimacy of the democratic model (Miller '83), legislators lack expertise (Haskell 2001), education/info access is now great Backer 1991
(2) practical issues are no longer relevant ('electronic republicanism, Grossman 1995; Strong democracy, Barber 1982; Teledemocracy, Toffler 1995)
(3) they can coexist

qs: have arguments changed?  does party affiliation influences arguments against referendums?  (one more)

results: institutional arguments dominate the debate.  Other recurring arguments: the wording of the question, expertise, and something called "instrumentalization"