Background and research questions

Go to my homepage cc.oulu.fi/~scoats to follow these slides (explore the interactive elements!)

Online language and gender: Small but significant differences in feature frequencies have been observed (e.g. Argamon et al. 2007, Bamman et al. 2014, Coats 2016)

English profanity: Males use more overall (Mehl & Pennebaker 2003) and on Twitter (Bamman et al. 2014, Wang et al. 2014); males use stronger profanities (McEnery 2006)
How is profanity (English and Nordic-language) used on Twitter in the Nordic countries?
Goals are to:

Explore extent of profanity use by gender
Quantify results by country and gender and identify most characteristic words
Use word embeddings to investigate semantic space by gender

Data collection | Twitter Streaming API

627m tweets with place attributes collected globally from the Twitter Streaming API 9 November 2016 – 5 June 2017 using Tweepy (Roesslein 2015)
From these “seed” tweets, identify users from Nordic countries, disambiguate gender, then download user timelimes (from REST API)

Data collection | Location disambiguation

Tweets can have three distinct location metadata values
The user location is a field where many users write their home location (although they don’t have to)
The geo field contains exact lat-long coordinates (if a user enabled GPS)
The place field optionally tags a tweet to a place
BUT user location can be any character string, geo changes when people move around, place can indicate a user went to a place, but also that he/she is tweeting about a place

Data collection | Location disambiguation 2

More accurate geolocation: Consider the metadata from both user location and place fields
1,627 strings corresponding to Nordic place names (cities, regions, country names) were matched with the user location attribute
Alingsås, Alvesta, Aneby, Arboga, Arjeplog, Arvidsjaur, Arvika, Askersund (etc.)
If the most frequent country within the place value for all tweets from that user matched the same country as the country of the user location, user was retained in the data and assigned to that country

Data collection | Gender disambiguation

Twitter users not required to specify gender (no attribute in profile provided for gender)
Tweets have a metadata field in which many users enter their real names, although it is not required (e.g. “Anna Hansson”)

Nordic Name Genderizer: Name frequency information for 15,406 names given to males and 17,856 names given to females obtained from Nordic statistical offices and used to induce a probability distribution for name gender

\[ \small P(name_x \in male) = \frac{\sum{name_x \in male}}{\sum{name_x}} ,\qquad P(name_x \in female) = \frac{\sum{name_x \in female}}{\sum{name_x}} \normalsize \]

author_name entity like “Anna Hansson” -> user classified as female
author_name entities such as = “swedengirl123”, “!!!!ROCKER!!!!” -> ignored

Data collection | Twitter REST API

Timelines (up to 3,250 tweets) were downloaded from the Twitter REST API for users who could be assigned to a Nordic country/territory and whose gender (male/female) could be inferred
Final data for analysis consists of 30,966,040 tweets by 36,001 users comprising 393,304,865 word tokens

Some Sample Tweets containing Profanity

Profanity lists

Lexical-item-based approach
924 potentially offensive English words (with optional plural ending)

“Core” profanities such as shit, fuck, piss, cunt, damn, others such as bollocks, faggot, fuq, heeb, hell, hillbilly, homo, honkey, hussy, jackass, jackoff, jigaboo, lameass, lardass, lesbo, lezzie, limey, limpdick, mcfagget, minge, mooncricket, nigger, paki, pansy, peckerhead, pikey, piss, pussy, spastic, snownigger, twat, whitetrash, wtf, etc.

1,602 potentially offensive Nordic words from Greenlandic, Icelandic, Faroese, Norwegian, Danish, Swedish and Finnish (includes morphological variants)

E.g. arnapalaaq, iteq, nipangerit, aumingi, böllur, djöfullinn, drusla, fífl, fíflingur, mogghøvd, skitni, pupp, pæss, rass, rasshøl, rompe, ronk, runk, ræv, ræva, rævhøl, rævva, bøsserøv, fisse, hestepik, klaphat, kussekryller, lort, fitta, fittig, helvete, jävlä, jävlar, knulla, kuk, huoraa, huorilta, huorilla, huorille, huorista, huorien, huoriin, huorissa, huorat, huoria, kusipäät, kusipäältä, kusipäällä, etc.

Sources of Profanity

English:

Noswearing.com scrape (347 terms), list of 1,383 potentially offensive terms created at Carnegie Mellon University, Pittsburgh, USA

Edited list to retain potentially crude/vulgar/offensive lexical items referring mainly to bodily processes, sexuality, and race/ethnicity
Nordic languages:

Online word lists from software tools created to filter user input on websites (here and here for Norwegian, Finnish, Swedish and Danish), Svensk og Dansk bandeordbog Crowd-sourced Youswear dictionary for some terms in Icelandic, Faroese, and Greenlandic, wiktionary.org, Oqaatsit | Ordbogen, Greenlandic-Danish dictionary, Beygingarlýsing íslensks nútímamáls, Inflectional Dictionary of Modern Icelandic, Íslensk nútímamálsorðabók, Dictionary of Modern Icelandic, Sprotin, Faroese dictionaries, SALDO, the Svenskt Associationslexikon, KORP, Språkbanken’s corpus tool, Ordbog over det danske Sprog, Dictionary of Danish, Bokmålsordboka | Nynorskordboka, Sprakrådet’s online dictionaries of Bokmål and Nynorsk

Use of profanity by country and gender

For each country, the frequency of each of the 924 English profanities and 1,602 Nordic profanities was aggregated according to gender
The log-likelihood measure G was used to compare the relative frequency of the items by gender

\[ G = 2\sum_{i} {O_{i} \cdot \ln\left(\frac{O_i}{E_i}\right)} \]

Items with high G scores (low p-values) are used much more or much less by one gender
Starting with Nordic lexical items

Iceland (Nordic profanity)

Total Nordic-language profanity per 1k words: Females 0.674, males 0.749

Norway (Nordic profanity)

Total Nordic-language profanity per 1k words: Females 0.442, males 0.561

Denmark (Nordic profanity)

Total Nordic-language profanity per 1k words: Females 0.391, males 0.37

Sweden (Nordic profanity)

Total Nordic-language profanity per 1k words: Females 1.583, males 1.775

Finland (Nordic profanity)

Total Nordic-language profanity per 1k words: Females 0.542, males 0.844

Totals, all Nordics (Nordic profanity)

Total Nordic-language profanity per 1k words: Females 0.982, males 1.133 (Includes Greenland, Faroes, and Åland)

Iceland (English profanity)

Total English-language profanity per 1k words: Females 1.951, males 2.266

Norway (English profanity)

Total English-language profanity per 1k words: Females 1.731, males 1.656

Denmark (English profanity)

Total English-language profanity per 1k words: Females 2.58, males 2.073

Sweden (English profanity)

Total English-language profanity per 1k words: Females 1.429, males 1.442

Finland (English profanity)

Total English-language profanity per 1k words: Females 0.996, males 0.995

Total, all Nordics (English profanity)

Total English-language profanity per 1k words: Females 1.549, males 1.493 (includes Greenland, Faroes, and Åland)

Profanity frequency by country and gender

Word embeddings

Linguistic elements that show similar collocational and syntactic distributions often exhibit similar semantics
“You shall know a word by the company it keeps” (Firth 1957)
Word embedding algorithms can be used to model semantic relationships based on the co-occurrence of lexical items within specified word spans

v(King) - v(Man) + v(Woman) = a value very close to v(Queen)

(Mikolov et al. 2013: 749)

Drawing

t-SNE to 2-dimensional space

Each word type in the data was converted to a vector with 400 dimensions
t-Distributed Stochastic Neighbor Embedding is used to reduce the 400-dimensional word vector space to two dimensions for visualization
The algorithm calculates the probabilities that any two word vectors are near to one another in high-dimensional space, then uses the t-distribution to estimate the probability that the vectors are near in lower-dimensional space
Words, including profanities, are now grouped according to similarity of context (and meaning)
Let’s compare males and females

t-SNE Visualization

Blue: 50,000 frequent words (out of the 602,203 unique types in the data). Red: The 300 (of 924) English profanity types that occurred at least once. Green: The 462 (of 1,602) Nordic profanity types that occurred at least once.

Caveats and future outlook

Results not necessarily representative of persons from these countries overall
Differences in rates of use are small and not categorical
Binary gender does not capture significant variation within and across categories
Gender interacts with other identity parameters

Comparability of lists: choice of lexical items
Recall: Extent to which offensive terms are included in lists. Precision: Are the terms really offensive?
False positives (slut, ho, fag, fart, dust, etc.)
Compound word forms
Use stemmers instead of counting inflectional forms (but stemmers are often not good for compound nouns and are not equivalent for different languages)
More comparisons possible (diachronic, language-specific, conversation-specific)
Sophisticated regex needed to capture non-standard variants (b1tch, bi7tch, b.i.t.c.h., b-i-t-c-h, b.1.t.c.h., ßitch, etc.)

Preliminary summary and conclusions 1

Males use more Nordic-language profanities overall (especially words pertaining to Satan and Hell: andskoti, fan/faen/fanden/fjandi, helvete/helvíti/helvetti, jævla, perkele, satan/sataana)
Females use more words for bitch/whore/slut: (drusla, hóra/hora/horan, kælling, luder, tussa)
Females more English-language profanities overall (especially fuck and bitch)
If we consider that: Females use more affect words in general (Newman et al. 2008) and in English-language Twitter from the US (Bamman et al. 2014)
Less “emotional force” is associated with taboo language by L2 speakers (Dewaele 2004, Eilola and Havelka 2010)
For Nordic females, English-language profanities have been “deprofanitized” - they are simply affect words
Profanities in the Nordic languages, however, still exert taboo

Preliminary summary and conclusions 2

In stable sociolinguistic situations, females exhibit lower rates of use of stigmatized language variables (Labov 2001: 266)
In linguistic change from above, women adopt prestige forms at a higher rate than men (Labov 2001: 274)
Evidence for language shift to English?

Thank you for your attention!

Acknowledgements

Also thanks to

Dorthe Larsen (Danmarks Statistik) for Danish name lists
Joar Skott (Statistika centralbyrån, Sweden) for Swedish name lists
Finland’s Center for Scientific Computing for data storage and computing resources

References

Argamon, S., M. Koppel, J. W. Pennebaker and J. Schler. 2007. Mining the blogosphere: Age, gender, and the varieties of self-expression. First Monday 12/9. http://firstmonday.org/ojs/index.php/fm/article/view/2003/1878

Bamman, D., J. Eisenstein and T. Schnoebelen. 2014. Gender identity and lexical variation in social media. Journal of Sociolinguistics 18(2), 135–160. http://onlinelibrary.wiley.com/doi/10.1111/josl.12080/full

Coats, S. 2016. Grammatical feature frequencies of English on Twitter in Finland. In L. Squires (ed.), English in computer-mediated communication: Variation, representation, and change. Berlin: De Gruyter. 179–210.

Dewaele, J.-M. 2004. The emotional force of swearwords and taboo words in the speech of multilinguals. Journal of Multilingual and Multicultural Development 25(2–3), 204–222.

Firth, J.R. 1957. Papers in linguistics, 1934–1951. London: Oxford University Press.

Labov, W. 2001. Principles of linguistic change, vol. 2: Social factors. Oxford: Blackwell.

Lui, M. and T. Baldwin. 2012. Langid.py: An off-the-shelf language identification tool. 50th Proceedings of the Association for Computational Linguistics, 25–30. Stroudsburg, PA: ACL. http://dl.acm.org/citation.cfm?id=2390475

McEnery, T. 2006. Swearing in English: Bad language, purity and power from 1586 to the present. New York: Routledge.

Mehl, M. and J. Pennebaker. 2003. The sounds of social life: A psychometric analysis of students’ daily social environments and natural conversations. Journal of Personality and Social Psychology 84(4), 857–870.

Mikolov, T., W. Yih, and G. Zweig. 2013. Linguistic regularities in continuous space word represen-tations. In: Proceedings of HLT-NAACL 13, 746–751. https://www.aclweb.org/anthology/N13-1090

Newman, M.L., C. Groom, L. Handelman, and J. Pennebaker. 2008. Gender differences in language use: An analysis of 14,000 text samples.Discourse Processes 45, 211–236. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.216.4267&rep=rep1&type=pdf

Roesslein, J. 2015. Tweepy. Python package [Computer software]. http://www.tweepy.org

Wang, W., L. Chen, K. Thirunarayan, and A. P. Sheth. 2014. Cursing in English on Twitter. In: Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing, 415–425.

Profanity on Twitter in the Nordics