class: inverse, center, middle background-image: url(data:image/png;base64,#https://cc.oulu.fi/~scoats/oululogoRedTransparent.png); background-repeat: no-repeat; background-size: 80px 57px; background-position:right top; exclude: true --- class: title-slide <br> ## <span style="color:black;-webkit-text-fill-color: orange;-webkit-text-stroke: 1px;">Civic Engagement with Local Government Videos: Comparing YouTube Transcripts with User Comments</span> <br><br><br> <span style="color:yellow;">Steven Coats</span><br> <span style="color:yellow">English, University of Oulu, Finland</span><br> <a href="mailto:steven.coats@oulu.fi">steven.coats@oulu.fi</a><br> <span style="color:yellow;">9th CMC-Corpora Conference, Santiago</span><br> <span style="color:yellow">September 29th, 2022</span><br> <br><br><span style="font-size:8pt;float: right"><a href="https://i.cbc.ca/1.6424620.1650474599!/fileImage/httpImage/image.jpg_gen/derivatives/16x9_780/town-of-raymond.jpg">image source</a></span> --- layout: true <div class="my-header"><img border="0" alt="W3Schools" src="https://cc.oulu.fi/~scoats/oululogonewEng.png" width="80" height="80"></div> <div class="my-footer"><span>Steven Coats&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;Comparing YouTube Transcripts and Comments | CMC-Corpora 9, Santiago</span></div> --- exclude: true <div class="my-header"><img border="0" alt="W3Schools" src="https://cc.oulu.fi/~scoats/oululogonewEng.png" width="80" height="80"></div> <div class="my-footer"><span>Steven Coats&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;Comparing YouTube Transcripts and Comments | CMC-Corpora 9, Santiago</span></div> ## Outline 1. Background, YouTube ASR captions files, data 2. Methods: Sentiment analysis and topic modeling with transformer models 3. Preliminary results: 4. Caveats, summary .footnote[Slides for the presentation are on my homepage at https://cc.oulu.fi/~scoats] <div class="my-header"><img border="0" alt="W3Schools" src="https://cc.oulu.fi/~scoats/oululogonewEng.png" width="80" height="80"></div> <div class="my-footer"><span>Steven Coats&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;Comparing YouTube Transcripts and Comments | CMC-Corpora 9, Santiago</span></div> --- ### Background YouTube as a platform for mediated quasi-interaction with three communicative levels <span class="small">(Bou-Franch et al. 2012, Dynel 2014)</span> - First level: Face-to-face spoken interaction - Second level: Corresponds to classifications of mass media - *Third level*: Additional modalities/affordances of platform (e.g. commenting) Mediated interaction can be important for societal stakeholders such as companies, organizations, and governments - How do people interact with content uploaded by local governments: what do they like and dislike? --- ### Prior studies of YouTube comments .pull-left[ - Quality of comments <span class="small">(e.g. Goode et al. 2011)</span> - Typological classifications <span class="small">(e.g. Herring & Chae 2021, Häring et al. 2018)</span> - Sentiment of comments <span class="small">(e.g. Ksiazek 2018)</span> - Like ratio vs. text of comments <span class="small">(e.g. Schultes et al. 2012, Siersdorfer et al. 2014)</span> This study: **Transcripts of videos vs. comments** - First large-scale comparison of discourse content of the videos and comments? - Exploratory study to be developed ] .pull-right[  ] --- ### Data: transcripts Corpus of North American Spoken English ([CoNASE](https://cc.oulu.fi/~scoats/CoNASE.html)): 1.25b-word corpus of 301,846 word-timed, part-of-speech-tagged Automatic Speech Recognition (ASR) transcripts <span class="small">(Coats forthcoming a)</span> - Mostly transcripts of meetings and other local government content - Freely available for research use; download from the [Harvard Dataverse](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/X8QJJV)  --- ### Focus on regional and local government channels Many recordings of meetings of elected councillors: advantages in terms of representativeness and comparability - Speaker place of residence (cf. videos collected based on place-name search alone) - Topical contents and communicative contexts comparable - In most jurisdictions government content is in the public domain --- ### Data: comments - For all videos in CoNASE, retrieve all available comments with [youtube-comment-downloader](https://github.com/egbertbouman/youtube-comment-downloader) - = 190,079 total comments, for 20,965 videos (6.95% of CoNASE videos), 116,009 unique users, 5,334,096 word tokens - Most of these videos have few views/likes/comments - Local government does not engage people as much as music videos, video game streaming, makeup tutorials and other popular YouTube content --- ### Sentiment - Transformer models outperform "bag-of-words"-based sentiment analysis - YT comments are rich in emoji, so sentiment models need to include emoji  Model: Twitter-roBERTa-base, trained on ~124m tweets from January 2018 to December 2021 <span class="small">(Loureiro et al. 2022, Barbieri et al. 2020)</span> - BERT-derived transformer model processing pipelines can typically only handle texts up to 512 tokens long - Code to chunk transcripts, assign values to chunks, take mean values - Assign sentiment values in the range 0 (negative) to 2 (positive) to all video transcripts and all comments --- ### Topic Modeling BERTopic <span style="small">(Grootendorst 2022)</span> - Groups lexical items and documents together in "topics" - A form of dimensionality reduction that can give insight into discourse/content of text data - BERTopic uses embeddings from sentence transformer models (take word context into account) ```python from sklearn.feature_extraction.text import CountVectorizer vectorizer_model = CountVectorizer(ngram_range=(1, 3), stop_words="english") topic_model = BERTopic(language="english", embedding_model="all-MiniLM-L12-v2", vectorizer_model=vectorizer_model, nr_topics="auto") topics, probs = topic_model.fit_transform(list(conase_subset.text)) ``` - Used [all-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2), a model trained on 1.7 billion words of web texts from various genres --- ### Research questions - What kind of discourse content is represented in the videos? - What does topic modeling tell us about the content of the transcripts? - Which content attracts positive/negative comments? --- ### 8 largerst topics (transcripts) <div style="text-align: center;"> <iframe style="text-align: center" src="8_topics_words_e.html" width="100%" height="500" id="igraph" scrolling="no" seamless="seamless" frameBorder="0"> </iframe> </div> --- exclude: true ### Topics <div style="text-align: center;"> <iframe style="text-align: center" src="plotly_c.html" width="100%" height="500" id="igraph" scrolling="no" seamless="seamless" frameBorder="0"> </iframe> </div> --- ### Topics interpretation .large[ .pull-left[ - Firefighting - City/community - Waste disposal and management - Art ] .pull-right[ - Police - Sports - Discourse (?) - City/discourse (?) ] ] --- ### Sentiment of transcripts by topic  --- ### Sentiment of comments by topic  --- exclude: true ### Data collection and processing - Identification of relevant channels (lists of councils with web pages -> scrape pages for links to YouTube) - Inspection of returned channels to remove false positives - Retrieval of ASR transcripts using [YT-DLP](https://github.com/yt-dlp/yt-dlp) - Geocoding: String containing council name + address + country location to Google's geocoding service - PoS tagging with SpaCy <span class="small">(Honnibal et al. 2019)</span> --- ### Mutual love score For a given channel with `\(n\)` videos, each of which has `\(m\)` comments, the mutual `\(love score\)` is the mean of the transcript sentiment times the mean of the comment sentiment: `$$love~score = \frac{1}{n}\sum_i^n{st_{i}} \cdot\frac{1}{m}\sum_j^m{sc_j}$$` - Ranges from 0 (negative) to 4 (postive videos and positive comments) --- exclude: true |transcript|transcript sentiment|comment|comment sentiment|LR| |----------|-------|--------------------|-----------------|--| |That's wonderful!| 2|I love this video|2|4| | | |amazing thank you!|2|| | | |the best!!|2|| |We love you!|2| This video sucks|0|2 | | | | I don't know |1|| | | | Love you too!|2|| `$$lovefest~ratio = \frac{1}{2}(2+2) \cdot \frac{1}{6}(2+2+2+0+1+2) = 2 \cdot \frac{9}{6} = 3$$` --- ### Lovefest ratio <div id="htmlwidget-3a7146d716afa79228b8" style="width:100%;height:auto;" class="datatables html-widget"></div> <script type="application/json" data-for="htmlwidget-3a7146d716afa79228b8">{"x":{"filter":"none","vertical":false,"data":[["1","2","3","4","5","6","7","8","9","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31","32","33","34","35","36","37","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63","64","65","66","67","68","69","70","71","72","73","74","75","76","77","78","79","80","81","82","83","84","85","86","87","88","89","90","91","92","93","94","95","96","97","98","99","100","101","102","103","104","105","106","107","108","109","110","111","112","113","114","115","116","117","118","119","120","121","122","123","124","125","126","127","128","129","130","131","132","133","134","135","136","137","138","139","140","141","142","143","144","145","146","147","148","149","150","151","152","153","154","155","156","157","158","159","160","161","162","163","164","165","166","167","168","169","170","171","172","173","174","175","176","177","178","179","180","181","182","183","184","185","186","187","188","189","190","191","192","193","194","195","196","197","198","199","200","201","202","203","204","205","206","207","208","209","210","211","212","213","214","215","216","217","218","219","220","221","222","223","224","225","226","227","228","229","230","231","232","233","234","235","236","237","238","239","240","241","242","243","244","245","246","247","248","249","250","251","252","253","254","255","256","257","258","259","260","261","262","263","264","265","266","267","268","269","270","271","272","273","274","275","276","277","278","279","280","281","282","283","284","285","286","287","288","289","290","291","292","293","294","295","296","297","298","299","300","301","302","303","304","305","306","307","308","309","310","311","312","313","314","315","316","317","318","319","320","321","322","323","324","325","326","327","328","329","330","331","332","333","334","335","336","337","338","339","340","341","342","343","344"],["<a href=https://youtube.com/channel/UCiHirW3MvrAV0-WdyLGfo_g>TownofRaymondVideos<\/a>","<a href=https://youtube.com/channel/UC_wmu2nBMpvzGJe1JSpFR5w>Summerland Chamber of Commerce<\/a>","<a href=https://youtube.com/channel/UCwHm1fTMp6kfNN9owU62LYQ>Houston Community College<\/a>","<a href=https://youtube.com/channel/UC6UORZ8U7NlaS3giNhyuAaA>PlattevilleWISC<\/a>","<a href=https://youtube.com/channel/UC4d7alg3ierVtPeghO_RpNg>Village of Schaumburg<\/a>","<a href=https://youtube.com/channel/UC5pGOg0PI8_MuUIJScsw6Gw>City of Cordova Alaska<\/a>","<a href=https://youtube.com/channel/UCiAVZN6799LJ7QAOGqqx_tw>Tourism Squamish<\/a>","<a href=https://youtube.com/channel/UCFg6-3XPBFETE7Wm6ijhrnw>Guelph Arboretum<\/a>","<a href=https://youtube.com/channel/UCC774tkGUZw8xoZtHhuQUsQ>Official Westchester Gov Videos<\/a>","<a href=https://youtube.com/channel/UC6bhx-Ri1DxvddOmAEGGbbw>Glenview Television<\/a>","<a href=https://youtube.com/channel/UCWyXkxEpHlUgmJOVxe9OtvA>Discover Flagstaff<\/a>","<a href=https://youtube.com/channel/UCR9afQAQb59-KiCKgutK_Ig>City of South Bend<\/a>","<a href=https://youtube.com/channel/UCv9lglsTl-dC1EqQ1E6rEmA>ElmhurstTV<\/a>","<a href=https://youtube.com/channel/UChloArUJjy-0MEljGN-T3FQ>Anderson Community Schools Corporation<\/a>","<a href=https://youtube.com/channel/UCi-DSvLhqIXrjmd9ZAkeQTg>The Town of High River<\/a>","<a href=https://youtube.com/channel/UCVxFLCImmHgKXs9RQkY6OkA>City of Dublin, CA<\/a>","<a href=https://youtube.com/channel/UCvAeUOT0Tzb3lS-sNap5oqA>FCPSMaryland<\/a>","<a href=https://youtube.com/channel/UCVBqwcL4AUmbkFF4lhlqPkA>Lethbridge School Division<\/a>","<a href=https://youtube.com/channel/UCssXh0qK126T6EnSKbtGrag>Tourism Winnipeg<\/a>","<a href=https://youtube.com/channel/UC32PBJYL6zl5zT1Z5QE8ocg>Charles County Government<\/a>","<a href=https://youtube.com/channel/UCEiZD35fzTujMhE4v59JGDQ>Sun Peaks Resort<\/a>","<a href=https://youtube.com/channel/UCtSJN8mcCBqY7Sbl49o0z0g>Monona Community Media<\/a>","<a href=https://youtube.com/channel/UC-1o28ZQVfsUWSqjd3lrVoQ>White Rose Community TV<\/a>","<a href=https://youtube.com/channel/UCZPdzYPLYRG-jSEy0Pq24QQ>Town of Kiawah Island<\/a>","<a href=https://youtube.com/channel/UCE04COwGlYYX0C3O8PPTx6Q>Columbia Housing Authority, Columbia, MIssouri<\/a>","<a href=https://youtube.com/channel/UCtb2hrPZ0ffRKGqyn_nFAkw>Centertexas<\/a>","<a href=https://youtube.com/channel/UCGl4hREwgcCQxKA52C7Y_Bg>RevereTV<\/a>","<a href=https://youtube.com/channel/UCECdhaQYVRJFozV3M25gUNQ>UniversityofDC<\/a>","<a href=https://youtube.com/channel/UC55upj7VD0p945dtBDWuGQg>Dutchess County<\/a>","<a href=https://youtube.com/channel/UCYtTkjboB8fxZKJGGkib7pA>Northern Lights College<\/a>","<a href=https://youtube.com/channel/UCil088ma5qFsgtgtKdgDgcg>LeeCountySchoolsNC<\/a>","<a href=https://youtube.com/channel/UCEwNH7PIM0c5u7yTnTDoSWw>Benton County<\/a>","<a href=https://youtube.com/channel/UC5nX-LSeYEShSFAPi1uVBCw>Town of Benson<\/a>","<a href=https://youtube.com/channel/UCE4TqOfgH8pFyfded-NciHg>CityofNixa<\/a>","<a href=https://youtube.com/channel/UCe8bTnymA2RZO6HYuO2Go0Q>St. Cloud Area School District 742<\/a>","<a href=https://youtube.com/channel/UCX8sT5Da66OZ_R-Le0JEUag>Parkway Schools<\/a>","<a href=https://youtube.com/channel/UCegM3TAyGPayKBUHIeQCv6w>City of Greeley Government<\/a>","<a href=https://youtube.com/channel/UCVtruSbTJikRjJdq0AO-_aQ>FlorenceTourism<\/a>","<a href=https://youtube.com/channel/UCx25HBJycq4TzfYDp8faR9Q>City of Owensboro<\/a>","<a href=https://youtube.com/channel/UCDkHqlB8DCVXibOOz_70ejQ>Spartanburg School District Six<\/a>","<a href=https://youtube.com/channel/UCAnCxVpCxAZZ9oRHW2UEk4Q>District of Clearwater<\/a>","<a href=https://youtube.com/channel/UCbQgiv99M9_Z6C30kXkfIfg>GRPSMyChoice<\/a>","<a href=https://youtube.com/channel/UCBkqExeIWUVfxF8rb74wtYg>D15 Communications<\/a>","<a href=https://youtube.com/channel/UC93ODVpBYeYlFhX4FboTjFA>Town of Neepawa, Manitoba, Canada<\/a>","<a href=https://youtube.com/channel/UCI3p9c__3qlEFi4iaQTOQaw>cityofhialeah<\/a>","<a href=https://youtube.com/channel/UC3zb-uPkLIJLzDii5G6xyNQ>PAISD Videos<\/a>","<a href=https://youtube.com/channel/UCGCK5sar0hUfuyKCO45BxJg>PasadenaISDTX<\/a>","<a href=https://youtube.com/channel/UC3N3m3B_Gx_nI4xjXRBqC_A>townofhighlevel<\/a>","<a href=https://youtube.com/channel/UCqUs1Uf0CtlAXcglZ5biJIw>TriCityMedCtr<\/a>","<a href=https://youtube.com/channel/UC7xxj-wYUd_oXYTcRk3jt8A>City of Rancho Cordova<\/a>","<a href=https://youtube.com/channel/UCY6C_wy8ju5hCODHdw4jYKg>Tift County Schools<\/a>","<a href=https://youtube.com/channel/UCMvAXLKfz93ZcGSTTXs2gJw>Huntsville Hospital<\/a>","<a href=https://youtube.com/channel/UCpZ77HlosYBDLPg-ahmKjWg>The City of South San Francisco<\/a>","<a href=https://youtube.com/channel/UCIPY7GVrwvTg_GPW3Rk08ig>Northern Development Initiative Trust<\/a>","<a href=https://youtube.com/channel/UC1zSezhaOcfdCCksoJPWyaA>Port of Prince Rupert<\/a>","<a href=https://youtube.com/channel/UCJvoKfg-P_xNBlJ7qR0lrjQ>APSExpectGreatThings<\/a>","<a href=https://youtube.com/channel/UCvXe4nswSUEaYrpwGk5KLIg>BellevilleCityHall<\/a>","<a href=https://youtube.com/channel/UCC0zQr_S1GiFE93EaFYjtsg>City of Bay City, TX<\/a>","<a href=https://youtube.com/channel/UCTgU_rclEQSW7XN-sgaCUBA>cityofsantafe<\/a>","<a href=https://youtube.com/channel/UC6E4c_jmhJPH7kR_GKnbpug>Rochester Videos<\/a>","<a href=https://youtube.com/channel/UCz9k8mxdzKOgnMbtMws7-VQ>Farmersville Unified School District<\/a>","<a href=https://youtube.com/channel/UC4iuKAYjenuKmt14TfZ927w>rpsschools<\/a>","<a href=https://youtube.com/channel/UCilFdbYCgeAlHqmoJRNogLw>Owego Apalachin Central School District<\/a>","<a href=https://youtube.com/channel/UCnBWxZ_9UWfQqoN05fiJaTg>City of Garden City, Kansas<\/a>","<a href=https://youtube.com/channel/UCBtF7lVsrY67_4AYQijfaIQ>DeKalbCountyGov<\/a>","<a href=https://youtube.com/channel/UCN-g_PdEzAYdE3dz0zaBKvQ>Granite Schools<\/a>","<a href=https://youtube.com/channel/UCtVmECDLM4GUXigXVBivC0A>Town of Peace River<\/a>","<a href=https://youtube.com/channel/UCbi5NfIOiul4U1H6pWcpBPQ>Whistler Blackcomb<\/a>","<a href=https://youtube.com/channel/UC_lZ_-d_lro16DEKRDnmWHA>Manitoba Hydro<\/a>","<a href=https://youtube.com/channel/UC2YlFTXECf4WMJ9IDylPxIg>Laurel TV<\/a>","<a href=https://youtube.com/channel/UCwSOXNasblJx1LKFge7ItkQ>City of Sidney, Ohio<\/a>","<a href=https://youtube.com/channel/UCqdM7PpJ84RvkqM87OzMAzw>City of Boston<\/a>","<a href=https://youtube.com/channel/UC5CMOTcmbImPexqczAXtqLg>Metropolitan Transit System<\/a>","<a href=https://youtube.com/channel/UCU7TFJrYzVyija6RC24dt3g>City of Surrey<\/a>","<a href=https://youtube.com/channel/UCuOlvuIIZLA05pfGbQ7o74Q>Harrisonburg Virginia<\/a>","<a href=https://youtube.com/channel/UCCiv_ztIR-N6SPONzaq0POA>CityOfIndependenceKS<\/a>","<a href=https://youtube.com/channel/UCOc4d14_JwVyb1pKUajsNEQ>Seminole County Government<\/a>","<a href=https://youtube.com/channel/UCuAklWn62xvWRerWFcv7t2A>Port Houston<\/a>","<a href=https://youtube.com/channel/UCZYKgaQJ-6R527CA2XNwB2w>Rush-Henrietta Central School District<\/a>","<a href=https://youtube.com/channel/UCQn4fnzRwHD5wPIuKpMUFbg>City of Brush!<\/a>","<a href=https://youtube.com/channel/UCv6ZxBRP7QyFEUruyAOYGlA>Hawaiian Gardens<\/a>","<a href=https://youtube.com/channel/UCRyTPtc3XEWCGGpT7ddmcJg>CityofLeagueCityTX<\/a>","<a href=https://youtube.com/channel/UCkoFvgazdqJtlEd8GkXTK8g>HCCFlorida<\/a>","<a href=https://youtube.com/channel/UC-shBGI5srOmks3p-z7ZZww>FWISDnews<\/a>","<a href=https://youtube.com/channel/UCeAOLKK_0fl-FiSAAzTB7rw>CityTVLakewoodCA<\/a>","<a href=https://youtube.com/channel/UCrIwOAePHp4jhDj4ogxonyQ>Capitol Corridor<\/a>","<a href=https://youtube.com/channel/UCz2nkGSI-aLX4U8sz8QsRew>Tourism PEI<\/a>","<a href=https://youtube.com/channel/UCitRWO0qH_VJoKRxI2W01Ag>City of Lancaster<\/a>","<a href=https://youtube.com/channel/UCCieoHiQw5MQvIv-_iYqM_g>Dave & Darlene Duprey<\/a>","<a href=https://youtube.com/channel/UCxcJiQfeZDIH4dGxu4eA36g>CreekTV<\/a>","<a href=https://youtube.com/channel/UCK_nnYIsgRh2k7InesoYSxw>NPSk12va<\/a>","<a href=https://youtube.com/channel/UCcPZKeZ1IUrxzE2DSIpzXAw>LouisvilleMetroTV<\/a>","<a href=https://youtube.com/channel/UCcwgozMyf2k6KA-XebMu5rQ>districtofmission<\/a>","<a href=https://youtube.com/channel/UCeE_KkLW1qXJvzHPt6jCquQ>CityofTacoma<\/a>","<a href=https://youtube.com/channel/UCcqK16-aoVF_-9DjMdfWUlQ>EISD YouTube<\/a>","<a href=https://youtube.com/channel/UCfjP3g6KlE5urBZCGamFKQg>Shaw TV Nanaimo<\/a>","<a href=https://youtube.com/channel/UCewuCq-6vn-vUXizl88W58g>mcsStealMine<\/a>","<a href=https://youtube.com/channel/UCIVAbvMJHZa46JEjbDRRB3g>Travel Yukon<\/a>","<a href=https://youtube.com/channel/UCjwzxPE7_AVVwOb_hXodoFQ>Norman Public Schools<\/a>","<a href=https://youtube.com/channel/UC_ECy0vAz0OVYsugPVe71ug>Edina TV<\/a>","<a href=https://youtube.com/channel/UCjio854txMnSkio-MfIszVA>Austin ISD<\/a>","<a href=https://youtube.com/channel/UC1hGizu74_ZR3wpjxbzrfCA>City of Auburndale<\/a>","<a href=https://youtube.com/channel/UCl92ObB0zFur9AcB5jeMUVA>Fairfax County Public Schools<\/a>","<a href=https://youtube.com/channel/UCqd9KIbeobOoooVQKA5L0fw>Greater Dayton RTA<\/a>","<a href=https://youtube.com/channel/UCgWC6TiLUWbrV9MaDGRq8Bw>CityofHagerstown<\/a>","<a href=https://youtube.com/channel/UCc1cljyxKwZYVnDapMi26eQ>City of San Antonio<\/a>","<a href=https://youtube.com/channel/UCda7A3Vg51w0J8CxLliA27g>Waterbury Public Schools<\/a>","<a href=https://youtube.com/channel/UCCm3eAbiTKAowAPMpSdd1Xg>MTLVille<\/a>","<a href=https://youtube.com/channel/UCzK7H1hHGf5mC84mSVfMhGw>City of Edmonton<\/a>","<a href=https://youtube.com/channel/UC6LdIwnNIOsI7aCc2I3HcNQ>Port Washington Public Library<\/a>","<a href=https://youtube.com/channel/UCrfdAtinUodsj_hcjgTJBVA>Discover Atlanta<\/a>","<a href=https://youtube.com/channel/UC6seOK4_MZtAVNkhmy4X2dQ>City of Whittier<\/a>","<a href=https://youtube.com/channel/UCRZHdlP_DS1-6TV6BmzlL_A>TCTV<\/a>","<a href=https://youtube.com/channel/UCPVyGRU6B7nGVVofYC5sN1w>OHAHawaii<\/a>","<a href=https://youtube.com/channel/UCSYpgzX8urkQMFd1ZfuuNVw>Swift Current<\/a>","<a href=https://youtube.com/channel/UCeXgk7tOM8-1pfSIFvN5pXw>GCS Schools<\/a>","<a href=https://youtube.com/channel/UCYq5MwQm3CT85DvdCpi6yTQ>City of Menifee<\/a>","<a href=https://youtube.com/channel/UC2SRiCabEdusCXtPcfyI4lQ>City of Casa Grande<\/a>","<a href=https://youtube.com/channel/UCNWU_vrFgz5_-tWIYmekSaQ>Southwestern<\/a>","<a href=https://youtube.com/channel/UCNNvw-AUAw-DxCLYhyFcefw>Norwalktelevision<\/a>","<a href=https://youtube.com/channel/UCBDT-Kl4nHgEvpwYfLb315w>Uncut Angling<\/a>","<a href=https://youtube.com/channel/UCQQgpaW6Wl7mlEWvHqvXESQ>Flagstaff County<\/a>","<a href=https://youtube.com/channel/UCghbpqymwe2w0btyh_FPi2Q>KGOV2<\/a>","<a href=https://youtube.com/channel/UCHOxxAuxzAqriVDVc2M-Kyw>McKinney ISD Media<\/a>","<a href=https://youtube.com/channel/UC1Y7EABe894l1iEQo8dS0Mw>Above Tumbler Ridge<\/a>","<a href=https://youtube.com/channel/UCMqPepacIlZcFkgEvXYXVtQ>mchs odessa<\/a>","<a href=https://youtube.com/channel/UC2KQIQVwqRJ8AVdPTdtTclQ>Public Library of Cincinnati and Hamilton County<\/a>","<a href=https://youtube.com/channel/UCInG-5NdDGSfP_57r9ZSmkQ>CityThunderBay<\/a>","<a href=https://youtube.com/channel/UCyEp0hRIKxCNFtupWrZAVvQ>cadlvideos<\/a>","<a href=https://youtube.com/channel/UC7dfKEn2Q0CxxHzQr84kFWw>Fort Collins Police Services<\/a>","<a href=https://youtube.com/channel/UCTrY_4M7gC0xhkutpfA-SsQ>MethactonSchoolDistrict Eagleville,Pa<\/a>","<a href=https://youtube.com/channel/UCVAB9qgeF2by22hlHmdtEig>City of Inglewood<\/a>","<a href=https://youtube.com/channel/UCk9PswI845zMloMn8kaIt1A>HCAM<\/a>","<a href=https://youtube.com/channel/UCUl7e4BB8uwZdCLmuv_4FyQ>Henderson County Schools<\/a>","<a href=https://youtube.com/channel/UCSy825gxjTMgruVG2iGjU9w>loudouncountyva<\/a>","<a href=https://youtube.com/channel/UC7G7JHQ7LXGpfGBhGf1Bjag>Website.com<\/a>","<a href=https://youtube.com/channel/UCxjJZGVQVfAVo0q_ZfiCfZA>Emerald Coast Utilities Authority<\/a>","<a href=https://youtube.com/channel/UCggibm5YxB6_Gu4NcWTSExw>Oshkosh Media<\/a>","<a href=https://youtube.com/channel/UCnhosplweaCO5GLuGVtXpyw>Placentia Library District<\/a>","<a href=https://youtube.com/channel/UCvnGbVPjftQjT8PmI2_2jpw>Town of Innisfil<\/a>","<a href=https://youtube.com/channel/UCaPksJSAkEmNPmO4t6uKNXA>City of Barstow<\/a>","<a href=https://youtube.com/channel/UCeQ92NVNJQvuRuQ8GVAlytw>South Metro Fire Rescue PIO<\/a>","<a href=https://youtube.com/channel/UCBHDwnG_eFIRoSs1sc8jdpw>Village of Ossining<\/a>","<a href=https://youtube.com/channel/UCDDof3RwIzZ413Al-7KAdBQ>CityofLondonOntario<\/a>","<a href=https://youtube.com/channel/UCL6ePNrVXchXxNuoRAL1uMw>Richland County<\/a>","<a href=https://youtube.com/channel/UCuYPL89Peh3oH_TIGWtzbvg>City of Renton<\/a>","<a href=https://youtube.com/channel/UCQYQDb3n5HakxG-9GVN8Mgw>itsmartanews<\/a>","<a href=https://youtube.com/channel/UCY6_bZfIJNIgVixdHmp2V3w>CTAConnections<\/a>","<a href=https://youtube.com/channel/UCBUuVy-Nfy1r9NAVM_BbitQ>PPS Communications<\/a>","<a href=https://youtube.com/channel/UC0-_BC50udU1JSKZ6ZmDaqQ>North Vancouver District<\/a>","<a href=https://youtube.com/channel/UCde2sWwb6pSapK6xi371nJQ>Elyria Schools Pioneer TV<\/a>","<a href=https://youtube.com/channel/UCCs9Fy2QlMXah1JPxRs9Xdw>KCLV Channel 2<\/a>","<a href=https://youtube.com/channel/UCqUHXqdEvrOIXphRMjqr1SA>Destination Osoyoos<\/a>","<a href=https://youtube.com/channel/UCAk6QkZXpc7a_7gwD8uhkew>MunicipalityofLeamington<\/a>","<a href=https://youtube.com/channel/UCqm_bOlzVVt50RzPLBt9wFw>townofnorthhempstead<\/a>","<a href=https://youtube.com/channel/UC0O_KSThofM9OjT__eEYkeQ>City of Pensacola<\/a>","<a href=https://youtube.com/channel/UCxhnbjW5_gD9I4gk_5t6-Aw>City of Mesa<\/a>","<a href=https://youtube.com/channel/UCePk2kOJdIOzJbd2D1vTENA>Paulding County<\/a>","<a href=https://youtube.com/channel/UCIbOv9DiKt7s8KPLAjRwD8g>City of Huntington, WV<\/a>","<a href=https://youtube.com/channel/UCHh0XD1oISXtkFGqExIoEMA>CityofDunedinFL<\/a>","<a href=https://youtube.com/channel/UCwixX1_E5eQKmutfJ0Qec0g>Charlestown Connection<\/a>","<a href=https://youtube.com/channel/UC7uqdUh3ReeRG3l3qXKvzFg>cityofdeming2012<\/a>","<a href=https://youtube.com/channel/UC65GAn9FXNEl365q5mquq8Q>MetroForward<\/a>","<a href=https://youtube.com/channel/UCE1pCgmE0ftQntp1tdzwZ4g>YourTCC22<\/a>","<a href=https://youtube.com/channel/UCkVx--fAe5zHmC2uHNvHhkg>WoodstockLibrary<\/a>","<a href=https://youtube.com/channel/UCkOUwdWdgm8a2PN8y62d4dQ>Hunt Fish Manitoba<\/a>","<a href=https://youtube.com/channel/UCwCXWeCYu0M2QBGkkrF5Aaw>DanvilleVAGov<\/a>","<a href=https://youtube.com/channel/UCGE09zZ01uLuuJ3xs6VWVng>Duluth Minnesota<\/a>","<a href=https://youtube.com/channel/UCS4M_3z2O5g5pg6mxG9KDdQ>Edinburg Cable Network<\/a>","<a href=https://youtube.com/channel/UCasQyO1K4yMq3Hi_0RQ0jfA>CharmTV Citizens' Hub<\/a>","<a href=https://youtube.com/channel/UCwGZQ9Uh8K4353zNWSH-Rxg>EPISD Live<\/a>","<a href=https://youtube.com/channel/UC7KDbF1XykrYlxnfhEH5XVA>Adams County CO Government<\/a>","<a href=https://youtube.com/channel/UCedAsDawpKL5Zk1RR_XXwvQ>Cullman Regional Medical Center<\/a>","<a href=https://youtube.com/channel/UCKpWDnPeJ9IQ9Sx7AfBORhg>MetroNashville<\/a>","<a href=https://youtube.com/channel/UCVPUJStF68PtwLtDQKSCceg>NashComCollege<\/a>","<a href=https://youtube.com/channel/UC-ltrT7T2PDRdEEoU5NtOEg>Hillsborough County<\/a>","<a href=https://youtube.com/channel/UC7oq1CaE5Oy4nGEL5-lSZtw>OUSDNews<\/a>","<a href=https://youtube.com/channel/UCQIMuBjgGLURPbZS-3RGzKg>AkronCantonAirport<\/a>","<a href=https://youtube.com/channel/UCL8OSG6rGN8QrXlZx6vH-3g>CityofPG<\/a>","<a href=https://youtube.com/channel/UCAT7CtUHi7CgtfTHfDChG_g>City of Maumelle<\/a>","<a href=https://youtube.com/channel/UCjbc4Pt4VyJWg9GnT0LgSuA>RideUTA<\/a>","<a href=https://youtube.com/channel/UCPRBTIJPBe60dKAtgCHYwpQ>City of Gainesville, Florida<\/a>","<a href=https://youtube.com/channel/UCHdpcxqVcUR4rXciaInacPQ>LibraryThing<\/a>","<a href=https://youtube.com/channel/UC66Rqt7C11xa7OvwaBASxvw>VIA Metropolitan Transit<\/a>","<a href=https://youtube.com/channel/UCP8H8krGA2LR6l5T_HmFWPg>Tampa Airport<\/a>","<a href=https://youtube.com/channel/UCG-Qd1N2JB9Jh3aoHl5_q_Q>Kellogg Community College<\/a>","<a href=https://youtube.com/channel/UC6zlM5AFR4TRiTWLfPXe2PQ>TheCityofAirdrie<\/a>","<a href=https://youtube.com/channel/UCCHkAge1ezSvjcI1MsItGTA>Environment Yukon<\/a>","<a href=https://youtube.com/channel/UCONmut1XJ22SWICm0olH52g>Fairfax County Government<\/a>","<a href=https://youtube.com/channel/UCjl3zDun9SazYI0UWWu6_1A>Gold Shaw Farm<\/a>","<a href=https://youtube.com/channel/UCYhhAd-JI21TZv7TypopN6Q>Niagara Falls, Canada<\/a>","<a href=https://youtube.com/channel/UCG-JNJVBZJ5d5BOXZqOFT1g>MySMUD<\/a>","<a href=https://youtube.com/channel/UCkTCqKsgLCa9JfggF_x9Rqw>Pioneer Library System<\/a>","<a href=https://youtube.com/channel/UClUKmPJquVtDXeJmN9C17rg>Brazos River Authority<\/a>","<a href=https://youtube.com/channel/UCXS0w8SMYNK9PfaSJHykQTQ>Dearborn Public Schools<\/a>","<a href=https://youtube.com/channel/UCfvxJiEZKRFEr8Wow89robw>Allen ISD<\/a>","<a href=https://youtube.com/channel/UCvRcnbpDMAQxbv2xazcfLgA>Ramsey County<\/a>","<a href=https://youtube.com/channel/UCTezqtA8UmUTHS_s4F9-Hpw>City of Saginaw, MI<\/a>","<a href=https://youtube.com/channel/UCGuHPM6HOErp4vucFzbUkEg>CCPTV53<\/a>","<a href=https://youtube.com/channel/UCXLOLKzXqOtD9BSlPGrDBHQ>City of Centerville, Ohio<\/a>","<a href=https://youtube.com/channel/UCZLXNtcdwwjECrPg6YrTiYg>Portsmouth, Virginia<\/a>","<a href=https://youtube.com/channel/UCvVcPb65YNELRQn9HocI71A>cityofkitchener<\/a>","<a href=https://youtube.com/channel/UCQ4nAwW757eqLiwezi3CxlQ>Golden Gate Bridge District<\/a>","<a href=https://youtube.com/channel/UCMjqZYGMJujjiVBQ-901x_Q>City of Malibu<\/a>","<a href=https://youtube.com/channel/UC1WvtY-9H0K4vdV1UrTrVVg>Armstrong School District<\/a>","<a href=https://youtube.com/channel/UCYe7gRZm9zkeyh7oIFspULQ>Steinbachonline Steinbach<\/a>","<a href=https://youtube.com/channel/UCAbQeRpOa1f12BZnXcYQlqw>City of San Diego<\/a>","<a href=https://youtube.com/channel/UC3gS30KiRTi4KdqwKCglm-w>Ann Arbor District Library<\/a>","<a href=https://youtube.com/channel/UCDz1-m1rzglGoYCHG-yTLaA>City of Jacksonville, Florida<\/a>","<a href=https://youtube.com/channel/UCWelwzDLIRvdJ7NaLQfS14g>cityofchicagotv<\/a>","<a href=https://youtube.com/channel/UCtQvnN0tcVVVQvoH3zukPFA>AGTV - Albert Gallatin Television<\/a>","<a href=https://youtube.com/channel/UClDSwF6xAK1ee5ODJjZKGrg>RochesterHillsTV<\/a>","<a href=https://youtube.com/channel/UCaDGCxAaOVqUVC-WUoTnXog>Prince Edward Island Government<\/a>","<a href=https://youtube.com/channel/UCx4-4RHo_bhTMQJNh6Du0AA>CityofTampa<\/a>","<a href=https://youtube.com/channel/UCdAjGelEeYtm52gaq2OXG1g>West Metro Fire Rescue<\/a>","<a href=https://youtube.com/channel/UCplFx2Ys-EbFp2GJb_o6euQ>WSIL Carterville<\/a>","<a href=https://youtube.com/channel/UCzQFTRvOv-RbSJt6xTbz4Iw>ParkTV St. Louis Park<\/a>","<a href=https://youtube.com/channel/UCa4MVjN_2slqln_Lu3rSHBA>Arlington TV<\/a>","<a href=https://youtube.com/channel/UCyWZ5JpMaMJbbjh8ZVaBeHg>apscolorado<\/a>","<a href=https://youtube.com/channel/UC0a6MFPBDmOOSgChqnnG1cQ>Washtenaw County Sheriff's Office<\/a>","<a href=https://youtube.com/channel/UCbjmiH_7BLB88LCtKbg-cQQ>Placer County Sheriff's Office<\/a>","<a href=https://youtube.com/channel/UCrDHsvyWIAUTKUD4Kxb8rog>County of Nevada California<\/a>","<a href=https://youtube.com/channel/UCaqt7VUKAE0HZdNVYca09ag>AlbanyKALB<\/a>","<a href=https://youtube.com/channel/UCGBD7uXRwp_lb2Mpu2XupBQ>SMHCS<\/a>","<a href=https://youtube.com/channel/UCRUWrixRTArzQZ8Wy-c0-Cg>WaterlooSchools<\/a>","<a href=https://youtube.com/channel/UCCqV9b2G61qin4Wq_xfubww>Lake County Schools<\/a>","<a href=https://youtube.com/channel/UCZWA_sjSCWBDT-DhgzlphHw>NiagaraCCC<\/a>","<a href=https://youtube.com/channel/UCgWL3llD99IP_BV52n2aC5A>TheCityofRedDeer<\/a>","<a href=https://youtube.com/channel/UC0C3sD_5sAeJdV6whYyk_fw>City of Moreno Valley<\/a>","<a href=https://youtube.com/channel/UC0MKBRS7teISJ5iGufoc_Iw>CSBTV20<\/a>","<a href=https://youtube.com/channel/UC3imja0b18Gwy3zmz6MzNPA>Prince William County<\/a>","<a href=https://youtube.com/channel/UCJeyak1VUsnc1rPehZPNswg>WestminsterCO<\/a>","<a href=https://youtube.com/channel/UCNJTPwTD4LZXxCy1uVYKrUg>Moore Public Schools<\/a>","<a href=https://youtube.com/channel/UC4GayWK8PEAkBD6jFZZT3YA>Chino 3 TV<\/a>","<a href=https://youtube.com/channel/UCIeockCHC47J2xvZDfgTmew>Iowa Secretary of State Paul Pate<\/a>","<a href=https://youtube.com/channel/UCb8NW1oSc60z42mvJ09w2fA>PoudreFireAuthority<\/a>","<a href=https://youtube.com/channel/UCuObz0s_dSopQF34rdf-WOA>Tinley Park Television<\/a>","<a href=https://youtube.com/channel/UClK33Il4G13AD2jK-uTFDGw>Cape City<\/a>","<a href=https://youtube.com/channel/UCM8Iyqh9P_EjTvqeMgGKBQw>Halifax Regional Municipality<\/a>","<a href=https://youtube.com/channel/UCmNhzj_PNx2VtdyIU77KDyg>Shawnee Mission School District<\/a>","<a href=https://youtube.com/channel/UCLbomPwqCgvwr3hPm4oeyfw>Waterloo Engineering<\/a>","<a href=https://youtube.com/channel/UC0z8L3WPKQX6IcxVbGWaVRw>Texas Education Agency<\/a>","<a href=https://youtube.com/channel/UCYUnwGUWAIyeB2aoU8gqzAw>DistrictofSquamish<\/a>","<a href=https://youtube.com/channel/UCw7L0sgbpJcKmLUGdSRbXcQ>Grant PUD<\/a>","<a href=https://youtube.com/channel/UC1cEzvomoz0YDP5q3waixHg>Boulder County<\/a>","<a href=https://youtube.com/channel/UCDcbH-w5Jybe8QgLo0HTXvQ>PhilaGov<\/a>","<a href=https://youtube.com/channel/UCM_0pKe-ktTrDYz0xsjIr5g>Tulsa Public Schools<\/a>","<a href=https://youtube.com/channel/UCjXfT5bQ9ElXouOpBwqijag>City of Vancouver<\/a>","<a href=https://youtube.com/channel/UC1ovpYGxKYZQqvjDeddNIeQ>MWRD<\/a>","<a href=https://youtube.com/channel/UCX4d01tc07sPQ1KNtvjoJjA>City of Saint Paul Communications Services<\/a>","<a href=https://youtube.com/channel/UCyMnM1aOr5368kcOXvFo5pA>AlamedaCountyCA<\/a>","<a href=https://youtube.com/channel/UCHUDjJ4VIFaf3a9ap9s5qEA>CityofDurhamNC<\/a>","<a href=https://youtube.com/channel/UCtLCerq6-j_YtubKQ6ihd7g>RVHBarrieON<\/a>","<a href=https://youtube.com/channel/UC1wZ8emVESh3m9NAXbiRu3A>Western Virginia Water Authority<\/a>","<a href=https://youtube.com/channel/UCAthAfV8Iqgyl8znznZodlQ>City of Miami Beach TV<\/a>","<a href=https://youtube.com/channel/UCEr96UIvnUDDfHDqIU5PfLw>Perrysburg Schools<\/a>","<a href=https://youtube.com/channel/UCPqZc6yr1qx5_poQhq_8CPg>cityofregina<\/a>","<a href=https://youtube.com/channel/UCWFX8KFAV1uOlXdAlVvwRCg>LasVirgenesMWD<\/a>","<a href=https://youtube.com/channel/UC5T8PtpOZ94eXt-wQFobSSg>AmesChannel12<\/a>","<a href=https://youtube.com/channel/UC_EUYuKP6AENi7tEcV4HTbw>Haines High School<\/a>","<a href=https://youtube.com/channel/UCqACnJuJJOcWDNudesNS-_w>Chesapeake College<\/a>","<a href=https://youtube.com/channel/UC636eeQWEcH-wXgqukWLOHg>City of Issaquah<\/a>","<a href=https://youtube.com/channel/UCxfKeKkbCJ6RyBjH3GJx__w>UBC Okanagan<\/a>","<a href=https://youtube.com/channel/UCOW5J0lHWf-JaCZHqho3wkg>RTAChicago<\/a>","<a href=https://youtube.com/channel/UCmn4I8PUQp1gzPFdM8EVdWg>Regional Municipality of Wood Buffalo<\/a>","<a href=https://youtube.com/channel/UCqUEi_lCwIdUg-CF5S8Kmbg>CSCBroward<\/a>","<a href=https://youtube.com/channel/UCGp-YWZV_7Urqx06v38D9QA>Town of Oakville<\/a>","<a href=https://youtube.com/channel/UC2-j1u6mIH5Hi6EXXZl29ow>BJWSA<\/a>","<a href=https://youtube.com/channel/UCXtdlVUaWK_h4kj2ufJLG_w>ThunderBayMuseum<\/a>","<a href=https://youtube.com/channel/UC9OalWVQgm0YhyzTu-GbAyg>South King County Fire Training Consortium<\/a>","<a href=https://youtube.com/channel/UCx7FQNzOFCbtExt_gRub9JQ>CityofPhoenixAZ<\/a>","<a href=https://youtube.com/channel/UCb0cvwGy6A8NmJD41qGu9-Q>Victoria Texas Videos<\/a>","<a href=https://youtube.com/channel/UCihhW7xDkKQcKVFuqD6koGg>CityRichmondHill<\/a>","<a href=https://youtube.com/channel/UC-FHLAacr6WH2WIq7Yhq_pA>publiclibrary1<\/a>","<a href=https://youtube.com/channel/UCaqMtBXuGAbj-McQB5R5wXg>San Juan College<\/a>","<a href=https://youtube.com/channel/UCOsrPzT_9EIvucUQgghOKbg>Village of Mount Prospect<\/a>","<a href=https://youtube.com/channel/UC_S2bvNTfk00xtCKy1a9S3g>Sequim Government<\/a>","<a href=https://youtube.com/channel/UC3z8yObfY1WmPws_ALCw_7g>LancasterSchools<\/a>","<a href=https://youtube.com/channel/UCtKY2DeY9GkBr35opV7E2Lg>City of Bellingham, Washington<\/a>","<a href=https://youtube.com/channel/UCb78Kw3EUviBPU4apNVmjUA>Region of Waterloo<\/a>","<a href=https://youtube.com/channel/UCAmaO1INgecMgeQhm2K1CvQ>OCFA PIO<\/a>","<a href=https://youtube.com/channel/UCJtB7XWSwft6zsvVe599Igw>City of McAllen<\/a>","<a href=https://youtube.com/channel/UCMS2L0Hbvg5vcNHRRU34gwg>City of Lacombe<\/a>","<a href=https://youtube.com/channel/UCuAa7DaemcOS_q6Ql9Qo5Yg>City of Des Moines – DMTV<\/a>","<a href=https://youtube.com/channel/UC9yI7q_1fhS4KpfQp74mokQ>City of Fort Collins<\/a>","<a href=https://youtube.com/channel/UCFxY4DNdnKNRh0rUyf__gFw>YorkRegionGovt<\/a>","<a href=https://youtube.com/channel/UCYIE-MF52AumRfD4Y60TxiA>City of Worcester<\/a>","<a href=https://youtube.com/channel/UCAqmoxJO5D6ysPQAgxYyN_g>Sonoma-Marin Area Rail Transit District<\/a>","<a href=https://youtube.com/channel/UCNhzABQUWkdxMtcqSoVBnWQ>Central Point Police Department<\/a>","<a href=https://youtube.com/channel/UCIvHqlTNSQ4cFg7r0k4fyCA>MultCoPresents<\/a>","<a href=https://youtube.com/channel/UCmWwxcTi1xtq5xeic4DbZzA>City of TLH<\/a>","<a href=https://youtube.com/channel/UCjS6cDls_cVOmf4hssZ3kSA>EasternMuni<\/a>","<a href=https://youtube.com/channel/UC_rkmrgBnMU9BrAbRWrnThg>ElPasoCountyCO<\/a>","<a href=https://youtube.com/channel/UCn658oAkMYf9xy5Cqpv4B1Q>Rancho Water<\/a>","<a href=https://youtube.com/channel/UCHMz_pY_x1LOf46sL3lr2Vg>cityofwindsor<\/a>","<a href=https://youtube.com/channel/UCRBc7ZWSh46vLti209C0SSA>City of St. Albert<\/a>","<a href=https://youtube.com/channel/UCbLf80mjuSlo7UkGmsNqduQ>Buncombe County Government<\/a>","<a href=https://youtube.com/channel/UC0sEt4nGevF5sKWQtWOpn8Q>ValemountCommunityTV<\/a>","<a href=https://youtube.com/channel/UCQtHqn81peJzY5rQrOny4BQ>Valley Medical Center<\/a>","<a href=https://youtube.com/channel/UC_0IXQTBtCL05UkvDdACNGg>CityofDunwoody<\/a>","<a href=https://youtube.com/channel/UC1eXshxSoeNx4EJpy7zwwJQ>CityofDecatur<\/a>","<a href=https://youtube.com/channel/UC5m_spz27XNayIy_Tzhg6ag>City of Lawrence Indiana<\/a>","<a href=https://youtube.com/channel/UC44dsuG34JC6nSUfPjbx2FA>Town of Leesburg, Virginia<\/a>","<a href=https://youtube.com/channel/UCqVPe2edQeGrfg4vmrFPa2Q>City of Stillwater TV<\/a>","<a href=https://youtube.com/channel/UC5Y7Xna3gAXRwzyF0TWNXGQ>Steve Segura<\/a>","<a href=https://youtube.com/channel/UC1buzbkjuy215CTejeVX6oQ>Tempe11Video<\/a>","<a href=https://youtube.com/channel/UCxjDiz8ddPpt6KLEEM2UVoA>Olmsted County<\/a>","<a href=https://youtube.com/channel/UC5GoNK27LOCirP-5wdF323A>City of • Ville de Côte Saint-Luc<\/a>","<a href=https://youtube.com/channel/UC2-H8TgM1ODhDxjRto0L8cg>CityofTucson<\/a>","<a href=https://youtube.com/channel/UCdLhi9MAMnX-z_Q4SURU1pA>TheCityofSacramento<\/a>","<a href=https://youtube.com/channel/UCUyEpgptKoEw_guKCOZ0fCQ>City of Dallas<\/a>","<a href=https://youtube.com/channel/UCupNaIcZnuwK8_UUod5Sfog>SantaCruzCOE<\/a>","<a href=https://youtube.com/channel/UCOfgXq5FloMZZmzvdPgYhUw>Bridge City News<\/a>","<a href=https://youtube.com/channel/UC2LXO8Hi2KqU07I4tWjEJQA>RiverviewMich<\/a>","<a href=https://youtube.com/channel/UCq8yOQxemSs914EHtSz03pQ>Madison City Schools<\/a>","<a href=https://youtube.com/channel/UCAe3tDrPfE9qZCZNFB5p4Gw>FernieSkier<\/a>","<a href=https://youtube.com/channel/UCOwwLknKpq-PTmltuoHkI6A>City of Victoria<\/a>","<a href=https://youtube.com/channel/UC49JYhKBzUidtsDzG-SmFEA>Lee Health<\/a>","<a href=https://youtube.com/channel/UCMCSFOcx285eg1hq6fla56Q>PUSDBoardMeeting<\/a>","<a href=https://youtube.com/channel/UC13W-m9GqbT1miA6x_5oaaw>NorthAndoverCAM<\/a>","<a href=https://youtube.com/channel/UCZKASNi4Nm27rw3gfUIjEKA>Maricopa County<\/a>","<a href=https://youtube.com/channel/UC6gHzwFJ4EAv3Z-LLGbwnQg>City of Griffin<\/a>","<a href=https://youtube.com/channel/UCXhoQ6_RPQ_vfkmLafxhppg>Marshalltown Community Television<\/a>","<a href=https://youtube.com/channel/UC4G2w_dF82g9KbJNiT9TxXQ>Montgomery City Hall<\/a>","<a href=https://youtube.com/channel/UCtC20a7KwHHqKmlOxgpXrlA>Town of Bridgewater, Nova Scotia<\/a>","<a href=https://youtube.com/channel/UCrOfNJ6bcBXFC_sQUvSmPRA>dawsonpublicpower<\/a>","<a href=https://youtube.com/channel/UCiKE6mqbKmGGlNAtccn04MA>CBC Vancouver<\/a>","<a href=https://youtube.com/channel/UCDA5elUbH3fcUFA6K-Ui6jg>pembinavalleyonline<\/a>","<a href=https://youtube.com/channel/UCZPnBKBOYuMmxQwpINlEgpg>LMCCCommunityTV<\/a>","<a href=https://youtube.com/channel/UCTYwOxX-2QNuKKI4tu8TtHw>AlpharettaDPS<\/a>","<a href=https://youtube.com/channel/UCNtpcDqcSShpqR4EysLc4Pg>City of Redwood City<\/a>","<a href=https://youtube.com/channel/UC95egG4Say-HvNrlJT_yCXg>City of Independence, Missouri<\/a>","<a href=https://youtube.com/channel/UCNb-baujcEbYoPffIc0Ch3g>Manatee County Government<\/a>","<a href=https://youtube.com/channel/UCUzanlS6hJadbrMqnO_iofQ>Salt Lake City Television<\/a>","<a href=https://youtube.com/channel/UCrsJ02lpmkSlUH7lo6pGxsw>Village of Key Biscayne<\/a>","<a href=https://youtube.com/channel/UC6HrH-NpYXmh2aIS8Mw1L_Q>City of Millville<\/a>","<a href=https://youtube.com/channel/UCZrrKaG9XAPghy-HQNdRrUA>OCHDNC<\/a>","<a href=https://youtube.com/channel/UCz_B2emsUv3njgbE48qaFzA>Arizona Western<\/a>","<a href=https://youtube.com/channel/UCtVLrJQkP5uAzLhdqrfZkDQ>EPHousingAuthority<\/a>","<a href=https://youtube.com/channel/UC8wAKHQhtbfNi6_EYC9D5og>RowlettTexasVideo<\/a>","<a href=https://youtube.com/channel/UCxfrfac6XmBdMrKBTowIPzw>CityofBurnaby<\/a>","<a href=https://youtube.com/channel/UC44hASccxUpLgmFOUBnUQaw>FCATMedia<\/a>","<a href=https://youtube.com/channel/UCeg42agpsybX9TfCu8jz2tA>Cook County Government<\/a>","<a href=https://youtube.com/channel/UC2QG4OKWo8tQ-h3fAwxr4eg>City of Asheville<\/a>"],[3.56,3.29,3.22,3.16,3.13,3.01,2.98,2.94,2.9,2.77,2.73,2.66,2.65,2.57,2.56,2.56,2.53,2.5,2.49,2.49,2.48,2.48,2.48,2.46,2.45,2.42,2.41,2.41,2.41,2.41,2.4,2.4,2.34,2.34,2.33,2.33,2.32,2.31,2.31,2.3,2.3,2.26,2.26,2.26,2.26,2.25,2.24,2.22,2.2,2.19,2.18,2.18,2.18,2.18,2.17,2.15,2.14,2.13,2.12,2.12,2.12,2.11,2.09,2.08,2.08,2.08,2.07,2.07,2.06,2.05,2.04,2.04,2.04,2.04,2.03,2.03,2.03,2.03,2.02,2.02,2,2,1.97,1.95,1.95,1.94,1.93,1.92,1.92,1.91,1.91,1.91,1.91,1.9,1.89,1.89,1.89,1.88,1.88,1.88,1.87,1.86,1.86,1.86,1.86,1.85,1.85,1.84,1.84,1.83,1.83,1.83,1.83,1.82,1.82,1.82,1.81,1.8,1.8,1.79,1.78,1.77,1.77,1.77,1.76,1.76,1.75,1.75,1.75,1.75,1.75,1.74,1.73,1.73,1.72,1.72,1.72,1.71,1.71,1.71,1.71,1.69,1.68,1.67,1.66,1.64,1.64,1.64,1.64,1.64,1.63,1.63,1.62,1.61,1.59,1.58,1.58,1.57,1.56,1.56,1.55,1.55,1.54,1.53,1.53,1.53,1.52,1.52,1.52,1.52,1.51,1.51,1.51,1.5,1.5,1.5,1.49,1.46,1.46,1.45,1.45,1.43,1.43,1.43,1.42,1.41,1.41,1.41,1.39,1.39,1.37,1.37,1.37,1.37,1.36,1.35,1.35,1.35,1.35,1.33,1.32,1.31,1.31,1.31,1.31,1.31,1.3,1.3,1.3,1.3,1.29,1.29,1.28,1.28,1.27,1.27,1.27,1.27,1.27,1.26,1.26,1.26,1.26,1.25,1.24,1.24,1.23,1.23,1.23,1.22,1.19,1.19,1.18,1.18,1.17,1.17,1.17,1.17,1.16,1.15,1.14,1.14,1.12,1.12,1.12,1.12,1.11,1.11,1.11,1.1,1.1,1.1,1.09,1.09,1.08,1.08,1.08,1.07,1.07,1.05,1.04,1.04,1.03,1.02,1.02,1.02,1.01,1,1,0.99,0.99,0.98,0.98,0.98,0.97,0.97,0.96,0.95,0.94,0.94,0.94,0.93,0.92,0.92,0.91,0.91,0.91,0.91,0.9,0.89,0.89,0.88,0.87,0.86,0.85,0.85,0.85,0.84,0.84,0.83,0.82,0.8,0.79,0.79,0.75,0.74,0.73,0.73,0.73,0.72,0.72,0.71,0.71,0.69,0.68,0.66,0.66,0.66,0.65,0.65,0.65,0.65,0.63,0.6,0.6,0.59,0.58,0.57,0.56,0.53,0.5,0.49,0.48,0.48,0.41,0.35,0.34,0.34,0.33,0.32,0.3,0.29,0.26,0.23]],"container":"<table class=\"cell-border stripe\">\n <thead>\n <tr>\n <th> <\/th>\n <th>Channel<\/th>\n <th>Score<\/th>\n <\/tr>\n <\/thead>\n<\/table>","options":{"pageLength":100,"dom":"tip","scrollY":"400px","rownames":false,"columnDefs":[{"className":"dt-right","targets":2},{"orderable":false,"targets":0}],"order":[],"autoWidth":false,"orderClasses":false,"rowCallback":"function(row, data, displayNum, displayIndex, dataIndex) {\n}"}},"evals":["options.rowCallback"],"jsHooks":[]}</script> --- class: small ### Comment sentiment map <div style="text-align: center;"> <iframe style="text-align: center" src="yt_comments_testmap_CMC9a.html" width="100%" height="500" id="igraph" scrolling="no" seamless="seamless" frameBorder="0"> </iframe> </div> --- ### Implications for local government - Citizen engagement leads to better communities <span class="small">(Gaventa & Barrett 2012)</span> - More engagement in the form of art/food/outreach videos, fewer police videos? --- ### A few caveats - Videos of local government not representative of speech in general - ASR errors (mean WER after filtering ~14%), quality of transcript related to quality of audio as well as dialect features <span class="small">(Tatman 2017; Meyer et al. 2020; Markl & Lai 2021)</span> - Low-frequency phenomena: manually inspect corpus hits - High-frequency phenomena: signal of correct transcriptions will be stronger <span class="small">(Agarwal et al. 2009)</span> → classifiers - High variability in discourse in different videos, high variability in number of comments (most few contents)  --- ### A few caveats Sample is small and probably not statistically reliable. Better approach: - Get channels with many, many videos - Randomly sample large number of videos - randomly sample large number of comments - need bigger datasets (coming...) - Transformer models (like all-MiniLM-L12-v2) are trained on segmented text with clear boundaries, but transcript text mostly has no punctuation such as periods or commas. --- exclude: true ### Summary and outlook - Large corpus of ASR transcripts from YouTube channels of local governments in Australia/NZ - Possibly useful for corpus studies of spoken language, dialectology, pragmatics - Double modals are more widespread than has previously been documented --- #Thank you! --- ### References .small[ .hangingindent[ Barbieri, Francesco, Jose Camacho-Collados, Leonardo Neves & Luis Espinosa-Anke. 2020. [TweetEval: Unified benchmark and comparative evaluation for tweet classification](https://doi.org/10.48550/arXiv.2010.12421). *arxiv*:2010.12421 [cs.CL]. Bou-Franch, Patricia, Nuria Lorenzo-Dus & Pilar Garcés-Conejos Blitvich. (2012). Social interaction in YouTube text-based polylogues: A study of coherence. *Journal of Computer-mediated Communication* 17, 501–521. Coats, Steven. (Forthcoming). Dialect corpora from YouTube. In: B. Busse & N. Dumrukcic (eds.). *Proceedings of ICAME*. Dynel, Marta. (2014). [Participation framework underlying YouTube interaction](https://doi.org/10.1016/j.pragma.2014.04.001). *Journal of Pragmatics* 73, 37–52. Gaventa, John & Gregory Barrett. (2012). [Mapping the outcomes of citizen engagement](https://doi.org/10.1016/j.worlddev.2012.05.014). *World Development* 40 (12), 2399–2410. Goode, Luke, Alexis McCullough & Gelise O'Hare. (2011). Unruly publics and the fourth estate on YouTube. *Participations: Journal of Audience and Reception Studies* 8(2), 594-615. Grootendorst, Maarten. (2022). [BERTopic: Neural topic modeling with a class-based TF-IDF procedure]( https://doi.org/10.48550/arXiv.2203.05794). *arXiv*:2203.05794 [cs.CL]. Häring, Mario, Wiebke Loosen & Walid Maalej. (2018). [Who is addressed in this comment? automatically classifying meta-comments in news comments](https://doi.org/10.1145/3274336). *Proceedings of the ACM on Human-Computer Interaction*, 2(CSCW), 1-20. Herring, Susan & Seung Woo Chae. (2021). Prompt-rich CMC on YouTube: To what or to whom do comments respond?. In *Proceedings of the 54th Hawaii International Conference on System Sciences*, 2906-2915. Ksiazek, Thomas B. (2018). [Commenting on the News](https://doi.org/10.1080/1461670X.2016.1209977). *Journalism Studies* 19(5), 650-673. Liu, Yinhan, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer & Veselin Stoyanov (2019). [RoBERTa: A robustly optimized BERT pretraining approach](https://doi.org/10.48550/arXiv.1907.11692). *arXiv*:1907.11692 [cs.CL]. ]] --- ### References 2 .small[ .hangingindent[ Loureiro, Daniel, Francesco Barbieri, Leonardo Neves, Luis Espinosa Anke & Jose Camacho-Collados. (2022). [TimeLMs: Diachronic Language Models from Twitter](https://doi.org/10.48550/arXiv.2202.03829). *arXiv*:2202.03829v2 [cs.CL]. Markl, Nina & Catherine Lai. (2021). [Context-sensitive evaluation of automatic speech recognition: considering user experience & language variation](https://aclanthology.org/2021.hcinlp-1.6). In *Proceedings of the First Workshop on Bridging Human–Computer Interaction and Natural Language Processing*, 34–40. Meyer, Josh, Lindy Rauchenstein, Joshua D. Eisenberg & Nicholas Howell. (2020). [Artie bias corpus: An open dataset for detecting demographic bias in speech applications](https://aclanthology.org/2020.lrec-1.796 ). In *Proceedings of the 12th Language Resources and Evaluation Conference*, 6462–6468. Schultes, Peter, Verena Dorner & Franz Lehner. (2013). Leave a comment! An in-depth analysis of user comments on YouTube. In *11th International Conference on Wirtschaftsinformatik, 27th February – 1st March 2013, Leipzig, Germany*, 659-673. Siersdorfer, Stefan, Sergiu Chelaru, Jose San Pedro, Ismail Sengor Altingovde, & Wolfgang Nejdl. (2014). [Analyzing and mining comments and comment ratings on the social web](https://doi.org/10.1145/2628441). *ACM Transactions on the Web* 8 (3), Article 17. Tatman, Rachel. (2017). Gender and dialect bias in YouTube’s automatic captions. In *Proceedings of the First Workshop on Ethics in Natural Language Processing, April 4th, 2017, Valencia, Spain*, 53–59. ]]