Research
I am a regular speaker on artificial intelligence and crowdsourcing for global impact. I've spoken at venues including Stanford, MIT, Oxford, Berkeley and Harvard Unversities, the United Nations, the World Bank, the White House, KDD, Strata, and many technology companies.
My PhD at Stanford University was as a Graduate Fellow: top 5% of engineering/science PhD candidates. My PhD was under Christopher Manning and studied how AI can adapt to low resource languages in health and disaster response situations.
I was awarded the University Medal (valedictorian) for my undergraduate degrees at the University of Sydney, where I decided to major in everything that interested me, which ended up being: Computer Science, English, Film, Information Systems and Linguistics.
Selected publications, presentations, panels, etc.
Monarch, Robert (Munro). 2020. Human-in-the-Loop Machine Learning. Manning Publications.
Monarch, Robert (Munro) and Alex (Carmen) Morrison. 2020. Detecting Independent Pronoun Bias with Partially-Synthetic Data Generation. The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).
Monarch, Robert (Munro). 2020. Uncertainty Sampling and Diversity Sampling. Open Data Science Conference.
Monarch, Robert (Munro). 2020. Panel: Tips for Writing an Effective AI and Deep Learning Book. DataCon LA. Panelist with Chris Fregly, Francesco Mosconi, Jessica Haberman, Rustem Feyzkhanov, and Dean Wampler.
Munro, Robert, 2020. The rapid growth of Human-in-the-Loop Machine Learning (Invited Talk). 4th Workshop on Interactive Adaptive Learning At ECML PKDD 2020
Munro, Robert, 2020. 5 Ways Data Scientists Can Help Respond to COVID-19 and 5 Actions to Avoid. KDnuggets.
Munro, Robert, 2020. Panel on AI Products. ScaledML.
Munro, Robert, 2020. Human-Centered Machine Learning (Human-in-the-Loop Keynote). Data Day Texas 2020
Munro, Robert, 2019. Advanced Active Learning Cheatsheet. Towards Data Science.
Advanced Active Learning Cheatsheet (PDF cheatsheet only)
Munro, Robert, 2019. Diversity in AI is not your problem, it’s hers. Personal blog post (trigger for New York Times article on bias in AI).
Munro, Robert, 2019. Discovering Your Model's Known Unknowns and Unknown Unknowns. Scale By The Bay.
Munro, Robert, 2019. AI Products. Scale By The Bay. Panel with Anima Anandkumar, Peter Bailis, Thanh Nguyen, Pete Skomoroch, and Manasi Vartak.
Munro, Robert and Radhu Basu, 2019. Ethical Human-in-the-Loop Pipelines. TWiMLCon.
Munro, Robert, 2019. Active Learning with PyTorch. PyTorch Blog.
Munro, Robert, 2019. PyTorch Active Learning Library (Poster). PyTorch Developer Conference.
Munro, Robert, 2019. Diversity Sampling Cheatsheet Towards Data Science.
Diversity Sampling Cheatsheet (PDF cheatsheet only)
Munro, Robert. 2019. Should You Open-Source Your Model? Ethical questions for open-sourcing Machine Learning models. 25th ACM Conference on Knowledge Discovery and Data Mining (KDD 2019).
Munro, Robert, 2019. Uncertainty Sampling Cheatsheet. Towards Data Science.
Uncertainty Sampling Cheatsheet (PDF cheatsheet only)
Munro, Robert, 2019. Knowledge Quadrant for Machine Learning. Towards Data Science.
Munro, Robert. 2019. Driving Diverse Product Feedback at Your Company. Product Management Insider.
Munro, Robert. 2019. Tactics for Leading and Optimizing PR/FAQ Reviews. Product Management Insider.
Munro, Robert. 2019. PR/FAQs for Product Documents — Everything Product Managers Need to Know. Product Management Insider.
Munro, Robert, 2019. Creating New Scripts with StyleGAN. Towards Data Science.
Munro, Robert. 2019. Dissecting the Controversy around OpenAI’s New Language Model. This Week in Machine Learning & AI (TWiML). Panel hosted by Sam Charrington, with Amanda Askell, Miles Brundage, Anima Anandkumar and Stephen Merity
Munro, Robert. 2019. Socially Impactful Natural Language Processing. AI for Social Good. (Invited Lecture) Stanford
Munro, Robert. 2019. Transfer Learning Today: the Good, the Bad, and the Ugly of NLP in 2019. Data Day Texas (Keynote). Austin
Munro, Robert. 2018. Vision: API and Cloud AutoML. Google Cloud NEXT. San Francisco. Intro to Figure Eight, and Q&A with Zain Shah, Eli Thomas and Francisco Uribe
Munro, Robert. 2018. Human-in-the-Lopp AI for Emergency Response and More. This Week in Machine Learning & AI (TWiML). (Podcast Interview with Sam Charrington)
Audio: https://www.youtube.com/watch?v=ZgKMts8MC5M
Munro, Robert. 2018. Real World Human-in-the-Loop Machine Learning. Train AI. (Keynote). San Francisco.
Video: https://vimeo.com/271417508
Munro, Robert. 2018. From PhD to Product: Panel Discussion. Train AI. Panelist with Anima Anandkumar, Kapil Gupta, and Olya Gurevich
Video https://vimeo.com/272703924
Munro, Robert. 2018. Combining Human Intelligence with Machine Learning for NLP and Speech. Data Science Central. Webinar.
Video: https://vimeo.com/270523394
Munro, Robert. 2018. Practical Human-in-the-Loop Machine Learning. Amazon Web Services (AWS). Webinar.
Video: https://www.youtube.com/watch?v=rJw7u8qyDf4
Munro, Robert. 2018. AI for Disaster Response. CogX. London.
Video: https://www.youtube.com/watch?v=bozLu7_8GEo
Munro, Robert. 2018. Introducing Figure Eight. CogX. London.
McClean, Matt and Robert Munro. 2017. Building an Artificial Intelligence Practice for Consulting. AWS re:Invent 2017
Video: https://www.youtube.com/watch?v=tSIG8ns7t_o
Munro, Robert. 2017. Human-in-the-Loop Deep Learning. Data Science Central. Webinar.
Video: https://vimeo.com/236176768
Munro, Robert. 2017. The Future of Language Work Given Advances in Language Technology. LiltCon. Panelist with Spence Green, Franz Och, Melvin Johnson, Katie Botkin, and Arle Lommel.
Munro, Robert. 2017. Strategies for Practical Active Learning. Open Data Science Conference (ODSC West). Silicon Valley.
Munro, Robert and Lukas Biewald. 2017. Introduction to Deep Active Learning and Data Annotation. AI Education Series. San Francisco.
Biewald, Lukas and Robert Munro. 2017. Intro to Deep Learning for Text and Speech. AI Education Series. San Francisco.
Biewald, Lukas and Robert Munro. 2017. Intro to AI, Machine Learning & Deep Learning. AI Education Series. San Francisco.
Biewald, Lukas and Robert Munro. 2017. Intro to Deep Learning for Computer Vision. AI Education Series. San Francisco.
Munro, Robert. 2017. Human-in-the-Loop AI. QCon. San Francisco.
Munro, Robert. 2017. Analysts Overview. Telecom Council Meeting on Deep Learning. Silicon Valley Bank.
Munro, Robert. 2017. Artificial Intelligence and Machine Learning in Financial Services. Financial Stability Board. Basel.
Munro, Robert. 2017. Get NLP Started with AWS's Deep Learning AMI. NLP Day Texas. Austin.
Munro, Robert. 2017. Revisiting Talking Robots and AI's Progress. Train Conference. San Francisco.
Munro, Robert. 2017. Climbing the Tower of Babel: Eliminating the Language Barrier in Crisis Response. Humanitarian ICT Forum. Mountain View.
Munro, Robert. 2016. Sentiment Analysis is a Market for Lemons: here's how to fix it. Data Day Texas. (Keynote). Austin.
Munro, Robert. 2016. Human Intent and Text Understanding. Data by the Bay. (Panelist, with Ricardo Baeza-Yates, Tiho Bajić, Jenny Finkel, Surabhi Gupta and Alexy Khrabrov). San Francisco.
Munro, Robert. 2016. Understanding Speakers of 7,000 Languages. AAAS 2016. (Panel leader, with Emily Bender, Steven Bird and Ellie Pavlick). Washington DC.
Long, Jessica, Robert Munro, Nicholas Gaylord, Kidus Asfaw and Evan Wheeler. 2015. Classification models for new language communities: building domain-specific message categorization. ACM DEV. London.
Munro, Robert. 2015. Adaptive Learning: The Next Generation of Machine Intelligence. Rich Data Summit. San Francisco.
Munro, Robert. 2015. Panel Debate: How Best to Promote a Language? The Oxford Global Languages Symposium. Oxford University.
Akred, John and Robert Munro. 2015. Building the World's Largest Database of Car Features from PDFs. Text by the Bay. San Francisco.
Goentzel, Jarrod, Lauren Seelbach and Robert Munro/ 2015. Analyzing Twitter Data for Situational Awareness During a Disaster. Annual Meeting of the Production and Operations Management Society. Washington DC.
Munro, Robert. 2015. Building Better Experts: co-optimization of human and machine intelligence Data Day Texas. (Invited Talk). Austin.
Munro, Robert. 2015. Artificial Intelligence for Social Good. From Languages to Information. (Invited Talk). Stanford, CA.
Munro, Robert. 2014. Bringing Data Science to the Speakers of Every Language. ACM Conference on Knowledge Discovery and Data Mining (KDD). (Invited Talk). New York.
Munro, Robert. 2014. Distributed Computing for Disaster Response. Workshop on FutureICT - Systems Resilience in the Digital Society (Invited Talk). NetSci'14, UC Berkeley.
Munro, Robert. 2014. Crowdsourcing and Natural Language Processing for the Social Good. 4th Stanford Conference on Computational Social Science (Invited Talk). Stanford, CA.
Munro, Robert. 2014. Natural Language Processing for Disaster Response. From Languages to Information. (Invited Talk). Stanford, CA.
Munro, Robert. 2013. Social Impact Through Big Data & Crowdsourcing. Workshop on Social Impact Through Big Data. Wharton School, San Francisco, CA.
Munro, Robert, Schuyler Erle and Tyler Schnoebelen. 2013. Analysis After Action Report for the Crowdsourced Aerial Imagery Assessment Following Hurricane Sandy. 10th International Conference on Information Systems for Crisis Response and Management. Baden Baden, Germany.
Chan, Jennifer, John Crowley, Shoreh Elhami, Schuyler Erle, Robert Munro and Tyler Schnoebelen. 2013. Aerial Damage Assessment Following Hurricane Sandy. Idibon, GIS Corps, Harvard Humanitarian Initiative and Humanitarian OpenStreetMap.
Munro, Robert. 2013. Language Technologies for Connected World. Strata, O'Reilly
Munro, Robert. 2013. Crowdsourcing and Natural Language Processing for Humanitarian Response. Crisis Informatics and Analytics. Disaster Resilience Leadership Academy, Tulane.
Munro, Robert. 2013. Crowdsourcing and the Crisis-Affected Community: lessons learned and looking forward from Mission 4636. Journal of Information Retrieval 16(2).
Munro, Robert. 2012. Processing short message communications in low-resource languages. PhD Thesis, Stanford University. Stanford, CA
Advisors: Christopher Manning, Dan Jurafsky and Tapan Parikh
Munro, Robert. 2012. Where's my Talking Robot?. Blog Post.
Munro, Robert. 2012. Where's my Talking Robot?. Ignite San Francisco. (Invited Lightening Talk)
Munro, Robert. 2012. The world inside words: information extraction and labeling in low resource languages through subword models. Invited Talk. Microsoft Research, Mountain View.
slides
Munro, Robert and Christopher Manning. 2012. Accurate Unsupervised Joint Named-Entity Extraction From Unaligned Parallel Text. The 4th Named Entities Workshop (NEWS), Jeju, Korea.
Munro, Robert. 2012. Talking to the crowd in 7,000 languages, Understanding Risk. World Bank. Cape Town. Presentation (.pptx,7MB)
Munro, Robert and Christopher Manning. 2012. Short message communications: users, topics, and in-language processing. Second Annual Symposium on Computing for Development (ACM DEV 2012), Atlanta.
Munro, Robert, Lucky Gunasekara, Stephanie Nevins, Lalith Polepeddi and Evan Rosen. 2012. Tracking Epidemics with Natural Language Processing and Crowdsourcing. Spring Symposium for Association for the Advancement of Artificial Intelligence (AAAI), Stanford.
Munro, Robert, Rainer Ludwig, Uli Sauerland and David Fleck. 2012. Reported Speech in Matses: perspective persistence and evidential narratives. International Journal of American Linguistics. 78:1, 41-75.
Munro, Robert. 2012. From Big Data to Small Languages: Communication and Information Processing in a Newly Connected World. Global Health & Innovation Conference. Unite For Sight, Yale.
Munro, Robert and Christopher Manning. 2012. State-of-the-Art Multilingual NER Using Loosely Aligned Text. Whitepaper, Stanford University
Munro, Robert. 2011. Tracking Epidemics Through Crowdsourcing and Natural Language Processing. International Conference of Crisis Mappers (ICCM). Geneva.
Munro, Robert, Eethar Al-Hassan, Autumn Albers, George Chamales, Lucky Gunasekara, Pooja Desai, Dan Lipsitt, Stephanie Nevins, Brady Page, Lalith Polepeddi, Evan Rosen, Victoria Sahakian, Lu Zhang, Nathan Wolfe. 2011. EpidemicIQ: Building the world's immune system. CrowdConf, San Francisco.
Munro, Robert. 2011. Big Impacts from Small Messages: processing communications in less resourced languages. Invited Talk. Microsoft Research, Redmond.
Munro, Robert. 2011. Linguistics in a Connected World. Invited Talk, Fieldwork Forum. University Of California, Berkeley.
Munro, Robert. 2011. Extracting Microbial Threats From Big Data. Strata. O'Reilly, NY.
Munro, Robert and Hal Tily. 2011. The Start of the Art: Introduction to the Workshop on Crowdsourcing Technologies for Language and Cognition Studies. Crowdsourcing Technologies for Language and Cognition Studies, Boulder.
Lewis, Will, Robert Munro and Stephan Vogel. 2011. Crisis MT: Developing A Cookbook for MT in Crisis Situations Annual Workshop on Machine Translation, EMNLP, Edinburgh.
Munro, Robert. 2011 Subword and spatiotemporal models for identifying actionable information in Haitian Kreyol. Fifteenth Conference on Computational Natural Language Learning (CoNLL 2011), Portland.
Munro, Robert. 2011. How we talk to the world and how sometimes it listens. The Human Computer Interaction Group, Seminar Series, Stanford.
Munro, Robert. 2011. Changing the way we communicate. Crowdsourcing Work Meetup, San Francisco.
Munro, Robert. 2010. Crowdsourced translation for emergency response in Haiti: the global collaboration of local knowledge. (Keynote) AMTA Workshop on Collaborative Crowdsourcing for Translation. Denver, Colorado.
Munro, Robert. 2010. Processing digital communications in less-resourced African languages. 33rd AFSAAP conference. Melbourne, Australia.
Munro, Robert. 2010. Addressing the crisis information bottleneck through crowdsourcing & machine-learning. (Ignite talk) International Conference on Crisis Mapping (ICCM). Boston.
Presentation (video)
Munro, Robert. 2010. Leveraging machine-learning and crowdsourcing to process text messages in the world's less-resourced languages. (Invited talks).
Center for Information Technology Research in the Interest of Society Seminar Series. University of California, Berkeley. Presentation (video)
and
ARC CS PIC Invited Speaker Series. IBM Almaden Research Center, San Jose .
Munro, Robert. 2010. Crowdsourced translation for emergency response and beyond. NSF Workshop on crowdsourcing and translation, University of Maryland.
Slides
Munro, Robert and Christopher Manning. 2010. Subword Variation in Text Message Classification. Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2010), Los Angeles, CA.
Munro, Robert, Steven Bethard, Victor Kuperman, Vicky Tzuyin Lai, Robin Melnick, Christopher Potts, Tyler Schnoebelen and Harry Tily. 2010. Crowdsourcing and language studies: the new generation of linguistic data. Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, Los Angeles, CA.
Munro, Robert. 2010. Haiti emergency response: the power of crowdsourcing and SMS. Relief 2.0 in Haiti, Stanford, CA.
Munro, Robert. 2010. Natural Language Processing for Social Development. The Research Centre for Linguistic Typology (RCLT) Seminar Series, Latrobe University, Melbourne.
Munro, Robert. 2010. SMS Emergency Response in Haiti. Aid and Relief for Global Disasters, Meeting of the Association of Clinical Faculty (ACF), University of California, San Francisco.
Munro, Robert. 2009. Amazon Mechanical Turk and the generation of new linguistic data. Stanford Seminar on Psycholinguistics, Stanford, CA.
Slides
Munro, Robert. 2009. Reported Speech and Perspective Rigidity in Matses. The 10th Stanford Semantics Fest, Stanford, CA.
[abstract]
Munro, Robert. 2006. Current design issues for digital archives: architectures supporting value-adding access via a user's preferred language(s) and granularity of materials. The Georgetown University Round Table on Languages and Linguistics (GURT 2006), Washington DC
Munro, Robert. 2006. Multilingual metadata: between translation and coexistent annotations. Proceedings of the 2006 ELAP Workshop on Meaning and Translation in Language Documentation documentation. SOAS, London. (abstract only)
Munro, Robert. 2005. The digital skills of language documentation. In Peter K. Austin (ed) Language Documentation and Description Volume 3. London: SOAS
Journal: http://www.hrelp.org/publications/papers/volume3/
Munro, Robert and David Nathan. 2005. Introducing the ELAR information system architecture. The Third meeting of the Digital Endangered Languages and Music Archive Network (DELAMAN III), Austin.
Slides
Munro, Robert and David Nathan. 2005. Towards portability and interoperability for linguistic annotation and language-specific ontologies, Proceedings of the E-MELD Workshop on Linguistic Ontologies and Data Categories for Language Resources (E-MELD 2005) Boston.
Slides
Munro, Robert. 2005. The certainty of chance: Review of Probabilistic Linguistics, Bod, R., Hay, J. and Jannedy, S. (eds). 2003. Linguistics and the Human Sciences Vol 1, 2.
Austin, Peter K., Robert Munro and David Nathan. 2005. Archives, linguists, and language speakers. The annual meeting of the Linguistic Society of America (LSA), Oakland, California.
Wynne, John and Robert Munro. 2005. Hearing Voices: speakers / languages. The Hans Rausing Endangered Languages Project, SOAS, London. [Interactive multimedia production]
To purchase see: http://www.hrelp.org/publications/multimedia/
Munro, Robert. 2004. Digital skills and obligations: is language documentation a new ICT discipline? Proceedings of the 2004 ELAP Workshop on Multidisciplinary approaches to language documentation. SOAS, London. (abstract only)
Munro, Robert. 2004. Gradational modelling of functional categories. Presentation given as part of the ELAP-ELAR research seminar series. SOAS, London.
Munro, Robert. 2004. A Probabilistic Representation of Systemic Functional Grammar. The 31st International Systemic Functional Congress (ISFC31). Kyoto.
Slides
Munro, Robert and Sanjay Chawla. 2004. An Integrated Approach to Mining Data Streams, TR-548, School of Information Technologies, University of Sydney.
Munro, Robert and Sanjay Chawla. 2004. Complex Spatial Relationships, TR-539, School of Information Technologies, University of Sydney.
Arunasalam, Bavani, Sanjay Chawla, Pei Sun, Robert Munro. 2004. Mining Complex Relationships in the SDSS SkyServer Spatial Database. Proceedings of the Twenty-Eighth Annual International Computer Software and Applications Conference (COMPSAC04), IEEE Computer Society, Hong Kong
Munro, Robert, Sanjay Chawla and Pei Sun. 2003. Complex Spatial Relationships, Proceedings of the Third IEEE International Conference on Data Mining (ICDM2003) Florida.
Munro, Robert. 2003. Seneschal: classification and analysis in supervised mixture-modelling, Proceedings of the Third International Conference on Hybrid Intelligent Systems (HIS'03) IOS Press, Melbourne
Munro, Robert and Daren Ler. 2003. Meta-Learning Orthographic and Contextual Models for Language Independent Named Entity Recognition. Proceedings of CoNLL-2003, Canada,
Munro, Robert. 2003. A queuing-theory model of word frequency distributions, Proceedings of the First Australasian Language Technology Workshop (ALTW'03), Melbourne
Munro, Robert. 2003. Towards the Computational Inference and Application of a Functional Grammar, (honours thesis). Department of English and School of Information Technologies, University of Sydney
Munro, Robert. 2003. Classification and analysis in supervised mixture-modelling, TR-536, School of Information Technologies, University of Sydney