Slides from recent talks by Mark Johnson

(Last updated 19th September, 2020)

• Computational Models of Hidden Structure Learning and Language Acquisition SCiL Workshop, January 16, 2019

• How much data is enough? Predicting accuracy on large datasets from smaller pilot data, talk presented at the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia.

• What does Deep Learning tell us about Language?, talk presented at the Workshop on Relevance of Linguistic Structure in Neural NLP, at the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia.

• How much is enough? Predicting how system accuracy varies with training data size, talk presented at the Macquarie University Centre for Language Technology meeting, 4th September 2017. Joint work with Dat Quoc Nguyen.

• Generation in Image Captioning, talk presented at the 1st Workshop on Learning to Generate Natural Language, at ICML 2017 Workshops, August 2017. Joint work with Peter Anderson, Basura Fernando and Stephen Gould.

• Natural Language Processing from a Machine Learning Perspective, talk presented at the The 8th International Workshop on Health Text Mining and Information Analysis (Louhi’17), August 2017.

• Introduction to Hypothesis Tests, talk presented at the CLT meeting, February 2017.

• Computational Linguistics: Past, Present and Future, talk to be presented at ALTA, December 2015.

• Natural Language Processing from a Machine Learning Perspective, talk given in November, 2015.

• Rational Inferences and Bayesian Inferences, talk presented at the Rational Inferences Workshop, Macquarie University, 30th October 2015

• Introduction to Boltzmann Machines, talk presented to the MQ Machine Learning Reading Group, September 2015

• Introduction to Neural Networks, talk presented to the MQ Machine Learning Reading Group, September 2015

• Synergies in Language Acquisition, talk presented at The Workshop on Infant Speech Perception, Macquarie University, September 2015.

• A survey of parsing and its applications, presented at Macquarie University, August 2015.

• Nonparametric Bayesian Inference for Topical Collocation Models, presented at Huwaei's Noah's Ark lab and the Chinese Academy of Sciences, July 2015.

• Improving Topic Models with Latent Feature Word Representations, presented at Huwaei's Noah's Ark lab and Tsinghua University, July 2015.

• Sign constraints on feature weights improve a joint model of word segmentation and phonology, presented in the Macquarie University Centre for Language Technology talk series, March 2015.

• Introduction to Computational Linguistics and Natural Language Processing, presented at the 2015 Machine Learning Summer School, in Sydney, Australia, 2015.

• My talk Exploring the role of stress in word segmentation using Adaptor Grammars for the Workshop on the role of prosody at Macquarie University, December 2014.

• A series of talks on using R for data analysis:
  • Introduction to data analysis in R, including reading CSV files into R, producing summary tables of various kinds, and using ggplot2 to plot the data.
  • Introduction to hypothesis testing and regression in R, including hypothesis tests such as t-tests and their non-parametric variants (Wilcoxon tests), and regression (including logistic regression).

• A talk on Crafting Scientific Papers, presented at Macquarie University, October 2014.

• A series of introductory talks on machine learning and natural language processing for the Macquarie University Machine Learning Reading Group:
  • An introduction to information retrieval
  • An introduction to classification
  • An introduction to clustering
  • An introduction to Naive Bayes classifiers
  • An introduction to n-gram language models
  • An introduction to Hidden Markov Models (HMMs)

• Introduction to NLP, presented at the CMCRC, August 2014.

• Life Stories models for improving (text) data mining and Beyond the 1-best pipeline, presented at the NICTA NLP workshop, Canberra, July 2014.

• Modelling function words improves unsupervised word segmentation, presented at the ACL 2014 conference.

• Exploring the role of stress in Bayesian word segmentation using Adaptor Grammars, presented at the ACL 2014 conference.

• Introduction to Computational Linguistics and Natural Language Processing, presented at the 2014 Machine Learning Summer School, in Beijing, June 2014.

• Life Stories and Text Data Mining, presented at the MQ Department of Computing seminar, May 2014.

• Life Stories, a talk about our new Google-sponsored research project with Wray Buntine, Lan Du and Anish Kumar, presented at the 2014 Google NLU meeting, Zurich, March 2014.

• Synergies in Language Acquisition (presented in Aix-en-Provence and Paris)

• Language acquisition as statistical inference, talk presented at the 19th International Congress of Linguists, Geneva, July 2013. The version of the talk presented at the University of Edinburgh has more technical details.

• From PCFGs to Adaptor Grammars and Beyond, slides from a talk given at NICTA, Canberra, May 2013.

• A gentle introduction to Maximum Entropy, log-linear, exponential, logistic, Harmonic, Boltzmann, Markov Random Field, etc., models, slides from a talk to the MQ Machine Learning Reading Group, March 2013.

• Natural language processing and computational linguistics: from theory to applications, talk presented at an MQ Informatics workshop, Macquarie University, October 2012.

• Learning words, talk presented at the CLSP Zero Resource Speech Technologies and Models of Early Language Acquisition mini-workshop at Johns Hopkins University, July 2012.

• Computational linguistics: where do we go from here? invited plenary talk, presented at ACL 2012, July 2012, Jeju Island, Korea.

• Grammars and Topic Models, presented at the NICTA seminar, April, 2012.

• Parsing in parallel on multiple cores and GPUs, presented at the 2011 ALTA workshop.

• My talk on grammars and the extra slides on Dirichlet-Multinomials and Gibbs sampling presented at the 2011 IPAM graduate summer school on probabilistic models of cognition.

• Bayesian models of Language Acquisition, or Where do the rules come from?, presented in the Stanford Linguistics department, June 2011.

• The impact of language models and loss functions on repair disfluency detection, to be presented at ACL HLT 2011.

• Bayesian Inference for Dirichlet-Multinomials and Adaptor Grammars. The following slides introduce probabilistic models for computational linguistics. These slides will be presented at the MLSS 2011 (the 2011 Machine Learning Summer School), Singapore.

• Adaptor Grammars: A framework for Bayesian non-parametric grammatical inference, November 2010.

• Slides, spotlight and poster for presentation at NIPS 2010 conference.

• Detecting Speech Repairs Incrementally Using a Noisy Channel Approach Simon Zwarts, Mark Johnson, Robert Dale, presented at COLING 2010.

• Unsupervised phonemic Chinese word segmentation using Adaptor Grammars, Mark Johnson and Katherine Demuth, presented at COLING 2010.

• Beyond rationalism versus empiricism. Mark Johnson, presented at the RING talk series, COLING 2010.

• PCFGs, Topic Models, Adaptor Grammars and Learning Topical Collocations and the Structure of Proper Names, presented by Mark Johnson at ACL 2010, July 2010.

• Reranking the Berkeley and Brown Parsers, presented by Mark Johnson and Engin Ural at NAACL-HLT 2010, June 2010.

• My talk at the Macquarie University CLT group on learning rules using Bayesian inference, February 2010.

• My talks at two NIPS 2009 workshops: on grammar induction and on adaptor grammars and topic models.

• Learning rules with Adaptor Grammars, talk presented at U. Penn., November 2009.

• Parsing and Learning, talk presented at the International Workshop for Parsing Technology (IWPT), October 2009, Paris, France.

• Adaptor grammars: applications and inference, July 2009.

• Improving nonparameteric Bayesian inference: experiments on unsupervised word segmentation with adaptor grammars, presented at the NAACL-HLT 2009 conference.

• How the statistical revolution changes (computational) linguistics, presented at The Interaction between Linguistics and Computational Linguistics: Virtuous, Vicious or Vacuous? workshop, 2009 EACL conference.

• Synergies in learning syllables and words, or Adaptor grammars: a class of nonparametric Bayesian models presented at the 2008 Northeast Computational Phonology workshop, Yale, November 2008.

• Using Adaptor Grammars to identify synergies in the unsupervised acquisition of linguistic structure, given at the ACL 2008, Columbus, Ohio.

• Unsupervised word segmentation for Sesotho using Adaptor Grammars given at the ACL 2008 SIGMORPPHON workshop, Columbus, Ohio

• Learning Rules with Adaptor Grammars given at the University of Chicago and the special edition given at University of California, Berkeley.

• Where do the rules come from?, given at the University of Geneva, UIUC and the University of Massachusetts.

• Bayesian models of language acquisition, or Where do the rules come from? at the November 2007 conference at U Penn Bridging the Developmental Divide: Sentence Processing Meets Word and Grammar Learning

• A talk for Eugene Charniak's computational linguistics class on how to write new feature classes for the reranking parser.

• A talk for the Northeastern Computational Phonology meeting at U Mass in November, 2007 entitled A gentle introduction to Maximum Entropy models and their friends.

• Three talks given at the 2007 IPAM summer school on Probabilistic Models of Cognition: The Mathematics of Mind, including Context-Free Grammars and beyond, Statistical models for natural language parsing, and Bayesian inference of grammars.

• Transforming Projective Bilexical Dependency Grammars into efficiently parsable CFGs with Unfold-Fold by Mark Johnson at the ACL 2007 conference.

• Why doesn't EM find good HMM POS taggers? presented by Mark Johnson at the EMNLP 2007 conference.

• Bayesian Inference for PCFGs via Markov chain Monte Carlo by Johnson, Griffiths and Goldwater at the 2007 HLT/NAACL 2007 conference

• Learning Grammar(s) Statistically, invited talk at the University of Stuttgart, Opening colloquium for SFB 732, November 2006.

• Powerpoint slides from the talk "Distributional Cues to Word Boundaries: Context Is Important" by Goldwater, Griffiths and Johnson at the 2006 Boston University Conference on Language Development, November 2006.

• Slides from my talk "Features of Statistical Parsers" given at the joint Microsoft Research/University of Washington computational linguistics colloquium, October 2006.

• Slides from my talk ``Collecting, err, correcting speech errors'' at the the Eighth International Workshop on Tree Adjoining Grammar and Related Formalisms (TAG+8) at ACL/COLING 2006.

• Slides from my talk at the 2006 Mayfest meeting at the University of Maryland. The material in this talk is described in greater length in our NIPS 2005 paper Interpolating between types and tokens by estimating power-law generators and our ACL 2006 paper Contextual Dependencies in Unsupervised Word Segmentation.

• Slides from my invited talk at the 2006 CUNY conference on parsing speech corpora (discusses statistical parsing of speech, disfluency detection, etc.).

• Slides from my talk on Non-Parametric Bayesian Models for Natural Language at the Pattern Theory group in Brown's Applied Maths department in March 2006. This talk discusses how to define recursive Dirichlet processes based on Context-Free Grammars.

• Slides from my talk at the Linguistics department at the University of Hawaii.

• Slides from my invited talk at the lunch-time symposium at the 2005 Boston University Conference on Child Language Development (a DVD of the symposium will also be available from the BUCLD web site).

• Slides from my invited after-dinner talk at the ESSLLI 2005 summer school

• Slides in 1-up format and the same slides in 4-up format for my class at the ESSLLI 2005 summer school.

• I prepared some extra material on Weighted CFGs for the ESSLLI'05 course, which I didn't get around to presenting, but you might find them surprising. I also presented my CoNLL'05 talk (the next item below) during the ESSLII'05 summer school course, so you might want to look at those slides as well.

• Slides for my invited talk at the CoNLL-2005 conference

• Slides from my lecture in the TTI summer school, May 2005. "An introduction to grammars and parsing". You may also find the paper I co-authored with Stu Geman useful: "Probability and statistics in Computational Linguistics: A brief review"

• Slides from my talk at Cornell University, April 2005 "Features of Statistical Parsers"

• Slides from Johnson and Charniak's 2004 ACL talk ``A TAG-based noisy channel model of speech repairs''

• Slides from Ciaramita and Johnson's 2004 talk at the Senseval workshop at ACL'04

• Slides from Hall and Johnson's 2004 ACL talk ``Attention shifting for parsing speech''

• Slides from Goldwater and Johnson's 2004 talk at the SIGPHON workshop at ACL'04 ``Priors in Bayesian Learning of Phonological Rules

• Slides from a talk at Stanford Collecting, err, correcting speech errors given in November, 2004.

• Slides from my invited talk at the IJCNLP on Hainan Island, China, in March 2004 on Probabilistic Parsing with a Wide Variety of Features

• Slides from my talks at the University of Tokyo in March 2004 on discriminative approaches to statistical parsing and on statistical parsing and speech recognition.

• Slides from my 2003 ALTA summer school course on Grammars, graphs and automata

• Slides from my 2003 talk at the Toyota Technical Institute at the University of Chicago on Features of Statistical Parsers

• Slides from my 2003 invited talk at COLT 2003 on Learning and parsing stochastic unification-based grammars

• Stochastic ``unification-based'' grammars, a talk given at Edinburgh in January 2002, which describes how to calculate the statistics needed for parsing and estimation from packed feature structures. Available in Adobe Acrobat format.

• Slides from my talks at LFG 2002 and ACL'02 (on dynamic programming in SUBGs) and ACL'02 (on empty nodes) (all in PDF).

• Slides on Compact non-left-recursive grammars using the selective left-corner transform, based on Johnson and Roark (2000), presented at COLING 2000.

• My ACL 1998 talk on left-corner grammar approximations (the paper from the ACL 1998 proceedings is here and the bib file is here). If you like this kind of stuff, you might like my notes on additional left-corner related transforms.

• A set of slides describing Neal and Hinton's (1998) paper "A view of the EM algorithm that justifies incremental, sparse, and other variants"

• Slides on CRFs and their relationship to HMMs

• Slides on estimating MaxEnt models and the relationship between Stochastic Gradient Descent and the Percepton