To remove any frames surrounding this page,
click here
Slides from recent talks by
Mark Johnson
(Last updated 19th September, 2020)

Computational Models of Hidden Structure Learning and Language Acquisition
SCiL Workshop, January 16, 2019

How much data is enough?
Predicting accuracy on large datasets from smaller pilot data,
talk presented at the
56th Annual Meeting of the Association for Computational Linguistics,
Melbourne, Australia.

What does Deep Learning tell us about Language?,
talk presented at the
Workshop on Relevance of Linguistic Structure in Neural NLP,
at the
56th Annual Meeting of the Association for Computational Linguistics,
Melbourne, Australia.

How much is enough? Predicting how system accuracy varies with training data size,
talk presented at the Macquarie University Centre for Language Technology meeting, 4th September 2017. Joint work with Dat Quoc Nguyen.

Generation in Image Captioning,
talk presented at the
1st Workshop on Learning to Generate Natural Language,
at
ICML 2017 Workshops, August 2017.
Joint work with Peter Anderson, Basura Fernando and Stephen Gould.

Natural Language Processing from a Machine Learning Perspective,
talk presented at the
The 8th International Workshop on Health Text Mining and Information Analysis (Louhiâ€™17),
August 2017.

Introduction to Hypothesis Tests,
talk presented at the CLT meeting, February 2017.

Computational Linguistics: Past, Present and Future,
talk to be presented at ALTA, December 2015.

Natural Language Processing from a Machine Learning Perspective,
talk given in November, 2015.

Rational Inferences and Bayesian Inferences,
talk presented at the
Rational Inferences Workshop,
Macquarie University, 30th October 2015

Introduction to Boltzmann Machines,
talk presented to the MQ Machine Learning Reading Group,
September 2015

Introduction to Neural Networks,
talk presented to the MQ Machine Learning Reading Group,
September 2015

Synergies in Language Acquisition,
talk presented at
The Workshop on Infant Speech Perception,
Macquarie University, September 2015.

A survey of parsing and its applications,
presented at Macquarie University, August 2015.

Nonparametric Bayesian Inference for Topical Collocation Models,
presented at Huwaei's Noah's Ark lab and the Chinese Academy of Sciences, July 2015.

Improving Topic Models with Latent Feature Word Representations,
presented at Huwaei's Noah's Ark lab and Tsinghua University, July 2015.

Sign constraints on feature weights improve a joint model of word segmentation and phonology,
presented in the Macquarie University Centre for Language Technology talk series,
March 2015.

Introduction to Computational Linguistics and Natural Language Processing,
presented at the
2015 Machine Learning Summer School,
in Sydney, Australia, 2015.
 My talk
Exploring the role of stress in word segmentation using Adaptor Grammars
for the
Workshop on the role of prosody at Macquarie University, December 2014.
 A series of talks on using R for data analysis:
 A talk on Crafting Scientific Papers,
presented at Macquarie University, October 2014.
 A series of introductory talks on machine learning and
natural language processing for the Macquarie University
Machine Learning Reading Group:
 Introduction to NLP,
presented at the CMCRC, August 2014.

Life Stories models for improving (text) data mining
and
Beyond the 1best pipeline,
presented at the NICTA NLP workshop, Canberra, July 2014.

Modelling function words improves unsupervised
word segmentation,
presented at the
ACL 2014 conference.

Exploring the role of stress in Bayesian word
segmentation using Adaptor Grammars,
presented at the
ACL 2014 conference.

Introduction to Computational Linguistics and Natural Language Processing,
presented at the
2014 Machine Learning Summer School,
in Beijing, June 2014.

Life Stories and Text Data Mining,
presented at the MQ Department of Computing seminar, May 2014.

Life Stories, a talk about
our new Googlesponsored research project with Wray Buntine,
Lan Du and Anish Kumar, presented at the 2014 Google NLU meeting,
Zurich, March 2014.

Synergies in Language Acquisition
(presented in AixenProvence and Paris)

Language acquisition as statistical inference,
talk presented at the
19th International Congress of Linguists,
Geneva, July 2013.
The
version of the talk
presented at the University of Edinburgh
has more technical details.

From PCFGs to Adaptor Grammars and Beyond,
slides from a talk given at NICTA, Canberra, May 2013.

A gentle introduction to Maximum Entropy,
loglinear, exponential, logistic, Harmonic,
Boltzmann, Markov Random Field, etc., models,
slides from a talk to the MQ
Machine Learning Reading Group, March 2013.

Natural language processing and computational linguistics:
from theory to applications,
talk presented at an MQ Informatics workshop, Macquarie University,
October 2012.

Learning words,
talk presented at the CLSP
Zero Resource Speech Technologies and Models of Early Language Acquisition
miniworkshop at Johns Hopkins University, July 2012.

Computational linguistics: where do we go from here?
invited plenary talk, presented at ACL 2012, July 2012, Jeju Island, Korea.

Grammars and Topic Models,
presented at the NICTA seminar, April, 2012.

Parsing in parallel on multiple cores and GPUs,
presented at the 2011 ALTA workshop.

My talk on
grammars
and the
extra slides
on DirichletMultinomials and Gibbs sampling
presented at the
2011
IPAM graduate summer school on probabilistic models of
cognition.

Bayesian models of Language Acquisition, or
Where do the rules come from?,
presented in the Stanford Linguistics department,
June 2011.

The impact of language models and loss functions on repair disfluency
detection, to be presented at
ACL HLT 2011.

Bayesian Inference for DirichletMultinomials
and
Adaptor Grammars.
The following slides introduce
probabilistic models for computational linguistics.
These slides will be presented at the
MLSS 2011
(the 2011 Machine Learning Summer School), Singapore.

Adaptor Grammars: A framework for Bayesian nonparametric grammatical inference,
November 2010.

Slides,
spotlight and
poster for presentation
at NIPS 2010 conference.

Detecting Speech Repairs Incrementally Using a Noisy Channel Approach
Simon Zwarts, Mark Johnson, Robert Dale, presented at COLING 2010.

Unsupervised phonemic Chinese word segmentation using Adaptor Grammars,
Mark Johnson and Katherine Demuth,
presented at COLING 2010.

Beyond rationalism versus empiricism.
Mark Johnson,
presented at the
RING talk series, COLING 2010.

PCFGs, Topic Models, Adaptor Grammars and Learning Topical Collocations and the Structure of Proper Names,
presented by Mark Johnson at ACL 2010, July 2010.

Reranking the Berkeley and Brown Parsers,
presented by Mark Johnson and Engin Ural at NAACLHLT 2010, June 2010.

My talk at the Macquarie University CLT group
on learning rules using Bayesian inference,
February 2010.

My talks at two NIPS 2009 workshops: on
grammar induction
and on
adaptor grammars and topic models.

Learning rules with Adaptor Grammars,
talk presented at U. Penn., November 2009.

Parsing and Learning, talk presented at
the International Workshop for Parsing Technology (IWPT),
October 2009, Paris, France.

Adaptor grammars: applications and inference,
July 2009.

Improving nonparameteric Bayesian inference:
experiments on unsupervised word segmentation
with adaptor grammars,
presented at the
NAACLHLT 2009 conference.

How the statistical revolution changes
(computational) linguistics,
presented at
The Interaction between Linguistics and Computational Linguistics: Virtuous, Vicious or Vacuous?
workshop,
2009 EACL conference.

Synergies in learning syllables and words, or Adaptor grammars: a class of nonparametric Bayesian models
presented at the
2008 Northeast Computational Phonology workshop, Yale, November 2008.

Using Adaptor Grammars to identify synergies in the unsupervised
acquisition of linguistic structure, given at the ACL 2008, Columbus, Ohio.

Unsupervised word segmentation for Sesotho using Adaptor Grammars
given at the ACL 2008 SIGMORPPHON workshop, Columbus, Ohio

Learning Rules with Adaptor Grammars
given at the University of Chicago and the
special edition
given at University of California, Berkeley.

Where do the rules come from?,
given at the University of Geneva, UIUC and the University of Massachusetts.

Bayesian models of language acquisition, or Where do the rules come from? at
the November 2007 conference at U Penn
Bridging the Developmental Divide:
Sentence Processing Meets Word and Grammar Learning

A talk for Eugene Charniak's computational linguistics class on
how to write new feature classes for the reranking parser.

A talk for the
Northeastern Computational Phonology meeting at U Mass in November, 2007
entitled A gentle introduction to Maximum Entropy models
and their friends.

Three talks given at the
2007 IPAM summer school on Probabilistic Models of Cognition: The Mathematics of Mind,
including
ContextFree Grammars and beyond,
Statistical models for natural language parsing,
and
Bayesian inference of grammars.

Transforming Projective Bilexical Dependency Grammars into efficiently parsable CFGs with UnfoldFold
by Mark Johnson at the
ACL 2007 conference.

Why doesn't EM find good HMM POS taggers?
presented by Mark Johnson at the EMNLP 2007 conference.

Bayesian Inference for PCFGs via Markov chain Monte Carlo
by Johnson, Griffiths and Goldwater at the 2007
HLT/NAACL 2007 conference
 Learning Grammar(s) Statistically,
invited talk at the University of Stuttgart, Opening colloquium for SFB 732,
November 2006.
 Powerpoint slides from the talk
"Distributional Cues to Word Boundaries: Context Is Important" by
Goldwater, Griffiths and Johnson at the 2006
Boston University Conference on Language Development, November 2006.
 Slides from my talk
"Features of Statistical Parsers" given at the joint Microsoft Research/University
of Washington computational linguistics colloquium, October 2006.
 Slides from my talk
``Collecting, err, correcting speech errors'' at the
the Eighth International Workshop on Tree Adjoining Grammar and Related Formalisms (TAG+8) at ACL/COLING 2006.
 Slides from my talk at
the
2006 Mayfest meeting at the University of Maryland.
The material in this talk is described in greater length in our
NIPS 2005 paper Interpolating between types and tokens by estimating powerlaw generators and our
ACL 2006 paper Contextual Dependencies in Unsupervised Word Segmentation.
 Slides from my invited talk at
the 2006 CUNY conference on parsing speech corpora (discusses statistical parsing of speech, disfluency detection, etc.).
 Slides from my talk on
NonParametric Bayesian Models for Natural Language at the
Pattern Theory group in Brown's Applied Maths department in March 2006.
This talk discusses how to define recursive Dirichlet processes based
on ContextFree Grammars.
 Slides from my talk at the
Linguistics department at the University of Hawaii.
 Slides from my invited talk at
the lunchtime symposium at
the
2005 Boston University Conference on Child Language Development
(a DVD of the symposium will also be available from the BUCLD web site).
 Slides
from my invited afterdinner talk at the
ESSLLI 2005 summer school
 Slides in 1up format and
the same slides in 4up format
for my class at the
ESSLLI 2005 summer school.
 I prepared some extra material on
Weighted CFGs for the ESSLLI'05 course,
which I didn't get around to presenting, but you might find them surprising.
I also presented my CoNLL'05 talk (the next item below) during the ESSLII'05 summer school course,
so you might want to look at those slides as well.
 Slides for my invited talk at the
CoNLL2005 conference
 Slides from my lecture in the
TTI summer school, May 2005.
"An introduction to grammars and parsing".
You may also find the paper I coauthored with Stu Geman useful:
"Probability and statistics in Computational Linguistics: A brief review"
 Slides from my talk at Cornell University, April 2005
"Features of Statistical Parsers"
 Slides from Johnson and Charniak's 2004 ACL talk
``A TAGbased
noisy channel model of speech repairs''
 Slides from
Ciaramita and Johnson's 2004 talk at the
Senseval workshop at ACL'04
 Slides from Hall and Johnson's 2004 ACL talk
``Attention shifting for parsing speech''
 Slides from Goldwater and Johnson's 2004 talk at the
SIGPHON workshop at ACL'04
``Priors in Bayesian Learning of Phonological Rules
 Slides from a talk at Stanford
Collecting, err, correcting speech errors
given in November, 2004.
 Slides from my invited talk at the IJCNLP on Hainan Island, China,
in March 2004 on
Probabilistic Parsing with a Wide Variety of Features
 Slides from my talks at the University of Tokyo in March 2004 on
discriminative approaches
to statistical parsing
and on statistical parsing
and speech recognition.
 Slides from my 2003 ALTA summer school course on
Grammars, graphs and automata
 Slides from my 2003 talk at the Toyota Technical Institute at
the University of Chicago on
Features of Statistical Parsers
 Slides from my 2003 invited talk at COLT 2003 on
Learning and parsing stochastic
unificationbased grammars
 Stochastic ``unificationbased'' grammars, a talk given at Edinburgh in
January 2002, which describes how to calculate the statistics needed
for parsing and estimation from packed feature structures.
Available in Adobe Acrobat
format.
 Slides from my talks at LFG 2002
and ACL'02 (on dynamic programming in SUBGs)
and ACL'02 (on empty nodes) (all in PDF).
 Slides on
Compact nonleftrecursive grammars using the selective leftcorner transform,
based on Johnson and Roark (2000), presented at COLING 2000.
 My ACL 1998 talk on leftcorner grammar approximations
(the paper from the ACL 1998 proceedings is here
and the bib file is here).
If you like this kind of stuff, you might like
my notes on additional leftcorner related transforms.
 A set of slides describing
Neal and Hinton's
(1998) paper "A view of the EM algorithm that justifies incremental, sparse, and other variants"
 Slides on CRFs and their relationship to HMMs
 Slides on estimating MaxEnt models and the relationship between Stochastic Gradient Descent and the Percepton