Louis Tiao

Louis Tiao

PhD Candidate

University of Sydney

CSIRO Data61

About

Hi. My name is Louis. I am a machine learning researcher and PhD candidate at the University of Sydney. My main research interests lie at the intersection of Bayesian deep learning, approximate inference, and probabilistic models with intractable likelihoods.

Previously, I was a research software engineer at NICTA (now incorporated under CSIRO as Data61) in the inference systems engineering group, working on scalable probabilistic machine learning. Prior to that, I studied computer science at the University of New South Wales, where I had a major emphasis on algorithm design and analysis, theoretical computer science, programming language theory, artificial intelligence, machine learning, and a minor emphasis on mathematics and statistics.

Education

  • Ph.D. in Computer Science

    University of Sydney

  • B.Sc. (Honours 1st Class) in Computer Science

    University of New South Wales

Experience

Research Experience

 
 
 
 
 

Applied Scientist (Intern)

Amazon

Jun 2019 – Dec 2019 Berlin, Germany
Conducted research in the area of AutoML, in contribution to the AWS SageMaker Automatic Model Tuning service. I had the good fortune of working with Matthias Seeger and Cédric Archambeau, and together, we tackled the challenges of extending multi-fidelity Bayesian optimization with asynchronous parallelism. The research developed during my internship culminated in a paper and the release of our code as part of the open-source AutoGluon library.
 
 
 
 
 

PhD Candidate

University of Sydney

Jul 2017 – Present Sydney, Australia
 
 
 
 
 

Software Engineer

CSIRO’s Data61

Jul 2016 – Apr 2019 Sydney, Australia

After NICTA was subsumed under CSIRO (Australia’s national science agency), I continued as a member of the Inference Systems Engineering team, working to apply probabilistic machine learning to a multitude of problem domains, including spatial inference and Bayesian experimental design, with an emphasis on scalability. In this time, I led the design and implementation of new microservices and contributed to the development of open-source libraries for Bayesian deep learning.

During this period, I also served a brief stint with the Graph Analytics Engineering team (the team behind StellarGraph), where I contributed to research into graph representation learning from a probabilistic perspective. These efforts culminated in a research paper that went on to be awarded a spotlight presentation at the field’s premier conference.

 
 
 
 
 

Software Engineer

National ICT Australia (NICTA)

May 2015 – Jun 2016 Sydney, Australia
As a software engineer with a specialization in machine learning, I was a member of a team of machine learning researchers and engineers engaged in an interdisciplinary collaboration with leading researchers from multiple areas of the natural sciences, as part of the Big Data Knowledge Discovery initiative sponsored by the Science Industry Endowment Fund (SIEF). During this time I helped lead the development and release of numerous open-source libraries for applying Bayesian machine learning at scale.
 
 
 
 
 

Research Intern

Commonwealth Scientific and Industrial Research Organisation (CSIRO)

Nov 2013 – Feb 2014 Sydney, Australia
I joined CSIRO’s Language and Social Computing team as a Summer Vacation Scholar for the summer of 2013-14 and worked on applying machine learning and natural language processing (NLP) techniques to develop a text classification system for automated sentiment analysis.

Projects

AutoGluon

AutoGluon is a library that implements numerous state-of-the-art methods for asynchronously distributed hyperparameter optimization (HPO) and neural architecture search (NAS). I was a core developer of the Gaussian process-based multi-fidelity searcher module.

Variational Bayes for Implicit Probabilistic Models

Expanding the scope and applicability of variational inference to encompass implicit probabilistic models.

Aboleth

Aboleth is a minimalistic TensorFlow framework for scalable Bayesian deep learning and Gaussian process approximation.

Determinant

Determinant is a software service that makes predictions from sparse data, and learns what data it needs to optimise its performance.

Revrand

Revrand is a full-featured Python library for Bayesian generalized linear models, with random basis kernels for large-scale Gaussian process approximations.

Teaching

COMP9418: Advanced Topics in Statistical Machine Learning ( UNSW Sydney)

The course has a primary focus on probabilistic machine learning methods, covering the topics of exact and approximate inference in directed and undirected probabilistic graphical models - continuous latent variable models, structured prediction models, and non-parametric models based on Gaussian processes.

Lab exercise on Gaussian Process Regression, running in JupyterLab.

This course has a major emphasis on maintaining a good balance between theory and practice. As the teaching assistant (TA) for this course, my primary responsibility was to create lab exercises that aid students in gaining hands-on experience with these methods, specifically applying them to real-world data using the most current tools and libraries. The labs were Python-based, and relied heavily on the Python scientific computing and data analysis stack ( NumPy, SciPy, Matplotlib, Seaborn, Pandas, IPython/Jupyter notebooks), and the popular machine learning libraries scikit-learn and TensorFlow.

Students were given the chance to experiment with a broad range of methods on various problems, such as Markov chain Monte Carlo (MCMC) for Bayesian logistic regression, probabilistic PCA (PPCA), factor analysis (FA) and independent component analysis (ICA) for dimensionality reduction, hidden Markov models (HMMs) for speech recognition, conditional random fields (CRFs) for named-entity recognition, and Gaussian processes (GPs) for regression and classification.

Recent Posts

Publications

(2020). BORE: Bayesian Optimization by Density-Ratio Estimation. In NeurIPS2020 Meta-Learn. Accepted as Contributed Talk (Awarded to Best 3 Papers).

Preprint PDF Code Poster Slides Video Supplementary material

(2020). Variational Inference for Graph Convolutional Networks in the Absence of Graph Data and Adversarial Settings. In Advances in Neural Information Processing Systems 33 (NeurIPS2020). Accepted as Spotlight Presentation (Awarded to Top 3% of Papers).

PDF Code Video

(2020). Model-based Asynchronous Hyperparameter and Neural Architecture Search. Preprint.

PDF Code Video

(2019). Variational Graph Convolutional Networks. In NeurIPS2019 Graph Representation Learning. Accepted as Outstanding Contribution Talk (Awarded to Best 3 Papers).

PDF Code Video Workshop Homepage

(2018). Cycle-Consistent Adversarial Learning as Approximate Bayesian Inference. In ICML2018 Theoretical Foundations and Applications of Deep Generative Models. Accepted as Contributed Talk..

Preprint PDF Code Project Poster Slides Workshop Homepage