Louis Tiao

Louis Tiao

PhD Candidate

University of Sydney

CSIRO Data61

About

Hi. My name is Louis. I am a machine learning researcher and PhD candidate at the University of Sydney. My main research interests are in topics that lie at the intersection of Gaussian processes, Bayesian deep learning, variational inference, and their applications in Bayesian optimization and graph representation learning.

Previously, I was a research software engineer at National ICT Australia (NICTA), which later folded into Data61, the artificial intelligence research arm of the Commonwealth Scientific and Industrial Research Organisation (CSIRO). Prior to that, I studied computer science at the University of New South Wales, where I had a major emphasis on algorithm design and analysis, theoretical computer science, programming language theory, artificial intelligence, machine learning, and a minor emphasis on mathematics and statistics.

Education

  • Ph.D. in Computer Science

    University of Sydney

  • B.Sc. (Honours 1st Class) in Computer Science

    University of New South Wales

Experience

Research Experience

 
 
 
 
 

Visiting Researcher

Secondmind

Sep 2021 – Dec 2021 Cambridge, United Kingdom
I will be spending the Fall of 2021 as a Placement Researcher at Secondmind (formerly known as PROWLER.io), a machine learning technology start-up based in Cambridge, UK with a strong focus on basic research and an excellent track record of scientific publications in the areas of Gaussian processes, variational inference, and Bayesian optimization (you may know them by their popular open-source library, GPFlow).
 
 
 
 
 

Applied Scientist (Intern)

Amazon

Jun 2019 – Dec 2019 Berlin, Germany
I spent the Summer-Fall of 2019 at Amazon in Berlin, Germany, conducting research in the area of AutoML in contribution to the Automatic Model Tuning service on AWS SageMaker. I was fortunate to been given the opportunity to work with eminent researchers in the field, Matthias Seeger, Aaron Klein, and Cédric Archambeau. Together, we tackled the challenges of extending multi-fidelity Bayesian optimization with asynchronous parallelism. The research developed during my internship culminated in a research paper and the release of our code as part of the open-source AutoGluon library.
 
 
 
 
 

PhD Candidate

University of Sydney

Jul 2017 – Present Sydney, Australia
 
 
 
 
 

Software Engineer

CSIRO’s Data61

Jul 2016 – Apr 2019 Sydney, Australia

After NICTA was subsumed under CSIRO (Australia’s national science agency), I continued as a member of the Inference Systems Engineering team, working to apply probabilistic machine learning to a multitude of problem domains, including spatial inference and Bayesian experimental design, with an emphasis on scalability. In this time, I led the design and implementation of new microservices and contributed to the development of open-source libraries for large-scale Bayesian deep learning.

During this period, I also served a brief stint with the Graph Analytics Engineering team (the team behind StellarGraph), where I contributed to research into graph representation learning from a probabilistic perspective. These efforts culminated in a research paper that went on to be awarded a spotlight presentation at the field’s premier conference.

 
 
 
 
 

Software Engineer

National ICT Australia (NICTA)

May 2015 – Jun 2016 Sydney, Australia
As a software engineer with a specialization in machine learning, I was a member of a team of machine learning researchers and engineers engaged in an interdisciplinary collaboration with leading researchers from multiple areas of the natural sciences, as part of the Big Data Knowledge Discovery initiative sponsored by the Science Industry Endowment Fund (SIEF). During this time I helped lead the development and release of numerous open-source libraries for applying Bayesian machine learning at scale.
 
 
 
 
 

Research Intern

Commonwealth Scientific and Industrial Research Organisation (CSIRO)

Nov 2013 – Feb 2014 Sydney, Australia
I joined the CSIRO’s Language and Social Computing team as a Summer Vacation Scholar for the summer of 2013-14 and worked on applying machine learning and natural language processing (NLP) techniques to develop a text classification system for automated sentiment analysis.

Projects

AutoGluon

AutoGluon is a library that implements numerous state-of-the-art methods for asynchronously distributed hyperparameter optimization (HPO) and neural architecture search (NAS). I was a core developer of the Gaussian process-based multi-fidelity searcher module.

Variational Bayes for Implicit Probabilistic Models

Expanding the scope and applicability of variational inference to encompass implicit probabilistic models.

Aboleth

Aboleth is a minimalistic TensorFlow framework for scalable Bayesian deep learning and Gaussian process approximation.

Determinant

Determinant is a software service that makes predictions from sparse data, and learns what data it needs to optimise its performance.

Revrand

Revrand is a full-featured Python library for Bayesian generalized linear models, with random basis kernels for large-scale Gaussian process approximations.

Teaching

COMP9418: Advanced Topics in Statistical Machine Learning ( UNSW Sydney)

The course has a primary focus on probabilistic machine learning methods, covering the topics of exact and approximate inference in directed and undirected probabilistic graphical models - continuous latent variable models, structured prediction models, and non-parametric models based on Gaussian processes.

Lab exercise on Gaussian Process Regression, running in JupyterLab.

This course has a major emphasis on maintaining a good balance between theory and practice. As the teaching assistant (TA) for this course, my primary responsibility was to create lab exercises that aid students in gaining hands-on experience with these methods, specifically applying them to real-world data using the most current tools and libraries. The labs were Python-based, and relied heavily on the Python scientific computing and data analysis stack ( NumPy, SciPy, Matplotlib, Seaborn, Pandas, IPython/Jupyter notebooks), and the popular machine learning libraries scikit-learn and TensorFlow.

Students were given the chance to experiment with a broad range of methods on various problems, such as Markov chain Monte Carlo (MCMC) for Bayesian logistic regression, probabilistic PCA (PPCA), factor analysis (FA) and independent component analysis (ICA) for dimensionality reduction, hidden Markov models (HMMs) for speech recognition, conditional random fields (CRFs) for named-entity recognition, and Gaussian processes (GPs) for regression and classification.

Recent Posts

Publications

(2021). BORE: Bayesian Optimization by Density-Ratio Estimation. In ICML2021. Accepted as Long Presentation (Awarded to Top 3% of Papers).

Preprint PDF Code Poster Slides Video Conference Proceeding Supplementary material

(2020). BORE: Bayesian Optimization by Density-Ratio Estimation. In NeurIPS2020 Meta-Learn. Accepted as Contributed Talk (Awarded to Best 3 Papers).

Preprint PDF Code Poster Slides Video Supplementary material

(2020). Variational Inference for Graph Convolutional Networks in the Absence of Graph Data and Adversarial Settings. In NeurIPS2020. Accepted as Spotlight Presentation (Awarded to Top 3% of Papers).

Preprint PDF Code Video

(2020). Model-based Asynchronous Hyperparameter and Neural Architecture Search. Preprint.

PDF Code Video

(2019). Variational Graph Convolutional Networks. In NeurIPS2019 Graph Representation Learning. Accepted as Outstanding Contribution Talk (Awarded to Best 3 Papers).

PDF Code Video Workshop Homepage