Enrolment information: NZ Citizens, NZ Permanent Residents, International Introduction The overall goal of Artificial Intelligence is to give computers the capabilities that define human intelligence, without necessarily replicating human intelligence exactly. These capabilities can be seen as being provided, roughly, by two interacting systems:  1) fast, reactive, and largely unconscious perception and action, and 2) slow, deliberative and largely conscious integration, synthesis, planning, and evaluation. Another way to explain the divide is that system 1 is largely non-symbolic, and system 2 largely symbolic.  Both of these have been the subject of an enormous amount of research in AI, but recently advances in deep learning and computational infrastructure have let to great advances in less-symbolic AI, giving us systems that are very capable in vision, speech recognition and synthesis, and, more surprisingly, game playing, language translation, and simple question answering. But these advances have all perfected particular skills, not broad competences. The other branch of AI is about knowledge and reasoning: representing skills for reuse, composing skills to solve novel problems, and deliberating on those solutions, for example, to provide explanations; in short it is about reasoning.  Towards Broader AI, skill learning and reasoning might be unified to allow computers to solve complicated problems using skills and knowledge, and move us closer to more general AI. The Broad AI Lab works on learning-based general artificial intelligence, with a focus on complex problem solving with Natural Language (NL); this is a very challenging area of AI research, with high potential commercial impact. We will combine the current Deep Learning revolution in AI with techniques derived from symbolic AI, including Knowledge Representation, Knowledge Capture, and Automated Reasoning, to give computers the powers of understanding and integration. This program is driven by the observation that knowledge can be seen exactly that which speeds up the learning of a new task, and that the main store of such existing knowledge is natural language.  While our long-term goal of human level understanding and reasoning is challenging, near-term advances in understanding text, diagrams and tables so they can be automatically repurposed and combined to answer questions are in reach and could be readily commercialised.

Enrolment information: NZ Citizens, NZ Permanent Residents, International Introduction Transformers, a recently invented class of deep neural network (DNN) models, have been successfully applied to a wide range of machine learning tasks ranging from natural language processing and image recognition to molecular biology. The core idea that led to these unprecedented successes of the Transformer model is its attention mechanism, which allows the network to learn to identify and prioritise informative aspects of the data. In molecular biology, this mechanism allows a network to identify the amino acid substitution matrix, binding sites and other biologically relevant positions in protein sequences. Google's AlphaFold 2 exploited this idea to accurately predict the tertiary structure of a protein given its amino acid sequence, one of the most important problems in molecular biology. Facebook’s MSA Transformer extends the network’s architecture from sequence data to multiple sequence alignments (MSAs), a typical data type in molecular biology applications. The MSA Transformer, trained on MSAs of proteins in an unsupervised way, is capable of predicting the structure and function of proteins. A key aspect of the MSA Transformer is that attention interleaves between rows (protein sequences) and columns (amino acid positions of the alignment) to leverage information from evolutionary related sequences across many protein families. This advance beyond the sequence-based data which characterised the original Transformer architectures offers great promise for a wide range of applications in computational evolutionary and systems biology. Thus, the stage is set to extend the remarkable performance of attention-based DNNs in predicting phenotype from protein sequence and MSA data to other molecular biology domains including genetic variation. The current gold standard for linking common genetic variations found in the general population to phenotype and disease is the genome-wide association study (GWAS). Despite being a powerful and commonly used statistical technique, GWAS suffer major limitations. Indeed, even the most advanced GWAS of European populations explain less than 7% of the variance in serum urate levels. In this project our overall aim will be to develop an interpretable, attention-based approach to GWAS which accounts for genetic interactions, is capable of modelling multidimensional phenotypes, and scales to the entire human genome. We will develop our approach specifically for GWAS data for serum urate levels and use this to focus on gout-related genetic variation in Maori and Pasifika populations. The project requires to: - Design, implement, and train an attention-based DNN suitable for large-scale genotype-phenotype data. - Interpret and analyse the results obtained with attention-based DNN for the UKBB gout data from both molecular biology and clinical points of view; identify aspects of the network architecture and data analysis pipeline that can be improved; extend the pipeline to account for biases due to European ancestry of UKBB data. - Unify our results with the natural language understanding (NLU) domain.