I am a PhD candidate in computer science at Cornell University, working with Carla Gomes. My research focus is on sparse and Bayesian optimization. On the applied side, I have collaborated with materials scientists in leveraging active learning for the discovery of novel meta-stable materials. After graduation, I will join Meta as a research scientist.
When I am not penciling Greek letters or hunting down missing minus signs in code, I enjoy cycling, dancing tango and playing the piano. Hear me play a tango that I transcribed here.
Bayesian Optimization (BO) has shown great promise for the global optimization of functions that are expensive to evaluate, but despite many successes, standard approaches can struggle in high dimensions. To improve the performance of BO, prior work suggested incorporating gradient information into a Gaussian process surrogate of the objective, giving rise to kernel matrices of size nd × nd for n observations in d dimensions. Naïvely multiplying with (resp. inverting) these matrices requires O(n^2d^2) (resp. O(n^3d^3)) operations, which becomes infeasible for moderate dimensions and sample sizes. Here, we observe that a wide range of kernels gives rise to structured matrices, enabling an exact O(n^2d) matrix-vector multiply for gradient observations and O(n^2d^2) for Hessian observations. Beyond canonical kernel classes, we derive a programmatic approach to leveraging this type of structure for transformations and combinations of the discussed kernel classes, which constitutes a structure-aware automatic differentiation algorithm. Our methods apply to virtually all canonical kernels and automatically extend to complex kernels, like the neural network, radial basis function network, and spectral mixture kernels without any additional derivations, enabling flexible, problem-dependent modeling while scaling first-order BO to high d.
@inproceedings{ament2022fobo,title={Scalable First-Order Bayesian Optimization via Structured Automatic Differentiation},author={Ament, Sebastian and Gomes, Carla},booktitle={International Conference on Machine Learning},pages={},year={2022},organization={PMLR},}
Sci. Adv.
Autonomous materials synthesis via hierarchical active learning of nonequilibrium phase diagrams
Ament, Sebastian, Amsler, Maximilian, Sutherland, Duncan R., Chang, Ming-Chiang, Guevarra, Dan, Connolly, Aine B., Gregoire, John M., Thompson, Michael O., Gomes, Carla P., and Dover, R. Bruce
Artificial intelligence accelerates the search and discovery of new metastable materials for energy applications. Autonomous experimentation enabled by artificial intelligence offers a new paradigm for accelerating scientific discovery. Nonequilibrium materials synthesis is emblematic of complex, resource-intensive experimentation whose acceleration would be a watershed for materials discovery. We demonstrate accelerated exploration of metastable materials through hierarchical autonomous experimentation governed by the Scientific Autonomous Reasoning Agent (SARA). SARA integrates robotic materials synthesis using lateral gradient laser spike annealing and optical characterization along with a hierarchy of AI methods to map out processing phase diagrams. Efficient exploration of the multidimensional parameter space is achieved with nested active learning cycles built upon advanced machine learning models that incorporate the underlying physics of the experiments and end-to-end uncertainty quantification. We demonstrate SARA’s performance by autonomously mapping synthesis phase boundaries for the Bi2O3 system, leading to orders-of-magnitude acceleration in the establishment of a synthesis phase diagram that includes conditions for stabilizing δ-Bi2O3 at room temperature, a critical development for electrochemical technologies.
@article{ament2021sara,author={Ament, Sebastian and Amsler, Maximilian and Sutherland, Duncan R. and Chang, Ming-Chiang and Guevarra, Dan and Connolly, Aine B. and Gregoire, John M. and Thompson, Michael O. and Gomes, Carla P. and van Dover, R. Bruce},title={Autonomous materials synthesis via hierarchical active learning of nonequilibrium phase diagrams},journal={Science Advances},volume={7},number={51},pages={eabg4930},year={2021},doi={10.1126/sciadv.abg4930},url={https://www.science.org/doi/abs/10.1126/sciadv.abg4930},}
ICML
Sparse Bayesian Learning via Stepwise Regression
Ament, Sebastian, and Gomes, Carla
In International Conference on Machine Learning 2021
Sparse Bayesian Learning (SBL) is a powerful framework for attaining sparsity in probabilistic models. Herein, we propose a coordinate ascent algorithm for SBL termed Relevance Matching Pursuit (RMP) and show that, as its noise variance parameter goes to zero, RMP exhibits a surprising connection to Stepwise Regression. Further, we derive novel guarantees for Stepwise Regression algorithms, which also shed light on RMP. Our guarantees for Forward Regression improve on deterministic and probabilistic results for Orthogonal Matching Pursuit with noise. Our analysis of Backward Regression culminates in a bound on the residual of the optimal solution to the subset selection problem that, if satisfied, guarantees the optimality of the result. To our knowledge, this bound is the first that can be computed in polynomial time and depends chiefly on the smallest singular value of the matrix. We report numerical experiments using a variety of feature selection algorithms. Notably, RMP and its limiting variant are both efficient and maintain strong performance with correlated features.
@inproceedings{ament2021sparse,title={Sparse Bayesian Learning via Stepwise Regression},author={Ament, Sebastian and Gomes, Carla},booktitle={International Conference on Machine Learning},pages={264--274},year={2021},organization={PMLR},}
