Deploying AI models at scale
Vivienne Zhang is a dual Master’s student in the Technology and Policy Program and the Department of Electrical Engineering and Computer Science (EECS) at MIT. She does research at the MIT Center for Energy and Environmental Policy Research (CEEPR).
What is the focus of your research? What sort of knowledge and disciplines does it bring together? How will it make an impact?
I do research on how machine learning and data science can be used to improve the predictability of wind power generation. Specifically, I am interested in how improving data quality can lead to much better predictions, often more so than experimenting with the model architecture itself.
This research brings together the disciplines of computer science, economics and policy. The models and data inputs fall within the realm of computer science, but there is also the question of whether the investment in developing the improved models for prediction is worthwhile. For instance, better predictions for wind power generation usually result in higher earnings for a power plant owner, but to get these results, they would need to invest in building and maintaining meteorological towers, for instance, which costs money. This is an economics question. Then there is the policy question of what kind of energy market design is needed in order to manage renewable resources more efficiently by making use of newer technologies like machine learning.
My research is based on the actual data of a utility, which is a member at my research center, CEEPR. The results help them think about the current technologies they use for predicting their wind power generation and how they would invest in machine learning to improve predictability of their wind power plants going forward. I hope that this learning can be generalized as well. I have seen a lot of interest and literature on building complex machine learning models for predicting wind power generation, but not much is done with the data used in prediction. The data I extracted from the National Digital Forecast Database for my thesis, to my knowledge, have never been used before. Hopefully my research can draw attention to data quality, because it definitely makes a difference.
This summer you interned with Nvidia. Who did you work with and what did you do there?
I worked with the Deep Learning software team at Nvidia. Nvidia is known for designing GPUs for computer graphics. In the last decade, the company has invested heavily in becoming the compute platform for AI applications. My team built products including CUDA, tensorRT, and most recently large language models like NeMo. I worked as a product manager where I led an early access program to build new functionalities for Triton, Nvidia’s open-source inference serving software for deploying AI models at scale.
How did the internship connect to your current research and future plans?
It helps to know that putting machine learning to use is not just about building and tuning models, but the entire process, from establishing a data pipeline, to inferencing, to effective visualization. The process is called Machine Learning Operations (MLOps). For instance, I could build a good machine learning model that gives predictions of wind power generation for a power plant, but the prediction is dependent on hourly weather data from three nearby weather stations. How do we pull the data into the model, do the inference efficiently, and display the predictions in a helpful and timely manner to the data user? As a researcher focused on the model itself, I didn’t have to worry about what happens outside of the model on a daily basis. But my internship asks me to think about these MLOps questions because Triton, the product I worked on at Nvidia, is designed to deploy AI models.
In terms of future plans, I will be returning to Nvidia to join the team full-time. I absolutely loved the team, the work, and the company culture in the summer. It was great to get a glimpse of Nvidia before making the big commitment. I feel lucky to have the confidence that they are the right place for me.
