giving the Zoom talk, um, at what's, like, 2 in the morning, 2.30 in the morning for you.
Um… but yeah, we're making the best of a…
tough situation, I think. Um, yeah, so I… I'm happy to introduce Yuan Seng. I, um, met him in…
Berlin, I think, or maybe we met… no, we met once before at MIT, maybe.
Um, at a workshop there. Anyway, um…
Yeah, we ran into each other in…
Berlin over the summer, um, and Yun Seng is doing very interesting things on…
Uh, using agents, AI, and agents for, um, research in Astro.
Uh, and since this is one of, uh, our interests here, both agents and also astroparticles,
Um, I thought it'd be great to invite him to give this talk.
And, um…
And let's see… yeah, so I… oh, and Sen also has the distinction of having worked on…
Uh, a topic that we also are very active in here, very close to our heart, which is this, uh, you know, using the kinematics of stars to probe the potential and mass density, um, using data, for instance, from the Gaia Space Telescope.
So that was, I guess, your PhD research with Gregory? Or…
Post thought.
No, so that was… yeah, that was, uh, a postal, yes.
Yeah.
Postdoc research, okay, postdoc research with, um, Gregory, yeah. So, so yeah, so we have a lot in common, a lot to talk about, I think.
And, um, and yeah, so I think without further ado, please take it away.
Okay, uh, let me share screen…
Sorry, I'm still, like, recovering from…
Oh, cool. Um, okay, can people see my screen?
Yeah. Yeah.
Okay, great, uh, thank you, David. Uh, and as David mentioned, I am really grateful that, uh, for, uh, you know, uh,
for you to indulge in me to give a remote talk.
Um, yep, so I would like very much to be there in person, but I couldn't do that.
Um, and I should also mention that, you know, I will be more than, like, happy to have, uh, a personal Zoom meeting
Uh, afterwards, um, maybe not right after, because it's…
When it's finished, like, it will be 3…
30 AM, but, uh, like, any time this week, next week, if you want to talk more, I will be, uh, be happy to chat.
Okay, so, uh, as Debbie mentioned, you know, um,
Today, I want to focus on AI Agent. It's something that I thought it was a hobby two years later, but it has
gain more and more prominence in the field, but also, you know, my thinking for AI, like, agent has changed over time. I find that to be more fascinating, so I spend more and more time
Uh, on it. But let me start by…
talking about why I think AI Agent is particularly interesting for physics and astrophysics.
Start me with the basic, we all know that, you know, there are lots of, like, interests for AI, for science, if you look at NSF.
Uh, you know, for all the $200 million on AI Research Institute, half of them is,
For stamps, right? And including the new two AI Center for Astronomy, one at UT Austin, and one at Northwestern.
Um, of course, then we also have the DOE, the Genesis program. There are lots of, like, interests with a lot of, like, money.
Like, flying, like, around, then the question, you know, I often got asked even,
a decade let it go, when I started to do more on deep learning, people, like, always ask me about,
If this is a hype, a myth, or a real deal, you know, this is a thing that, when I was at the IAS, uh, like, I was an odd one out that, you know, get, like, fascinating about deep learning, when…
Most of the physics is still quite less skeptical.
Um, I would think that the answer is all three of them, and I will explain why that is the case.
Um, I think the proponent for AI, for science will always point to this one example that we love, which is the Alpha foe.
Uh, which is not a terrible example, because certainly, you know, if you look at
how we do, uh, chemistry, it has been completely transformed because of AI, right?
But, uh, the follow-up question that I often got asked is, uh, why that… what would be the equivalent of the Alpha foe?
for astronomy. Um, to answer that question, I think, like, there will not be an equivalent
answer for that. But to answer why there is no equilibrium, uh, no equivalent in astronomy,
wanted to explain a little bit what we have been doing, right, in terms of astronomy, so I know that not all people here are astronomers.
Uh, for people who are interested in what has, like, happened in the last decade, I will point to
my end-loader review. I spent, like, almost a half a year, like, writing these pieces. Uh, but I will not talk about anything in that piece, because this is a summary of what I think has happened in the last decade.
It really comes down to, like, what has happened in the last decade, I think both in physics, but also in astronomy.
Uh, people have been using, like, AI to extend, uh, statistical methods, right?
in cosmology, for example, we are always wanting to come up with a better way to do, uh, inferences without, like, relying on some human, uh, heuristic. So we want to do some Bayesian inferences.
But we don't have a very good models. So how do you, like, really…
connect the simulation with the observation. So, lots of the AI really comes down to this idea, now become quite a popular
in physics and astrophysics called the simulation base.
our inferences. And this is something that I work, like, a lot in my, uh, Lear Before chatbot at ILM. I think these are still important, but, like, these are the things that I would call that
individual tasks are AI, right? So in the timeframe of from 2013 to maybe, like, 2022,
Like, a lot of the AI that we have been, like, dealing with are sort of the classical deep learning rearm. We are training
a regression models, we are, like, training a probabilistic model, a classification of models.
to do individual tasks better. And some of that, of course, is important, as David mentioned, you know, like, just understanding how to
model the kinematics of star without having an explicit
our models are still useful.
But I choose not to talk about those topics, because I also find that those topics will
unlikely to have solved the very profound, like, impact.
in astronomy, not the 0 to 1.
type of, uh, of impact.
And why that is the case, uh…
It really comes down to both
fundamental physics, both in the particle physics and, uh, as well as astronomy.
I would argue that the complexity that we are dealing with is too low for AI.
Right, so if you're thinking about, like, chemistry or biology or material science,
of course, the complexity is high, and this is why, you know, like, AI can do the wonderful jobs, like the alphaFo.
But, like, in astronomy, the…
the analogy that I love to give is that my niece has more non-Gaussian information comparing to the entire
our universe. Like, a lot of things, the thing that we deal with,
Uh, only weekly non-Gaussian, or weekly non-linear.
Um, and this is why, because of the complexity is low, the individual task is unlikely to make a big impact. So if you think about what is the individual task,
quite often, you have some data, you have some theory, you try to analyze the data, and decide if the theory is true or false.
Like, in the case of, like, chemistry or biology, they are bottlenecked by some of these tasks, right? Thinking about, like, protein folding,
There are simply no good human heuristic.
Uh, that can do the job. So this is why when the alpha fall, like, happened, it has solved one task,
But that task is so important that, like, it would have a zero-to-one type of,
Tried up our transformation in the field.
But because the astronomy complexity is much lower, if you think about much of the things that we deal with in astronomy or particle physics,
We already have a… not a very bad
way of dealing with that, right? We have good human, uh, uh…
a heuristic that know how to deal with that, thinking about large-scale structure, we know that we need to go slightly beyond the power spectrum, but we have some way to do that.
So,
So, like, even though I think, you know, those study of SBI and all this,
can be important, but they are always at the level of improve, you know, 5 to 20% of the inferences.
And that itself might be important, right, if you want to detect
I want to, like, constrain dark matter, if you can, like, improve by 10%, it's a lot.
But still, you know, I want to choose to talk about, like, topics that I think is…
more, uh, avant-garde, but not less established, but I think can have a bigger, like, impact in astronomy with, uh, the way she's thinking about using AI, uh,
agent to expedite the discovery.
So, um, when I give this talk to two years ago, I need to…
explain what is agent, what is a large… a language models.
But I think I no longer need to do that, because now you're, like, becoming quite a common thing that people hopefully know, right? So, like, sort of the idea of using
agent using large, uh, a language model to solve a problem, meaning that we try to use
Asian as a pseudo-human. So instead of dealing with individual tasks with deep learning,
We are using the agent to make plans, and we are using
some of the, uh, the reasoning ability of the agent to solve more holistic tasks.
Um, and you also shouldn't surprise people that, you know, like, people like myself, and also I know, like, in rugust, people are thinking about that, just because, especially…
Since last year, it seems to have a phase shift in terms of the ability of
of AI to do real things.
Right? Since last year, uh, the AI has started to,
gain gold medals in all sort of, uh, Olympiads, including the math, the physics, the informatics.
Uh, at OSU, we look into the astronomy Olympiad, because I coached the team in Malaysia, so…
And those are people, and we also have a student just turned out to be
the team, uh, leader in the US.
So, we are, uh, uh…
are able to work with the astronomy Olympiad, and also have the grading.
Uh, that is aligned with how the students are being graded. Long story short, like, even in astronomy, like, Olympia, of course, it won the gold medal, but more importantly,
If you look at the score, even with the last year, like, models,
No student would have beaten Legit 5.
So, the GBD5 will have won not only the gold medal, uh, like, it will rank the first place.
in the entire comp… well, a competition.
Uh, and this type of, like, high school competition, of course, is not cutting-edge research.
But it's also not, like, simple, right? So this is one of the theory questions about,
a neutron star by a binary.
So, a bigger question is, like, we know that the AI can do some math. The question is whether or not in the open world, like, setting, especially
like, in the task that the AI are not trained to do.
Can AI also reach
somewhat a human, uh, researcher.
allowable.
So these are something that I got interested two years ago. So, the first attempt that we tried to do…
Uh, in 2024 is looking at the James Webb data, right? So we know that the James Webb data is exciting, and that a lot of people are analyzing the data, but we also know that from just collecting the James Webb data,
to publishing the paper, but there are still many human intervention is needed.
So, if, for example, if you take the James Webb data, it's what we call the SED of the galaxy. The SED stands for Spectral Energy Distribution. You can think about this as a
call us a resolution spectrum. And the game that the astronomer play is try to find a physical model that can go through the data point.
Right? Of course, the challenges here is that there are many, many physical models that we can assume, so there's no one fifth
One physical model, and you just do a parameter optimization. So there are many, like, building blocks of, like, physical processes.
that you can add into the models.
So, of course, if you have a James Webb data, and you have your favorite student, you will give your data to your students.
And ask your student to first fit a baseline of models, right? So make some physical assumption and try to fit the models.
And quite often, of course, your student will come back and say, hey, you know, the model seems okay, but there are some discrepancy.
Right? So, like, as a good supervisor, what you would do next is not to ask your student to, you know, exhaustively try all the models.
What you would try to do is to ask, okay, we can look at some of the trends and decide what are the physical, like, assumptions that we need to change. So in this case, for example, if you see
That is a wavelength-dependent discrepancy. Maybe the
the solution here is to change the dust.
properties, because we know that the grain size of the dust can…
create a wavelength-dependent discrepancy.
Or you can focus on the ultraviolet and notice that the current physical models
Uh, under the predicting the ultra-wide flux, and therefore, you should change the parameterization of the star formation.
a history. So this is, of course, a very simple example. I'd love to start with this simple example.
Because I want to highlight the fact that, like, a lot of what we call, like, research,
in the grander scheme, you can think about this as optimization, because, like, everything is just about, like, building a physical, like, models.
But not all physical models they're building are a simple optimization of problems.
Uh, especially in astronomy, for two reasons. One is because the data that we have is not that clean, right? So…
Sometimes, a model that gives you a better chi-square
can be less physically plausible, right? So there is some layer of…
possibility that you need to fold in. But more importantly, it's just that the action space, like, if you're thinking about, if I want to build a physical, like, models,
The action space are often just too vast, so it's more than just setting, you know, 10 parameters and optimizing over them.
And humans are particularly
capable of doing that. Like, if you think about what human is,
like, better than, you know, just a gradient descent out.
algorithm is human art particularly good at navigating a high-dimensional
action space and take shortcuts. So we use our intuition or experience
To decide what would be the next step to find the solution.
Right. So the question is, like, this type of intuition or experience, like, we know that most, like, humans learn from experience,
can also a language model learn from its own, uh,
or experience.
So, this is why, two years ago, we started to work on this thing called Mephisto. Uh, you might ask why we call that, like, Mephisto, because according to the German folk law,
Le Brafisto is a demon that trades soul for knowledge. This is exactly what we do. We trade, like, our soul to our AI, like, overlord, uh, for knowledge.
Right? So, the Mephistor is, like, uh, like, at this time, you know, like, most people have built some agents, so the idea of agent is just that you
break down your question into small pieces.
And like the chatbot to do
each of the pieces, right? So in the case of the James Webb things, you need to have, like, a few,
agents. So you have the first, like, agent,
That is the proposing, like, agent, meaning that given the current fit, given the current physical models,
What are some of the assumptions that I can change? So this, like, agent is, uh, connecting to all the paper and the database that we have, like, curated.
To decide what would be the next step to take.
The other agent turned out to be, like, actually the most work.
that we have done 2 years ago, is to execute the action.
Right? So, the challenges two years ago, I think this gap is getting smaller, but two years ago,
Most of the codebase in astronomy are not interfacing with
the agent, meaning that, you know, your chat GPT might not know how to run
your specialized code. So a lot of the things that we do, like the back then, is to refactor all this code into what we
they now call the MCP. So, the model context protocol,
to really make sure that the tool is what we call agent ready, meaning that, you know, the task, the pipeline is easy, but to be able to make sure that your domain that's specific to
can be used by the agent, take some work to get there.
But more importantly, I think why it's important, uh, is that once you have made a action, then you have an agent that can, uh, reflect on
why this works, or why this do not work, why something is better? Is this the action leads to a better outcome or not?
And then you have another, like, agent that will distill the knowledge. Interestingly, uh, in the modern data lingo, like, if you think about cloud code,
This is what we call the agent skills, alright? So, agent skills become quite a common term. Like, if you play with the open claw, that will be what you call the memory, right? So, like, all these things,
has becoming more and more, like, popular in the last 6 months.
But what we built back then is quite a proxy of what we call the agent scale and the memory on, uh, in, uh, in OpenClaw.
So, if people are not, like, familiar with
language models and the lingo, you could just think about this as a walker in the action space. So,
You are doing some tree search, just like human, but you are letting the agent to learn from the tree search, and also, uh, update the tree search.
over time.
Uh, in practice, this also works. So, back then, we are comparing with GPT-4-0.
So this is looking at some of the James Webb data and the chi-square of the feet.
on average. And, uh, looking at the chi-square as a function of the iteration steps. So you can think about, like, Mephisto is like playing… it's like the alpha 0. So essentially, you are doing, like, the reinforcement learning through the memory,
But just without changing the weights in the neural network.
So, what has happened is that when, like, Mephisto played more and more games in the, like, in this case, the games is…
to fit the spectrum, change the physical
models to be able to fit the spectrum, then Mephisto get, like, better and better over time, because it has collected a lot of experience and memory that can help it to do a better job next time.
So what would be the type of, like, knowledge that, like, the memory, or in the modern lingo, the agent skills?
That, uh, the AI has learned. So this is one of them. So, you know, if the fit is overestimated in the UV and optical, what you should change is the properties of the dust. This is also what a experienced astronomer would learn over time.
Right, so this is sort of, like, rule of thumbs.
that, uh, that you will learn, like, over time.
And this type of, like, memory, short-term and long-terms, will help the agent to do a better job. So why do we know that? For example, we also…
comparing with some of the data set. So, in astronomy, we have this deep dataset called Cosmos 2020,
And the classical way of, like, analyzing this galaxy spectrum is by building a grid of models and find the best fit.
Right. So, what we try to compare here is to say we give only 1% of the budgets for Mephisto, meaning that we have built a grid, but we say, so let's say the grid is
100… I think in this case, it's 100, uh, millions, uh, grid points.
And then we tell, like, Mephisto, like, beforehand, is that you can only try a million models, meaning that 1%.
Do your best, like, based on your memory and your knowledge.
a random walk in the hypothesis space and find the solution. But just using 1% of the computation.
And the y-axis here is showing you the comparison of the chi-square,
Uh, you can see that, you know, in many cases, like, Mephisto is the performing, uh, equa- uh,
equally, like, if not, like, better, despite using 1% of the computation, meaning that it has learned
how to update the models based on the knowledge that it has gained.
Right? So, uh, this is, again, old works. I'd love to start with this, that this is 2 years ago, but Discord has been made public, the agent with all the agent skills.
has been made, like, public, and people have started to use that. Uh, so this is the word using, uh,
of Bephisto to analyze a ultra-diffused galaxy.
This is, like, another work that is using, like, Mephisto to analyze the quasar spectrum.
to find a solution for this.
bought, uh, sources.
So, we also highlight that, you know, if this thing is not something that you have done before, uh, recently I chaired the NASA AI and ML Science and Technology Interest Group,
One of the key things is to build AI literacy by curating a set of lecture. I, myself, I give the first few lectures exactly on this topic, so how do you build using the modern, large language models and build
a, uh, a agent take a system,
Um, you know, building, like, MCP for using tools and so forth.
So, if you're interested, I will, like, encourage you to go through the lecture. Nowadays, like, the barrier is quite low, uh, because things are made much easier, so once you
follow the field lecture here, you should be able to do much of what I just described.
Can I… can I ask a question, Yuan Singh?
But… yep.
Yep.
Um, so the, uh, going back to the, uh, the knowledge that it learns, is that… first of all, that's knowledge that, um,
it's, like, interpretable, it's in natural language?
Yeah, it's in natural languages, so it's very similar to the Asian skill, because the, uh, language models will also output
Uh, the languages. So what is important is that, you know, I didn't mention this, the fourth agent here, it will distill the knowledge,
But we also have the loops that will verify the knowledge, meaning that you can come up with all the proposal,
But then we will take, uh, another, like, validation set.
To see if such a knowledge, like, actually works in most of the time. So, like, only if those, like, knowledge is useful, then we append to, uh, other memory.
Yeah.
I see. And then, I guess I… it went by a little fast, so this loop that produces the knowledge…
Yeah.
or the memory, the skills,
Yeah.
Yeah, it's in a loop.
Is that sort of, like, should we think of that like a training loop? And then now you've sort of got the agentic system that's trained, and you can…
True.
Yeah, I think it's more like a memory loops, because the weight has not been updated, so it just really like a pending longer and longer context, and also just trimming…
the context, like, over time.
Uh, the closest analogic, if people have been playing with
with open claw, it's very similar to open claw.
So you have a memory base, and you keep
Just like human, you recall… you keep writing down what works and what do not work, but you also keep, like, a trimming.
from your memory, what actually led to not worth.
for Compactify. Yep.
So I guess… so when you release Mephisto, it's with some knowledge that's been already accumulated, but then these papers you…
Yeah.
You mentioned that are starting to use it. Do they, as they use it, do they generate…
more skills for themselves, or…?
Yeah, I don't think they generate more skill, so I think because, like, for a single, like, sources, it's hard to come up with new skills.
So I think, like, lots of things just, like, are using the skills that we already, uh,
are provided.
I see. And then you, you provided those… you generated those skills by having it do what again?
By training?
Uh, we are training on Cosmos 2020 and Jade.
Oh, Cosmos 2020, okay, I see.
Yeah. Yeah.
I see. So you had some, basically some…
Yeah.
test bed, task, set of tasks for it to basically train itself.
Yeah.
to… and then with that, you release it, and you hope it's useful for other people.
Yep, exactly.
Okay, okay. Got it. Thanks.
Right. So, of course, you know, like, these type of things is… like, this is, like, back in 2024 and 2025,
in the AI time is a hubber time. Uh, things are still quite limited, so, you know, the type things that the agent can do are limited, but we have, like, started to explore some of
Uh, the things that the astronomer needs to do all the time, but also require some human intervention. So I will give you two more examples.
So, one is equivalent with measurements. So, for people who work on stellar spectrum or stellar astrophysics, you know that this is
the bread and butter scale that any grad student and postdoc need to learn. So, basically, we collect some spectrum,
And then you want to calculate the area under the curve.
And this allowed to be something that seemingly easy, but people have decided that the human in the loop is important, because there are many edge cases,
They cannot just write… so there's basically no one, uh, robust
algorithm that can fit all. So this is why most of the students, grad student, postdoc,
in that field, are trained to do this type of tasks. So why the human in the loop is important for this type of things?
So, for example, you can run a… write a algorithm, but sometimes it will miss some of the blended line. A blended line means that
For one line, you have two features, right? So this is why the human need to decide if this is really another lie, or just noise.
Uh, the human eye are still the best to do this thing.
And more practically also, you know, because we need to, uh, normalize the spectrum and find the continuum.
It's also, like, something that, uh, historically would require a loss of the human intervention.
This is another case where the RK code that we have in astronomy are not agent-ready, so we use what we call, uh, IRAF.
So it's a very old code, and interface that, you know, all the astronomers love and hate.
Uh, but in order to make it to be Asian ready, we essentially rewrite.
the version of, like, IRAF into a version that the AI can work with.
But with that, we also show that the AI can do these tasks all by itself, meaning that if you decide
when to… how to adjust the continuum, like, to decide when to…
get, like, another modes in the feeds. Uh, this, of course, has important implications, I know that quite well, because
Not too long ago, when we worked on a project like this, where a few hundred spectrum, I have to
a higher… a very experienced postdoc. By that time, I…
he make more money than me? It's because he's a very experienced poster. Even that, you know, like, he needs to work for 6 months.
Uh, to painstakingly doing this at a human intervention. But with this code, we call agent equivalent with our agent, uh, we redo the entire thing and find that to be, uh,
are consistent with the human measurement, but with only, like, $100 with a few hours.
Right, so this is the thing that we are talking about. Even though these are not groundbreakingly, like, difficult tasks,
But the automation is real, and a lot of the things that we are doing in the past
that consume, like, a lot of time can already be automated.
Uh, the other example that I will show is, uh, the citizen science, right? So, in astronomy, we love to…
ask the public to help us to do some classification, because we still believe that
no algorithm can catch all of the outlier, and the human intuition and the human
visual, like, the ability are still the best, right? So, in OSU, we run the assassin assuray, which is a decade-long
time domain as a way. We even have an assassin zoo, which is asking the public to classify the light curve.
What we have done also is showing that, you know, with proper, like, tool use,
We redo the Assassin Zoo.
With only a few hours and few hundred dollars, and we are…
able to find some outlier from the light curve that were missed by all the, uh, the people in the Asa Shin Tsu.
So this type of, like, automation is also, uh…
are happening.
So, I know that you might be thinking,
Um, maybe this is the future, right? There's no grad student, like, there's no postdoc,
Uh, not so fast, I think, like, you know, I…
So the second part, I want to give you a more optimistic view of what can be done.
But I think that there's also be good to be more clear-eyed about what is the limitation. So I think, like, that is the plot twist.
Um, it suffices to say that, you know,
I think AI are changing really quickly, so each time I have to give this talk, I need to…
temper this third part of the talks, because I feel like AI keep exceeding, uh, my imagination. But even now, I think it's still fair to say that AI are still struggling in many tasks that are easy for humans.
So I'll give you two. One is understanding the scientific charts, right? So this is something that even a freshman student can do quite well if you give them an archived paper. Like, at least you should be able to read the plots
quite accurately. Uh, this is a somewhat old benchmark by Princeton, but back then, in 2024,
Uh, is, uh, the AI are not doing that well. So the gap is closing now, but I don't think it reached the human, uh,
a level yet.
There's also an example that I think is more striking. One is the ARC AGI2 benchmark.
So this is a task that is very easy for human, but turned out to be quite difficult for AI.
So the background goes as follows. So it's kind of the games that you love to play. Not you, I love to play. As a kid.
probably you two, uh, is the game goes as, you know, if top…
Right is corresponding to top left, what would be the bottom right?
Right? So if you just screen for maybe, like, 2 seconds, then you realize that
Uh, in order to connect the top left to top right, what we are doing is to color code the different blocks with the number of holes in the block, right? So if I have one hole, I will…
the color code that we've green. If I have no holes, I will color code that with a yellow, and so forth.
And therefore, for the bottom panel, uh, you should do the same.
So, humans have no problem, right? So this is a task that you do not need a PhD to do very well.
But 6 months later, when Sam, uh, Altman declared that the AGI is coming,
Uh, the GPT-5 is only, like, getting 10% correct, right? So…
But to understand why this is the case, uh, the best way to understand is this is what we call the Model Vax paradox. So, even back in the 1990s, people have noticed that there are things that are easy for human,
thought out to be hard for computer, and vice, uh…
Aversa. And to understand why,
more of X parallel depths happens. I think the best way I love to explain to people is AI are essentially
trying to reverse, uh, engineer the evolution, right? So, things that are happening
The last, for mammals, uh, will be the easiest to be, uh, be, uh, be, uh, uh, uh, uh,
are imitated. So this is why a lot of things that we, uh, we process in the frontal lobes, right? Like, like calculation, like deriving, like, equations.
Those things are actually…
quite mechanical, and it's, uh, easier to be, uh, be imitated.
But a lot of tasks that we tend to not
price? Uh, turned out to be quite hard for AI, right? So these are the things that…
Even the earliest, uh…
a memo has, but, uh, it's, uh, uh, but it's just something that we don't pay attention.
Right, so, like, spatial reasoning is one, like, common sense, uh, visual… the reasoning. So those happen much…
earlier in the evolution, but turned out to be more difficult for AI.
Right? So, and this is also, of course, like, reflected in all of the benchmarks, including the one that we mentioned earlier. So, in the astronomy Olympiad that we work on, yes, you know, the AI could have won first place,
But if you look at what are the questions that it loses point,
Quite often, you're just, like, coming down to some of these things that is very, uh, easy for students, right? Like, understanding the plots.
Uh, it's still quite difficult for AI. So this is why I think I want to highlight this just to mention that
AI systems are getting quite strong.
But there are still ecosystems that we need to build for AI, and…
For astronomy or particle physics,
Uh, big companies simply do not have that much an appetite to make sure that all of your tools
are ready to be used, uh, by agent. So there are lots of, like, groundwork learning to be done.
So I will show you, like, some of the things that…
like my group is doing, uh, just a quick flash. So one is, uh, uh, improving the understanding of charts, right? So, so this is, like, something that we work…
with people at ANU, my, uh, my previous employer, so I still have some students there working on
what we call non-natural, uh, image, like, understanding, because AI are getting quite good at, uh, natural,
images, but non-natural, like, images like charts are still quite difficult.
For AI. Of course, this gap is…
rapidly changing, like, if people have played with Nano Banana, you know that, you know, uh, like, Google seems to be
knowing what they are doing, and they are doing quite well in that domain.
The other thing that we are doing…
Yep.
Yeah.
Sorry. Is it possible to ask questions? On your previous slides?
Well, when you showed the, um, the statistics of what the, um,
Yeah.
the AI is getting wrong.
Yeah.
Right. Is that a language
parsing problem, like, it's not…
parsing correctly the language of the question, or is it, like, it cannot synthesize?
I think… yeah.
Or find the answer through its patterns? Like, should it be able to…
solve it? Like, do we know what's the reason why it's not gathered?
Yeah, it's a ladder, because we are using the multi, uh…
a model, so we are not describing the plot, we are, like, giving the plot.
for the AI. Uh, you might also see on Twitter that, you know, like, sometimes, like, GBD-5, like, cannot solve a high school trigonometry, like, problems, because
It does not understand what is fine X, so if you draw a triangle, you know,
Like, it's still not bad, but there are… these are very simple tasks that you simply do not have the ability to understand chart, uh, not… it does not have the ability, it's just not, like, performing at
the level of, uh, a human, uh, can do. I think this is still real, right? If you…
even use, like, nano the banana to edit your plot, quite often, you know that it makes some mistake.
Uh, it's still a task that is quite that difficult.
Uh, to answer your question?
Thank you.
Okay, okay. Another thing that I think is important to notice is, you know, because the database that we have, you would not want the AI to just…
give you the answer, right? You want to have some, like, recommender, like, system based on the knowledge that you have. And this is something that, like, my group has been working as well, meaning that we are using the literature in astronomy and curate a knowledge graph, so this is the connection of all the concepts.
But more… more recently, what we have been doing is to take the entire corpus of astronomy,
distilled all the objects and the concepts, and tried to make a recommended system. You can think about this like, like Google Search, right?
Just like, even with AI, you would need a Google search to be able to do a robust
recommended that system. And for a very niche domain, like astronomy or particle physics, it's particularly important that we have to do that, because, like, no one will do that.
Uh, for us. So a lot of the work that we are doing is building the groundwork for the knowledge graph, and also improving
some of the ability of the AI system.
Right. So…
The other thing that I would mention, but I don't think is… I… I will start by saying that I don't think it's no longer a very interesting route of research. It's training specialized models.
Right, so… two years ago, we started to work with the national labs and say, how well we can fine-tune models
that can perform in quite well in a certain task. In this case, we are looking at
the Astronomy Q&A, right? So we curate some benchmarks in Q&A, and we're asking,
Like, can… like, can we just fine-tune models based on all the astronomy, like, corpus, and try to improve the performance?
So, because of that, we need to build a benchmark, uh, and we have been tracing the performance of all the AI models, so this is the score, the accuracy.
as a function of cost. By the way, the human accuracy is 66%, right? So if I were to do the test,
I will fail the task because I'm not that good at recalling some of the facts.
Uh, but if you find your models, uh, like, you can do quite well. So I think what we have learned is that, yes, if you have a single task, like astronomy Q&A, you can fine-tune the models with, like, 10,000
GPU, uh, uh…
hours, then you will be able to, for this singular task,
a 70 billion models can, uh, deliver a performance like HGPT-5.
But it's only for this task. So the reason that I think, like, this is not a very fruitful way is because if you want to use AI as a… as an, uh…
are Asian. You want them to have more holistic ability, right? So it's just like…
someone can do.
Well, in a who wants to be a…
billionaire might not be a good, like, researcher. But yes, if you fine-tune a human to just train on who wants to be a
a millionaire or Japanese, uh, like, you can do quite well.
So, sorry, can I just ask about this?
Yep.
So, the cost you're counting is the cost to train the entire GPT-5?
No, so the cost here for the other, like, models is the, uh, is the inference cost.
Oh, inference costs, okay.
So for the open source model, we are asking if I were to pay a company to load
Yeah.
my, uh, Hugging Face models, how much would that cost for the inference?
the training cost of projects.
inference, I see. So you can make it a lot cheaper for the inference. Okay, got it.
Yeah, yeah.
But then for the training,
Yeah, the training is horrendous. I think we… for the 70 billion models, we train for about
3 times 10 to the power 5 GPU hours.
Yeah, I think it's less than a billion.
But quarter of a million.
Yeah. Yeah.
So, like, what's the… I guess, what's the cost of that versus… I mean, and then the GPT-5, they already… well, I don't know, yeah, okay. Okay, I understand what the comparison is, alright.
Yeah, so it's an academic, like, exercise, which is why I say, uh,
Identify… so this is something that we started, but we… we wrapped up, but I…
Uh, I think the conclusion is interesting. For a small academic group, you can
like, fine-tune a 70 billion models.
But I just don't think you want to do it.
Uh, the gain is small.
Alright. Uh, but nonetheless, because we are curating our own, like, benchmark, we are able to track
Since the beginning of time, figuring out what is the cost efficiency, meaning that, you know, if you think about is the slopes here.
Right, so it's the score, like, divided by the cost.
Right? So, uh… so this is… if you just, like, ignore the astro station, just looking at all the open source and proprietary models,
The cost efficiency in terms of answering astronomy-like questions?
has improved by 6 orders of magnitude since 2023.
I'm not sure that people have, like, noticed this, right? So even AI stopped improving in terms of the ability.
Just the cost efficiency is crazy, right? So, I have not seen, like, anything in my life that has improved by a
by a million phone in costs of…
efficiency. So even though, like, astronomy can do
simple tasks, and even the few things that I just mentioned, like measuring the equip the equivalent width.
Like, we just take no cost, like, in a few years, just because, you know, the language models are improving, and the costs are dropping.
Right. Uh, one…
Uh, side note, not only the coin is dropping, the ability is unlocking on a daily basis, right? So this ARC AGI benchmark that I mentioned,
that GPT-5 did not do well in September. Last month, late last month, with GPT-5.4 and the Gemini 3.1, this actually reached 85%.
So it suffices to say that even for things that we think are very human,
AI are closing the gap very, very, uh, drastically.
Right. Uh, so this is why…
I think that this is a thing that I think the community has to grapple with. I think there are things that, you know, you can be worried, but there are also things that, like, we…
We as a community has to think about.
Uh, the obvious thing is that, you know, how do we train a student? Or what is the… like, how do we even encourage
supervisor to take on, like, students, right? So, as we all know,
a lot of the things that we do is much more efficient with just Cloud Code.
But it does not mean that the apprenticeship is not important, right? It's just that these two
objective now start to, like, bifurcate. If you just want to optimize your scientific, like, output as a senior, like, researcher,
that is a very, like, dangerous path, because, like, you might come to…
the conclusion that the trainings a student is not worth it. I see a question.
Yep, uh, yep. So, so, so, this is why we, uh…
are hosting this workshop. I thought it would be in person at Ohio State, but we will change that to a remote.
Uh, to discuss, like, about this.
Uh, what is the future of the scholarship? I think that is something that we need to think about, which leads to my last topic in the last few minutes. Something that I got interested for no reason, and is kind of like,
a bit off-topic for any, like, physics and astronomy talk, which is the epistemic, like, implication of AI.
Uh, because we are at the level that, you know, AI are automating so much of the science,
Even as a scientist, you start to need to think about what is the epistemic, like, implication of science.
This something has caught my attention, so this is why last, uh, late last year, we held, uh, conferences in, in, uh, at OSU. We bring together the philosopher of science,
the linguistic and also the scientists to talk about what is the implication.
I wish the finding has, uh, culminating into this piece.
That actually just came out, like, yesterday. I do put a archive, like, version much earlier, uh, but the nature, like, astronomy version is just…
came out, so I worked with the philosopher of science, uh, to think about this question. So I will, like, summarize what we find, uh, but
This is so important that, you know, I started to direct a new, uh,
initiative at OSU that are focusing on, like, agent and the epistemic
the implication of agent. So, what we have found, right? So, I think…
I don't think I am at hawkish as, you know, like Sam Altman or Dario…
Amode, because…
Uh, I still feel like humans cannot be, uh, eliminated from science.
I'll put it this way, I think, like, by definition, by the epistemic
the definition? Science, as we define it, will cease to exist.
If the human is not, uh, participating.
Right, so why that is the case? I think that the key here is that
Uh, AI have no interest in understanding the universe. We want to understand
the universe. So this is why we construct story and context.
to understand the universe. I think that some of the things that the engineer miss is that they're doing science or physics or astronomy are not making prediction.
The making the prediction is one of the things that we do,
But ultimately, what we try to do is to be able to
find some characteristic that we can construct some, uh, some narrative.
And try to make the world intelligible to us.
not intelligible to the AI. I will encourage you to read that piece if you are interested, because I spent a lot of time to think about that.
that thing. I don't think I will do, like, justice to the piece, but I want to give you some unders… so why…
I think human has to participate. Because science is about, like, narrative. We are telling a story to ourselves.
And because we are telling a story to ourselves,
Then, the narrative becomes
are very important. No words can contain all the information that we
one, two. So, the words…
is not a sufficient carrier of all the contacts that we…
we understand, right? So, give you an example, uh, we talked with a linguist, right? So the linguist will always point to this example from
Uh, the sticks with a story from Hermanue, so Herb, uh, Hermingway is a
Great master in the writing six-word story. One of the famous ones goes as follows, uh, for sales,
baby shoes, and never worn.
Right, so this is a six-word story, but I think for most of the audience,
the emotional weight is more than the six words. So this is my point, because a lot of the things that we understand, like the science is the same,
Now, a lot of the scientific understanding that we have as a community is circulating
as a meme in the community, right? But this is something that is not encoded.
in the text.
And all the science really comes down to, like, making the world intelligible to us.
AI can help us to make the world more intelligible, but ultimately, we want to tell the story such that
the other human can also retell the story, right? So this is why
Uh, I am not that…
pessimistic, I don't think, you know, it's not like, oh, we make a switch, and, like, AI will do all, like, all the signs.
And the human will be…
irrelevant. Exactly because I think that some of the people in the industry or engineers, they simply miss the fact of
What is the philosophy of science in the first place? So I find that quite, uh, illuminating to talk to the philosophers, to understand, like, where we stand.
But at the same time, I'm also not, you know, turning a blind eye. I really find that this is a talk that is particularly… not particularly hard to give.
Because, like, every few months, what I say is obsolete. Like, if not, you know, like, I think, like, half of the talks here is already, like, obsolete.
Things are moving really fast, and we are just living in a very interesting time. So I will stop here with this code that I wrote in the piece, and take any question that you might have.
Alright, we have plenty of time for questions, so please, uh, don't be shy.
Yeah.
Are people talking in the seminar room? Because we can't quite hear…
Okay, okay.
No, we're just trying to see the chat. We're working on it.
Come on.
Well, while you're waiting, I have a bunch of questions, for sure. So, yeah, you mentioned that things are moving very fast, and that, you know, it's tough to give the talk.
Yeah.
Because things become obsolete quickly. So, going back to the beginning of your talk in the agentic systems you've developed,
Yeah.
Um, compared to, say, Cloud Code, which a lot of us are using these days,
Yeah.
Uh, with great success, I would say. Like, how would you say… what would you say is the…
space, uh, where we could hope to improve on
on the Frontier Lab products, versus just…
Very cool.
Yum.
Yeah.
Use them. So the kind of research into developing these agentic systems that you did,
Yep.
like… like, does it have any value?
Anymore.
Yeah. Yeah, yeah, yeah, this is the thing that I keep…
Okay.
debating, because ultimately,
what the system do not…
naturally do is to be able to connect some of the code that we wrote.
Just because… but it's also not that hard if you just say, you know, these are the all…
all the codebase and turn that into some MCP.
like, call, click, and do that. Just that…
Clot has my band back. So this is the part of…
project that I keep at debating, because…
I have a student, I was, like, thinking to…
you know, turn most of the well-used codebase into some agentic framework.
I did a pretty good.
Boss, I feel like, you know, yes, you can do that, but also, what is the value? Because all people, like, wherever you need to build a system, like, you might just ask Claude Code to…
to refactor that. So…
Uh, unclear. I think that for the, sort of, like, baking…
agent LED2.
Unless the code is, uh, very, uh, uh…
huge, right? Huge piece of code.
then yes, I think you want to do that yourself. But if, like, for simple code,
then they will not be worth it.
Mm-hmm, mm-hmm.
Uh, because, like, you're thinking about some of this,
Votron code. I don't think, uh, is the one prompt for cloud code to make that into something that, like, other can use. So you need some people to do that.
All right, um, but I didn't find that more interesting to think about some of the… some of the knowledge graph and recommended system. I think, like, that is something that…
the big company is unlikely to do. I think that has a more enormous implication to the accuracy.
Of things, uh, that we need.
Yes, thank you.
I see a question on Zoom. Yannis, do you want to ask it yourself? Or, uh, go ahead.
Yes, uh, first of all, thank you for a great presentation and a very… on a very thought-provoking and cutting-edge subject.
Now, my question is, why do humans think of AI as non-human,
After all, it's been programmed by humans,
It's trained on human-produced content.
And it's used by humans for human purposes. So why do you think we think of AI as something…
Oh, uh-oh.
outside of the human realm.
Oh, no, I don't think they are non-human, but I don't think they are human, so I think, like, the…
The epistemic implication? So at first, we were just thinking that, you know, like,
maybe an angle to understand the impact of AI is, like, simulation, right? Like, 20 years ago, like, simulation happens,
And no one, I believe that simulation is science until people have to accept that simulation is part of
the knowledge that we have. But what is really different from simulation is that
the AI talks back. So this is, like, uh, I thought the biologists, I think that it's very fun, I think the remark is that, uh, AI is the first
species, non-human, that can talk to a human.
Which is true. You think about, you know, all the species on Earth, none of them can communicate with us.
AI can, but it's… it's not human in the classical sense.
When you have such a system that is expanding your perception,
How do you really redefine?
What is the knowledge? Become more involved than just treating this as no…
simulation, right? So I think, like, this is the thing that we are…
like we are grappling with.
Of course, I think the impact is also likely to be, uh, larger than simulation, but I think it's…
kind of a definition, but you are right that we should treat them as
We shouldn't treat them as non…
non-object. I think that is a…
Because I think the character… the characteristic is that
it worked with us. It does not give us answer. It's a system that communicates with us.
And how do you include such a thing in…
the endeavor of…
a scientific inquiry.
is unprecedented, right? We never have
like, any, like, kangaroos come in and say, oh, no, kangaroos,
like, brain is part of the scientific endeavor.
Yeah, so I think this is what we are…
like, grappling with.
Any other questions from the room?
I think you'd kind of touched on the question that I've been thinking about in terms of the agentic stuff in our discussions here.
We were kind of feeling that a lot of the agent systems we'd been seeing in.
you know particle. physics discussions have been kind of a ways to get around the context.
Right.
Limits. Um… Is that… do you think that that's a useful way to think about sort of what people are doing with with agent systems? I don't mean that sort of in a dismissive way, right? Like the context window is a real can be a real issue.
Yeah, I'm not sure there's, like, a conclusion to that, right? So there's always, like, the retrieval system, and also just, like, long context
should be able to retain the everything.
Um,
you know, just as a physicsist training, I always love to…
hope that we can do some algorithm, meaning that a retrieval, a system that is effective.
not just relying on Infite90 long, like, contacts.
It's just that there does not seem to be…
Uh… up.
are elegant, but to my taste.
But it could also be true that if the context is long enough and you can retain, like, everything, so…
Who cares? Yeah, I'm not sure I have a good answer to that.
Thanks.
Saurabh has a question on Zoom. Um, I can read it out loud, or Saurabh, do you want to…
Janice, I guess.
Um, yeah, you speak…
Ask it.
But better than me, so I will let you read out the question, and then I will answer.
Oh.
Anos… well, yeah, so I guess Saurabh is asking… okay, he can't talk or something, so he's, uh, asking, yeah, what do we… how do we advise, uh, junior and senior…
researchers, uh, what skills should they be learning? What would be your advice?
Yeah.
I always tell my student the same thing. I think, like, um, I think there are two things. I think, like,
Not only we need to tell the junior as…
junior students, because I don't feel like they are the problem. I think, like, the professors are the problems, because
the professor, I decided not to hire a grad student because I caught cold, right? I think that is a bigger problem, but…
For students,
I don't think that anything changed, I…
The only thing that I always, even before AI, I always emphasize on a good taste.
Right? Your… this is, like, the narrative order thing, right? I think, like, what makes a good student a good student really come from
the acquired taste of what is a good prop… problems.
Uh, I think I find the ability to find good, good problems,
become even more important now. It… it…
In the past, it always has to be important, just somehow, I think the…
the whole industry of science has
makes students find
of putting more emphasis on problem solving.
But not problem-like finding.
I find that the junior student, I think, no. I think this is the danger, because…
If… and how, like, how do you learn from, like,
problem finding, quite often, you learn that from your supervisor. The supervisor has a lot more experience, and this is why the craftsmanship
get… get passed on.
But if your supervisor are just using Claude code to do everything, then you don't have the opportunity to learn
that pop, uh, that is very, uh, of…
are valuable. So, you know, I always say, you know, just…
Even more important, you should go to the clock game, or you should go to, like, coffee, you should go to, like, talk to people.
But if a student can use Claude Code, then maybe they can problem find much faster. You know, they could…
I'm not the optimistic, because…
probably a lot of different problems and see which ones lead somewhere. Much, much more than they could have in the past.
I'm not sure what the people experience, like, at least my experience is that the…
ability to use Cloud Code is…
almost super linear…
are dependent on your… your maturity in the field. So if you know what question to ask,
what is a good answer, you get to answer quite quickly.
student, you can use Cloud Code, but I don't think it fundamentally… I think, like, like, there might even be loss in all the generation, and…
Yeah. That's interesting. Yeah, we're just starting to play around with it here, I think, but I don't know, I have a suspicion that may be true.
Yeah.
Yeah. So, yeah, yeah.
They've done studies, though, right? The effectiveness of these tools is… they've done big studies, right? Like, it's proportional here.
education level.
Yeah, I think it's like… but I think, like, that study also has been debunked. I think it's, like, N-square, so if your ability is N, then with AI, you are N squared.
So if you are one, then you stay 1.
If you are 10, then you… I think that's the problem.
Uh, and Janice has a problem, has a question?
Yeah, go ahead, Yanis, yeah.
Yeah. Yes, actually, my question touches on both the last question on, like, you know, problem finding versus problem solving, but also
more on the…
context parsing.
Now, in the Natural Sciences, I think the natural sciences are better suited
for the uses of AI that we think of, because we have
controlled experiments, we are very well-defined set of assumptions, right?
So, uh…
So I guess then it's easy to code and have AI help us
with the computational complexity, or even the linguistic complexity, which is
that goes into the problem finding. Oh, I haven't thought of that problem, or I haven't thought in that direction in AI.
can do the computationally intensive part.
of doing the parsing that we don't have the time or the focus, or the direction.
to think at.
So, that's understandable, but, uh, could we also, as physicists,
um… be… raise more
loudly, the problem of interpretability,
Yeah.
which AI suffers from in the non-natural-based
Golden Quote sciences.
And…
Yeah.
So, can you speak a little bit about that, which has to do with about context parsing, or assumptions, which are lurking
Yeah. Yeah.
very profusely in the non-natural setting versus the natural sciences, and could, could,
AI be using the natural sciences
Yeah.
to kind of, like, reverse engineer. You do an experiment, right? You do experiments, you think you have all your ducks in order,
Yeah.
And then you produce some results which you think are correct, but maybe you've missed an assumption. Could you use AI to go backwards and tell you, uh, ah, your assumption is not correct, that's not…
the right experiment. Along these directions, if you can elaborate.
Yeah, yeah, I think, you know, yeah, I think…
You're right, you think about the whole thing that we are doing in science, what the biggest impact is really in the middle part, right? So…
I think AI are weaker on the… both the beginning and the end. So what I meant by the beginning is the assumption.
Right, so what assumptions are making sense? Why do you want to ask?
There's such a question.
AI can do something, but…
I don't think it… I think, like, asking it to auto-correct for the assumption.
is too much for us, but the fact that you can do the middle part extremely quickly,
Meaning that the human can try, like, many things, like, very quickly.
Right, so I think, like, it's not changing the beginning, but it's helping, because, like, a lot of the projects that I would not even thinking about doing the calculation because, you know, why would I want to…
apply for an SF grant for a student, and then wait for the grant who comes in, and then do the calculation, but now we can do that right away. So we can change the assumption.
But also, the end part is weak, right? I think the end part is the story, like, telling. I think less so about, like, particle physics, but astronomy is a lot of the narrative, right? We…
describe Galaxy evolution based on these five things that we think are important.
Not because these 5 things are the most predictive one, it's just that we believe, you know, in the whole context of that knowledge.
These are the characteristics of the system.
So, what I see now is that the middle part is
kind of, like, being obliterated by AI.
all the calculation, everything.
But what is really, like, lacking that no one talked about is really the top end.
the bottom, right? And this is particularly important because
I think as a community, we need to…
be, you know,
not just giving up your data to the company, kind of.
I think… I think that's the danger, because
Um, I think the dangers of this AI for science hype is they are changing
the entire evaluation of the value of a scientist. So there…
turn something that we thought to be very important, meaning that, you know, we find a very good question,
We… I always find, though, it's very difficult to find a good question, and very hard to come up with a good
the narrative. And this is still something that we should really hold…
hold tight. But…
the AI for science paper will tell you that they can solve things much faster, and therefore, science is over.
I didn't think that is true, but they are changing how they see signs, and that is the danger, because, you know, they have
the power to change,
the narrative at the…
upper level, right? If the people who decide on funding, deciding that it's more important to solve 100
not important that question quickly, then, like, finding someone to…
find some interesting, like, question.
then that would be the problem.
But I also find that maybe AI at some point can also do that. I still haven't seen that. I also do not know how to…
algorithmically to structure that.
So, not sure I answered your question, but I feel like
Yeah, I don't think AI can auto-correct for the assumption, to some extent. They can, but…
Uh, that we… the beginning part has tool.
Currently, it's still mostly come from human.
Can I ask you a question? I continued in that kind of like a flow of thought.
Yeah. Yeah.
So… I think you have taught this more than I think anybody else here or like me. So, for example, your in your article that that you just like said that go to read. So.
Yes, yes, yes.
Yeah.
Now that we are getting to this like era that, for example, your article is in nature, and I have this joke, like, oh, if it's in nature, it's most likely wrong, because you go for high instead of so yeah, so again, I think there have been a story that if you go, like.
Yeah.
Yeah. Yeah.
Yeah.
and read the nature paper like most of them are like just wrong. It's just because the incentives to publish are more in the hype than being right, right? And so, yeah. So like we are… Starting in an age that do like to do things are becoming cheaper and cheaper, meaning that if you want to analyze some data, understand some code, or even do your own code, now this… the marginal price of this is going to zero pretty much right now. You can do it.
Yeah.
Yeah. Yeah.
So this is going to start to rise. a lot of, I guess scientists are going to start to publish solo, and then you're going to start to have a lot of noise right? Then the noise is gonna like a lot of things.
Yeah.
Papers coming out and then it's gonna be hard to disentangle the noise from the signal.
Yeah.
But in a sense, I think it's good, because now also you are kind of like this benchmark that become the goal has now, like, they don't have value anymore. So yeah, so, like, I don't know, like, how are we gonna, like, yeah, like, value, like, what is good science or not, or it's gonna be a taste of each individual science, I don't know.
So I think the institutional, like, level is really the difficult part. I'm not that worried about AI
doing the calculation for me, but you are right that I am more worried about
the… it will change, like, the nature of…
of what we do. And I don't think most of the institutions are ready to respond to that.
Right? So…
I'm not sure even, like, you know, how do you even… as you say,
If I decide to write
the level of a undergraduate, like, paper, that I wrote.
I can probably write once.
In two days, right? So, each year I can write, like,
180 first-order paper.
that are feeling like isochron to something, right? And that is not what we want, but if you really want to just say the metric, then… I think, you know, that is…
where I think a conversation like this is important, like, I think the community need to know
What is the valuable and what is not valuable?
Now, whether or not how to solve that, I do not know, just because we will likely to be swarmed by neighbors.
the… I should say, the symbol, I think, is that, you know, when that happened, and that will happen, is that, like, people have to come up with something, because
What else can you do? You cannot read all the paper, and all the papers are, like, basically…
That will finally get rid of these stupid metrics, like citation counts and age indices. Finally,
Yeah, yeah, it's kind of like you…
So, those metrics will die.
Yes, yes, exactly, that's what I went.
I think that's the best outcome, but I'm not sure, you know, I think to get to the outcome, you need to do some work. It's not like you tosh…
the whole thing, and then say, oh, you know, good thing will happen. I think, you know, at least, like, it's easy to say, wow, no, of course, we have to do something else, because what else?
Mm-hmm.
But I don't think that we'll naturally happens. I think it's easier in the scientific community, but
If you go outside the community, you know, there's…
It's just… people don't know what we are doing.
And if you look at, yeah,
is difficult, yeah.
Uh, so I find… so this is why I find I spend more time on the… the epistemic level, because I feel like
I don't always have good ideas, or the idea can wait, but I think the institutional, like, implication
Okay, not a week. This is happening, and we need to know what to do.
Thanks.
Okay, yeah, well, thank you very much, Jensen, for the very stimulating talk, and I think we can all give you another round of applause.
So, um…
Thank you.
Uh, yeah, we could… we can probably see about setting up some kind of virtual visit.
Yeah, yeah, I, yeah, uh…
Um… so yeah, let's be in touch about that.
Yeah, for sure.
Yeah