What’d Meta Say? And the Super Soiree
Disclaimer: Everything below is a mix of what I observed and heard during the event. The goal isn’t to pinpoint "who exactly said what," but to share (usually) an outsider's view and overall perspective on these industries. I’m not here to act as a definitive firsthand source—readers should do their own research. I hope this inspires you to attend events, explore new industries, and hear what leaders are presenting. These notes combine my observations with thoughts on how things could run smoother and how ideas connect (IMO). I’m not an expert, you know? Just hanging out in the room with them. Enjoy.
Ratings (Meta Event): Venue (4/5), Food (5/5, Speaker Content (1.5/5), Networking (2/5), Likeliness to Return (2/5)
〰️
Ratings (Meta Event): Venue (4/5), Food (5/5, Speaker Content (1.5/5), Networking (2/5), Likeliness to Return (2/5) 〰️
Photo Collage + Commentary (soon)
Notes from The Event:
Arrived in time for lunch
- A friend I invited was here, and we had a good time catching up + planning on a partnership together involving getting my programs (and his) into public schools. Will take a bit of work but we can make it happen. Then I had a meeting online - then joined the talks.
This is super high-level conversation going on here.
Final speech is all I could make.
Two kinds of techniques, the data perish (perish??) The GPU processes batches of data and the second time - oh prefetch - the data looks at something (this guy’s accent is hard to understand)
Prefetch data.
The combination of these mechanisms are high latencies.
Alright, so moving onto some results. This table shows the latencies before and after the integration . Before it was (his head is in face of hte number shshsh o you can’t see) but its 90% improvements so he said. From many minutes to just a few for AVG P90, P99 and MAX.
Alright, that’s mostly what I have to cover for a technical perspective. Now the closing thoughts
A new guy comes up on stage to talk about the “Key takeaways” and reads off the PowerPoint.
Modern AI workloads are “data hungry”
- Storage performance directly impacts the cost and speed of innovation of AI
AI workloads shift the tradeoff space enough to rethink the entire architecture from GPUs to Storage
Tiered caching, prefetching, and on-demand hydration enable geo-distributed training
The work is fast evolving. We continue to innovate in this field. This is what we’re doing over the next few months and years.
We’re scaling storage to network limits. When you go from GB 200 to 300, the nice double but CPU resources haven’t changed at all. In storage, we gotta work with that.
Now, one of hte key bits that we’ve talked about is their loading pad, as well as optimization of checkpointing. More frequent and more optimizations that we have to make to our staff. All we talked about today was training, and inference workloads are here which is the next challenge we are working on
Thank you everyone. It’s time for Q&A.
The audio is wild, robotic.
He reads questions off of his phone - you have to submit your questions virtually lol. Safe.
Largely things are about sequential rights and findings and such. It tends to be a mix of small range.. a different type of challenge. In addition, with inference overload, the GPUs are doing all sets of (idk) mental and KB cache, etc.
The resource budget in general tends to be much tighter. How can you achieve high performance with small range raves and different amounts of memory. Latency matters a lot.
In terms of QLC, QLC forms in a different spectrum in terms of costs. Many have approached them to have a high demand for — idk. This guy is so hard to understand. I’m looking in the audience and of 15 people I see, 12 aren’t paying attention. They’re on their phones or laptops. Lol.
Next, welcome back to the stage a woman from Meta. Then the hype music.
Okay - I’ve got hte pleasure of introducing and moderating hte fantastic panel with esteemed guests
All these strong men lift up comey chairs and put them on the stage, then a woman places water bottles on the chairs - so funny.
AI agents are doing work and critical infrastructure. They’re mitigating, acting. Not will they, but how we deploy them safely economically and safely when failure has immense consequences.
She keeps looking down and reading her script off a screen. Then the men put water bottles on the chairs
This panel includes people from huge companies - one is a company I remember seeing protestors show up and hate on at a big event last year.
Also, a guy just walked out of this event (maybe leaving early) in a shirt with a screaming cat running away from a UFO.
Okay the crew walks up on stage. 5 people walk up on stage. LOL. This room is like 15% audiences.
Then the woman says she’s forgotten to introduce one guy.
Thank you all for being here. I wish we had 5 hours for this, she says.
What is the context citing for agentic operation. T he vantage points for infarct infrastructure and agencies. Where will they hit the wall? The system typology can fit
- Dependency graph is noble.
When you’re operating at hte scale of any hyperscaler, any change will cascade through layers of abstraction that agents can fully map.
IT may be burning tokens, traversing everything it can do. So, we’re sorta at this funny middle ground in the evolution
The guy next to me is just looking at FIFA updates. Doesn’t understand this stuff I guess (ahhaha and btw its my friend I invited sitting next to me. lol)
How do we think this through with respect to when we pull in the humans and let the agents cut their teeth bit. How do we think about this? Take it away.
One guy says - sure we’re using agents for networking. It’s where I have most amililarity. Many speakers have said it’s more about context than what goes into prompts.
Honestly, some of hte best documents we’ve had to start our journey are the troubleshooting guides to bring on our human engineers.
AI can look at database queries to try to figure out what is going on. To extract the semantics of what’s in those tables from the queries. To reformulate and put those queries back together and surface that information.
Carefully picking what we put into that context. Documentation is written for human engineers; we can quickly bootstrap our agents without a human riding along. Humans make mistakes, and AI makes sense, so we have to have ways to provide extra guidelines next time t deals with an agent.
She says to a guy: " That looks like it resonated with you - and the next guy talks. Yes, as usage has grown, people start and it could be so massive that the agents start making lots of mistakes. IT compacts the contexts and it starts doing. Random stuff.
Teams are encoding memories and skills specific to their projects so as agents traverse the repo, they can quickly ramp up to a moderately experienced engineer and pick it up quickly.
Even with that, we’re always trying to upscale and distill new higher-level abstractions
Lol I wanna go outside, get a drink, grab snacks for my daughter - then come back. Will do - brb
LOL a guy is sitting on the floor next to my charging phone. Thats the thing with these places, they so often don’t have enough chargers. So much that C-level whatever 6 figure people are sitting on floors. Just to charge their stuff.
We think about the role of not only management but hte guarantees we’re making to customers at scale.
Seems they’re running late. This is supposed to be over in 4 minutes - this panel. And it just started basically.
Many agents have issues which woudl take 7 day to fix in the past and now it takes 7 hours.
Let’s tease one thing out of there, if LLMs are all operating on nondeterministic problems, what point does this flip into knowledge we would encoagede in a deterministic sense. At what point does this become stale?
One guy is sleeping in the audience. I wanna count how many people are here. There are 14 chairs per row 19 rows… but then also like 3 sets of 14 chairs per row (3 sections)… so 3*9 = 27, times 14…. 378 chairs (omg I used to do this type of math all the time when I was dying of boredom as a kid in school. I’d count the tiles on the ceiling or walls, etc). Okay, so how many people d I count here in one section… 40… so lets imagine there are the same number of people in each section, there are 120 in the audience. So 120/378 = 31%. That’s more than I thought. I was thinking the room was 15% full but it’s double that! Hahah. Still, surprising for a FREE event with breakfast, lunch, and happy hour. And hosted by META - but boy oh boy this is high level.
I had a great time before this during hte break talking to my friend who I’ve gotten to know from attending so many events. He’s workign to launch an app and we see each other a few times a week these days. Our goal is to launch programs at schools together. He’s already got a program he can launch but he can add me to it and connect me with schools locally - and then across the country.
I’m happy to see that attending all of these events can and will turn into big partnerships.
Can you do the same with less? That’s better for everyone. Let me pick up this thread of efficiency and costs. Thinking about my experience with an autonomous vehicle, that first set of nerves you get think gin OMG with your engineering hat. There is an incredible influx of data and sensors every millisecond allowing us to have that reassurance of confidence and safety.
How much additional data do we need to have a full handle of whats going on.
But thats a cost decision, engineering, a total resource decision.
How ar we thinking of this across the space?
Thank goodness we have agents
The panel laughs - not the audience hahah
When training the latest models, think of coasts, the computing costs. After that, you’re looking at storage costs aight behind it. Networking costs. Finally observability. In the eternal scale, we’ve done when trying models: log it all. At the end of the day, anything you can get to help you debug a run failure ends up being worth it if you shorten the large-scale training and get it up and running.
Much of the past couple of years’ transformation has flipped the cost on its head. Used to save 2-5%, but now you’re saying, well, I need to roll several gigawatts, how can I harness this compute coming online? No more saving on the old systems.
A combination of hardware, sensor packet, and software… we are not fast, we take time. Because 99.9% of times usually a few hours a year of trucks crashing, that’s not okay. We learned a lot of methodologies in transportation companies that we are safe.
It’s a bad day if a product goes down. It’s not an existential threat.
All leaders have a responsibility of the business. Our comparison is simple. If agents dont exist, how long does it take to resolve it.
When your gates are not good enough, you can get more errors than in the past. That’s where now in the renewed interest, there are ways to build better gates. What has happened in two months, your errors were increasing. Your error rate goes up. Agents are being applied to make a defense system. This year we are showing up.
Okay - I zoned out cause a huge amount of events just opened up for registration for Tech week. So I’ve signed up for them and now this guy is still talking. They’re running late. So I just signed up for like 20 events. lol.
Most of hte code is being written by AI these days. Massive increase of productive activity 10-100x over time. It’s pretty amazing. Counter to popular narrative, we’re finding that each of these changes inside of software is safer. Not causing more incidents. The amount of incidents from changes in code or soft was relatively flat. Constant. But that started to change in November and October and the beginning of hte year as we introduced more AI. Every change become safer. Over 1 year a 50% preduction.
Time to deal with any incident, dealing with operational results… okay I may just get going. Cause this guy is running so late. And I need to go jump on the trian and be on time for this lemma see how far it is = the suairee I’m going to.
They wanna give rayban video glasses to all veterans in the USA for free and then that’ll go back to models. It is exciting and meanifnul to him
omg surveillance state times a million. geeze.
Now lets go back to the picture of Neil Armstrong not on the moon.
LOL, the fake moon landing. I wonder how many people in the audience agree with me, cause over 2/3rd of my LinkedIn audience believe it was fake (from a poll I gave)
Now we’re spending twice hte money as the Apollo mission. (on what?)
And it’ll take me 30 minutes to get where I’m going next.
Last year he ended his talk on how the overall AI timeline was. W’ere not at the middle or end, we’re at the beginning, where we will see massive changes in this tech and all the people we interact with. You’d think we’d have moved much further on that line, but that’s too optimistic. We’re not at our move meant. Like Sputnik.
He keeps stuttering a bit like he’s reading off this script. Lol. Keeps lookin down at his script its not authentic.
The estimated Capx is $1T. One followed by 12 zeros, almost impossible to think about. One more perspective and the context, that’s the combined network of one person *he puts up a picture of elon musk”
He laughed and says: Jokes aside, I couldn’t help myself.
eyeroll
That’s the amount of money we’re talking bout here.
Throughout the progress we’ve made, think of the distance of Sputnik and the ground versus Sputnik and the moon. Training, tolerance. We’ve done a lot of work but failures interrupt jobs. We’ve got to get much better at detection and mediation systems. All of that changes with every new GPU platform that comes out. We’ve cracked software and coding. Deployments, long tail migrations. All parts of our job that agents are yet to solve and we still have a long way to go.
Many of you have been involved in this. GPUs are holding us back. Far from helping us, the $1T will make this even worse. Not just GPU constraints to everything.
Okay I’m done taking notes. He’s about to finish up. Talks about hte moon missions. We have control of our destiny. … the people themselves.
Never mind. I just hate people reading scripts on stage. It’s so hard to listen and meet them where they’re at.
He loses his train of thought. And eventually his speech is over. lol.
Okay, a lady comes back on stage and she says thank you, great talk and I really love your abuility to weave in challenging topics from today’s event and bring it back home with the reason why we toil away
She has on a black shirt and blue jeans - like dark blue and black. Isn’t that a fashion crime (joking but serious) i feel ike that’s 101 in fashion/color matching. Don’t do it. Surprising move by someone being the Emcee. I wonder if she did it on purpose just to like, idk. Push/test the norm a bit or whatever.
Thanks to the crew that made this possible and the teams that trained us all to get to this point. This has been a fantastic day. Hope you leave with valuable connections and insights. Then come back and share them at a future scale. Recordings and the related blogs and resources will be available at the at scale and YouTube channel so be sure to check that out and share with your teams. Please come to the happy hour in the next room. Thanks for spending your day with us
Okay I’ll stay for 15 min, eat snacks, then head out. Off to the soiree.
SOIREE NOTES:
this was basically just a huge fundraiser. They gifted me a ticket last minute, so I joined, schmoozed. I complimented literally every person whose outfit I loved. It was fun to be enthusiastic.
Also, I had just this week finally confronted the lady from the “boats are boring” blog who saw me blackout my last time in Seattle a few years ago. She was super nice about it. So we said hi, finally, and even I asked her for a hug.
The soiree was alright- not amazing.
But when it was the DJ’s time to start the dance party, they started with “everybody dance now” - which somehow made me want to do the opposite and I head home. I was exhausted.
But they did raise over 150k from that night alone. Amazing. Its a great group, they gift clothes to women, have free trainings, workshops, and lectures… and have even gifted me a free laptop. I love them.
Until next time, I wish you the motivation and success to search for opportunities around your area. Search and explore: Who is out there giving talks? There are new things happening all of the time.
Find relatable or interesting topics you like and check them out! Maybe even something hosted at a cool venue, if there’s no other reason to go. Let’s see what you can learn and discover not too far from home. 😊
