Episode 537
537: Going ULTRA with RTX Spark and New Surface Devices
October 19th, 2026
57 mins 10 secs
About this Episode
Microsoft’s new Surface Laptop Ultra pairs an ARM chip with NVIDIA RTX graphics and unified memory, promising workstation-class power for creators and local AI—at a premium price. The conversation explores why CUDA support and Windows ML matter, how local and cloud models can work together, and how Microsoft’s new container-based isolation aims to keep autonomous coding agents safer. A look at ambitious hardware and the software ecosystem needed to make it useful.
https://blogs.windows.com/windowsexperience/2026/10/07/building-windows-for-hybrid-intelligence/
Follow Us
- Frank: Twitter, Blog, GitHub
- James: Twitter, Blog, GitHub
- Merge Conflict: Twitter, Facebook, Website, Chat on Discord
- Music : Amethyst Seer - Citrine by Adventureface
⭐⭐ Review Us ⭐⭐
Machine transcription available on http://mergeconflict.fm
Support Merge ConflictTranscript
Welcome back everyone to Merch Confiture Weekly Developer Podcast.
Talking about all things in the world of
software development, AI, and new hardware.
I'm one of your co-hosts who works at Microsoft James Montemagno,
and with me, the one and only who used to work
at Microsoft, Frank Riggas, and gone buddy.
Oh, it's been a while. Do they still call it Microsoft? Is that still the name?
Is that trademark registered? Yeah. Okay.
MSO, still the ticker.
Tick tick.
Hi, James. It's wonderful to be here on your show.
This is one of my favorite weeks where we are going to talk about hardware spoilers.
I know we like to ease into our topics,
Read Full Transcript Hide Full Transcript
like talk about random stuff for 15 minutes,
but I feel like we have a lot to cover this week because
Microsoft's in the hardware game, you know, like they put their toes in,
and then they step out of the water. They're like, "Who's that?"
It's a little cold, and then they get back in,
and then they're like, "Eh, I'll try it again."
This is one of their, "We're just going to cannonball into the water."
And I don't know what you want to talk about.
What I want to talk about is this big, cool laptop
they just released, and I just want to dive into that puppy.
James, have you heard about this laptop that a little MSFT company has released?
Well, a little company out of Redmond, Washington.
I have one sitting in front of me, so yes, yes, I know.
Okay. Well, aren't you rich because, man,
the starting price of this thing is just as big
as its beautiful screen.
Well, no, there's a big hardware event,
but also a software event and a local, like,
a model event too. It was kind of... It was
like a windows and hardware event, and to me,
it was also kind of also somewhat a developer
event, a little bit too, I want to say.
I mean, we're getting all the attention right now,
right? Because anytime you say AI, it's like,
"Okay, business people do something with it.
I'm really not sure what business people do.
I assume they just chat with it all day or
something like that. Maybe it fills out their
Excel sheets or something, but I think you and I
have the most experience with agents and everything
as developers, as I'm sure everyone listening,
and we're kind of the crowd that all the
agency stuff is really catered to at the moment.
So I feel like anytime anyone's talking to AI,
there's a little bit of our club talking, at least
for now. It's taken over the world, for sure,
but we're the ones deepest into this fiasco
called AI. That's whatever it's called.
Well, there was two devices. One got the most
attention. The second one is the one that I'm
also extremely excited about, but I am excited
that you're excited for Microsoft's hardware. I've
been a huge fan of the Surface line of devices
for a long time. They're beautiful hardware.
Frank, I'll let you lead, because I know all the things about this device.
So got the sale you from an outsider's perspective,
what I noticed, I noticed Microsoft released
and partnered with NVIDIA to create a new RTX
spark, whatever the heck that is, Surface laptop
Ultra, the ultra being very important because
there's a million surface things out there,
laptop being important, the ultra meaning it's fast.
This is an arm chip, and you will believe
how many web pages I had to look through to find
out what the architecture of the chip actually was.
This is an arm chip, everyone, and it's an
arm chip bolted to a giant GPU, or it's a GPU
bolted to a small arm chip. Not really sure which
one it is. All crammed inside of a laptop case,
and speaking as an Apple Dweeb, this is
definitely a big competitor to the M5 and the big
big powerful Pro Max that have been coming out lately.
This is Microsoft stepping into the ring
NVIDIA joining them because we need GPUs and they're
the only ones that can afford RAM these days.
So they're all getting in there releasing this
big beautiful monster of a machine with a monster
price tag. I mean, not forget to keep saying that,
but it's a beautiful machine. Like you can't argue.
The video they used to introduce it was very
Apple-esque, the machine itself is very Apple-esque.
It's a big beautiful Pixel Sense 200 and let's
just go with 50 pixels per inch. It's high res.
That's all that really matters. It's an HDR
monitor, and it's got a ridiculous number of
NVIDIA cores in it. And as someone who does a lot of
ML still, like I train my own models and everything,
it's really cool to have actual NVIDIA hardware on.
Basically, it's a personal computer, I suppose.
It's more of a workstation crammed into an laptop,
but it's cool to have that NVIDIA hardware because
as much as I love my apples, the metal ecosystem
just isn't nearly as advanced as the CUDA NVIDIA
ecosystem. The kernels are just a little bit different.
Some of them are buggy. A lot of them are much
slower than the CUDA world. Right now, my room is
20 degrees warmer than it ought to be because I
have my RTX 3090 crushing away on some numbers,
doing tensor math, using all the flops in the world,
tops. I don't even know what they are anymore.
That's all NVIDIA hardware. That's CUDA. That's
my happy place. You can run Python. Any model you
download from the internet is going to happily run
on CUDA hardware very efficiently, and that's
what they stuffed into this laptop case. So for
$2,600, you can get a giant GPU somehow attached to
a monitor and a keyboard and a trackpad all glued
into a very fine-looking case. It's a big price
tag, but gosh, it's a beautiful machine. How can
you not be excited about it? You'd have to totally hate tech to not be excited by it.
No, I mean, you're right. It's a big, beautiful
machine. The starting price is $2,700. It goes up
to almost $6,000. Now, I do agree with you.
The real core is actually the 32 giga-ram
unified, so that's going to get you basically,
I don't know, 3300 or whatever. There's a bunch
of different tweaks on here. But yes, yeah, I mean,
you're right. It's the RTX Spark. I'm looking
over here so I can get the tech specs. N1x,
it has either, there's two models for the laptop.
One is 5,120 core GPU with an 18 core CPU or a
6,144 core GPU and a 20 core CPU. There's a lot
of cores, Franker, and it has, I don't know if
you've heard of this thing called unified memory.
Can you just describe what that means? Unified memory.
That's why it's an arm. I should have known
just from the unified memory that it was armed
because I don't know Intel's still not doing it
for whatever reason. Unified memory means the GPU
and the CPU are sharing the same memory bank,
which is a wonderful thing, A, as programmers,
because when I'm doing development, I have to
first get my data into system RAM, and then it has
to be uploaded to the GPU. That's because I have
an Intel system with non-unified memory. It's a
pain in the butt because there are two different
speeds, honestly. DDR4, usually as your CPU RAM,
some people have modern computers, GDR6, 5, 6,
but it's usually quite a speed difference between
the two, and in the past, it was graphics RAM.
Graphics RAM was more expensive and faster
than stuff running inside the GPUs. Nowadays,
it's getting a little bit weird. You always had
this very awkward programming model of you would
have to really write pretty sophisticated
software to keep feeding data into your GPU,
to keep the GPU hot, spinning so that your room
increases by 20 degrees whenever you're in it.
That's very important. But it's kind of a pain
in the butt because you're constantly having to
synchronize these two, basically there's two different computers in your computer,
and you're having to synchronize the memory through
it as programmers. They would try to hide it from
you, but like all bad abstractions, it was a
leaky abstraction. And if you wanted to get high
performance out of training or inference, either
side of the ML world, you had to get really good
at memory transfers. You had to fully understand
all the DMA channels and all that. Unified memory
obviously simplifies all that. I think it
premiered on the Xbox one, Xbox, maybe just Xbox.
Apple's been doing it forever. The beautiful
thing is we don't have to do that synchronizing
visualization as developers. The second beautiful
thing is when you increase the RAM of the computer,
it's all just available to the graphics card.
I love my RTX 3090, but it only has 24 gigabytes
of RAM. And the machine has 16 gigabytes of CPU RAM.
So technically I have 40 gigabytes of RAM,
but can I use that for training? No, James, I cannot.
I am stuck with 24. And just two hours ago,
my training crashed because I ran out of memory.
How app or po? Oh, wow. Unified is great for two
things. A, you get more. B, it simplifies the
programming model. That's why we all like unified
memory. It's important. It's the future.
Some would say the past if you're an Apple user.
Yeah. And I think that's one thing that Apple's called out a lot in the M series,
the unified memory between the CPU and GPU and
it's kind of the architecture that is kind of like
moving forward enables a lot of scenarios.
And yeah, so this thing goes anywhere from 24 gigs of
LPDDR 5X unified memory up to 128.
And it also goes up to, it says a removable.
You know, that's the dev box is a removable.
Is it is the surface laptop?
One removable. I think the laptop had screws for the hard drive also. If I was,
I was, I couldn't tell because I had a really
smooth video where the whole machine explodes.
And I couldn't tell what you're allowed to explode
yourself and what you're not. But it looks like
you could unscrew that hard drive from it says
removable says removal, solid state drive, Gen
five SSD, the Gen four for the 512, but Gen
five for the one terabyte and two terabyte.
Yeah, the resolution of the screen is 32 70 by
2180, 262 PPI on it. And three by two ratio,
like all the other service devices, H.R. all that stuff that's on there.
But I think in general, yeah, it's a, it's a,
as you saw, like a super gorgeous device.
And it's a really high end device. I think what,
Pavan was saying was you need device for creators
and the kind of like Harkins to what Apple
originally had like with the pro model of devices. And now they saw the, obviously,
the pro and the studio models, but then they
delineate them with the specific like chips,
like your pros and your max chips that are out
there. And, you know, by the name of this, the
ultra, you know, that is more than what the
surface laptop and the studio would be is like,
they're saying, this is the ultra, this is the top
top to the top end that you can get for creators.
And they showed a lot of different scenarios,
specifically of running a lot of local models.
And we'll talk about local models and training
and things, but a lot of different developer
workflows, creator workflows. There was one
specifically that they showed about an artist,
that does a lot of like different artwork, you know, kind of like mock up artwork of
space and stuff that you and I, you know,
gel with those kind of very like Firefly ask.
Serenity asks, but like that type of that world
is kind of like, you know, illustrations.
And he was running it through all of these different models that once he had drawn,
drew, draw the base of it, then he could run
it through all these models, like locally,
like five different local models, some cloud
models, do a whole do a whole screen, go through
blender and do all this work all locally.
And he was like, things that used to take me like
weeks now take me like a day to do like all on
this device, like with the power of it. And
it's not just these models, but also the, they're
talking about super nemotron. And then also the
MAI code one flash model, which is 120 gig, 120
billion perimeter model running locally on these
devices. And I think that's like super crazy impressive.
I'm still running, having R. Dem running
any models on my, you know, M4 Pro 64 GBS. I can do it.
It's just a struggle. That's all I'm saying.
It's going to be a struggle. I mean, except for MAI,
it's going to be a struggle with everything.
Running a local model sadly is a little more
complicated than it needs to be. But you hit the
nail on the head because you can make the most
beautiful machine in the world, but you need to
have good software to run it. And that's kind of what I was saying about the, the,
CUDA and video world. That's the one nice thing
about that world is there's a lot of AI software
out there tuned to it, but it's not just that.
It's video production software is tuned to it. And video
not only is a large profitable company, but it
turns out they've taken over a lot of the software
ecosystem. So a lot of your high end plugins for
things take advantage of all these Nvidia chips.
We shouldn't forget gaming. This is going to be
a monster gaming device, but I don't know what
gamers are going to spend the way they showed gears
running on it while they were doing all this stuff.
Yeah, definitely more for the creator crowd.
If you're doing like a lot of CAD, if you're doing
movie production, all that stuff, things that can
take advantage of these ridiculous GPUs they're
putting in there. But yeah, building up that
software ecosystem is difficult. So being within
video, they're lucky. They're just inheriting a
bunch of software that's just going to work with it.
But then they made a bunch of enhancements to
Windows also that's going to take advantage of it.
The one that I found kind of interesting was,
and I'm you're going to have to help me here,
because I could never quite remember its name.
But there's machine learning on Windows. It's kind
of the equivalent of core ML on the Apple side
where you can kind of recompile your models to run
nicely on Windows. And I believe like, on X models,
ONNX models, which is an interchange format,
has like native drivers to run officially on
Windows. So it seemed like a big part of this
announcement wasn't just this hardware, but all
these software upgrades to Windows specifically
for being able to run these more advanced scenarios
more easily. Because you could always do it like
I'm saying, you could always get torch up in Python
and get these things cranking. But it wasn't easy.
It wasn't fun. And so I think, and you've had that
experience on Mac. So getting these things to be
easy to run is important. Yeah, and that's what it
was specifically. So there's a big blog post on
the Windows experience blog, which is, is late
to power this. So Windows ML is an AMVIT.
Thank you.
It is a scalable runtime to deploy model across
GPU MP and CPU. And then they announced support
for llama CPP, giving developers more open source choices and also, pie torch.
As well for Windows ML. So pie torch and also
llama CPP, bringing those directly into it,
which is pretty rad. So in general, what that
looks like, they have a diagram on this website is
like your native app or web app. There's like a
custom open source model for on X, pie torch or
GGUF goes into Windows ML and then interlaces with
the Windows Silicon environment, whether it's
Intel, Qualcomm, AMD or NVIDIA. And then
that accesses the GPU, MPU and the CPU too.
Yeah. And let me kind of break this down as like
a practitioner of ML. You're going to use your
pie torches and NVIDIA CUDA stuff to develop
ML stuff. And it doesn't even have to be large
language models. It can be all sorts of things.
It can be the imaging stuff. It can be audio stuff.
It can be video stuff. But that's going to be like your development environment.
llama C++ is very nice for the nerds out there
who are building their own harnesses and
development environments and everything that
are going to be executing these models. Because
a lot of the open source models out there come in llama C++ native formats GGUF.
And so you're just going to need that file
format and something that can execute that file
format. That's why it's a big deal that llama C++
comes out. But as app developers, if you actually
want to deploy the stuff, you really honestly don't
want to use either of those. Because deploying
torch and Python is gross and annoying. llama C++
is just hard to keep up with. It's a moving target.
So that's where the nice Windows ML. It's nice
to have a stable deployment target. And in the
past, it was onyx. But onyx has onyx is kind of
just the file format. So there can be different
engines that execute those onyx models.
So as developers, if you want to add custom ML stuff to
your software, you want to be deploying with
something stable and platform approved. And that's
where like Windows ML comes into play. So it's all
those different areas like from development up to
nerdy stuff or you're willing to get your hands
a little dirty up to this is the clean polished
rosy path for deploying your things. And it says
here, you know, they're contributing back to llama
CPP and in partnership with NVIDIA to your point
here in the broader community. They're delivering
a bunch of performance improvements to llama
CPP and GUF support in the format through CUDA
kernel optimization, kernel fusion improves
CPU, GPU scheduling, weight repacking, and
kutigrafts that work also adds Eagle three, MTP
and D flash two, speculative decoding, multi-GPU
execution, NVF P4, new model architectures and
back end samplings, all sorts of stuff. I don't
know what any of that means, but sounds cool.
I could tell you every single one of those, but all
you need to know is that's the modern stuff invented
to make these things fast. We're just trying to
go fast. That's all we're trying to do. It's easy
to train these things, but making them fast. But
it's funny, that's not actually how the presentation
started. They didn't start with fast. No, what
did they start with? Do you remember? Well,
we started mostly with I think I think Kayla was the
first demo. Was she the first demo or the second
demo? Even earlier, I think it was safety and
security James. It was actually, yeah. I think
Satya said safety was actually the very first thing
that he mentioned. I'm not really with him.
I say throw caution to the wind, let these things
have full access to your computer, give it every
login you have, allow it to use all the memory in
your house. Just put RAM inside the computer and
turn the crank until it has enough RAM. That's
what I think. But who am I to argue with Satya?
What they announced was the Windows container
support. I don't know, was it in a beta or something?
Because they made a big deal. It sounded like
this was almost an official release or something.
Was that what happened? It was in preview a build.
We announced the Microsoft execution containers MXC.
MXC specifically works across Windows, Mac,
and Linux. What it does is it gives process and
session isolation. Obviously, on the different
operating systems, it's going to use different
container systems that are there. Obviously,
on Windows, because it's the stack that they own,
they can go a lot deeper on it. On Windows,
it has a lot deeper process and session isolation
through MXC locally with these one-off process
isolated sessions. It now supports WSLC, virtual
machines, and Windows 365 for agents, which is
really cool. I was demoing this myself the day
before. I'm very in tune with it. Actually,
by demoing it, I'm in love with this more than ever.
It was cool that it was the first thing.
I had been -- this is just sandboxing people.
We heard about sandboxing before in VS code and
the Co-Pilot app and the CLI. MXC now is
rolled out across codex, Co-Pilot, OpenClaw,
Replit, LM Studio, Unscent AI, and a bunch more
that are releasing support now that this thing
is GA, which is super big because the standardized
support gives us developers. It's actually just
an open source library that you can integrate into
your application to provide this functionality,
like everyone else has. Then as users, we get
this really rich features and capabilities.
So you can think of it as you're able to control
disk access, network access, clipboard access,
all these other things that you're able to really get fine green control on.
And what the first thing I did was I'm like,
why ever what I want to give, for example,
the CLI, VS code, or the app access to my
OneDrive, my work OneDrive. So you can just
say block this folder. Just like, you never need
to go to my OneDrive. That's my business stuff,
right? So you're there. Now, a lot of these
sandboxing are really nice. So it'll be like,
hey, I'm trying to access this file. It's in a
sandbox. It's a shik did. Am I allowed to? And
you could break out of the sandbox. If you
need to, there's always those gates just like,
do you approve this thing or not? Now auto approval
on this thing, it needs this human interaction
to do that. And it would like turn it off with
that call and then turn it back on and back and
forth, which is fascinating. But that's up to the
client, that's up to the app developer to enable
that functionality. Now, the real cool part about
it is that most of the apps today are doing process
isolation. So that means for most intense purposes,
and that can be, that could be like a five second
boost up to like they could have a process that's
running that's isolated. So when you boot up
codecs, like every session has its own process that
is then isolated, but you could make it so you
make a request an agent and as soon as that that
turn is done, you kill it, right? You could do
that as an option. Normally, even with the CLI
or VS code has a process that's running that's
sitting there for each of the sessions.
And those are in isolated process there. However,
the thing that's even the cooler than that is
session isolation. And session isolation is cool.
And because Open Claw has it integrated today.
And what session isolation does is it actually
gives a full like user isolation for that application running. So the Open Claw
that is running, the agent that is running in
Open Claw has its own basically user account
and its own user folder that you can peek into,
but it can't poke out to, which means every
process that is spun up in the session of the
app, like the entire app is running as its own
separate user account, which is 100% then isolated
from you. And that's like built into this thing.
And that is like super crazy cool, actually.
I thought that was like really, really neat.
And I've just like really fallen in love with
it, funnily enough, because I was demoing it
nonstop and I was like, oh, crap, this is like
really cool, you know, in general, you know,
and then you can have like a allow list and
denial list and read only, write only the only
things. The only problem is you really got to
set it up. You know, that's the main problem
in general. However, what I'm really thinking
is, okay, let me have some standard defaults.
Such as maybe don't be able to do this stuff in
my one drive folder, your drop box or whatever,
you know what I mean? So just some like default
standards, but you could imagine in the enterprise,
this is like crazy important. And they showed a
bunch of other windows features of like really
deeply integrating these things were really agents
almost really have their own identities on their
on your machine, which is like so cool. Which is
right, you know, Unix invented this in the 70s.
Yes, user accounts are good and they can be used
for other things are than users. That's just a
good isolation mechanism. So it's good to see it
actually extending. Honestly, I have a little bit
of mixed feelings here because I think with any
new security model, there is that time to figure
out what should be the defaults and whatnot.
You know, so every time I open a session in VS code,
it has to ask me permission to like read something
from Microsoft. I'm like, you can you can use
the Microsoft.com website. It's fine.
It feels like no matter how many times I say allow,
a new security model gets introduced and that
allow is turned off or something like that. So,
as a user, I get a little bit frustrated with
there's a little bit of theater with our security
and there's a little bit of real security with
our security. And I understand those differences,
but it can still be frustrating to find that
fine line between them. And I just totally
understand from the enterprise standpoint why
they want it. But some of that stuff coming back
onto real users, it's a bit frustrating. I mean,
talking about Windows Vista, like that we were
the first ones to add the annoying pop up dialogues.
Hey, do you really want to do this? Do you want
to give permission to do this? And it took quite
a few versions to get those dialogues refined.
And I feel like we're still just in that annoying
stage. But hopefully with MCX coming out and
everyone kind of consolidating on it, we'll have
some a better defaults and be just, I can just say,
hey, yeah, here's a whole set of directories on
the computer. Go have fun. I don't care what you
do over there. Exactly. And I do think that this
is going to be ever important as these agents are
sort of like autonomous, they're just running on
machines separately. Like you said, you know,
it's like, yeah, this thing is just running over
there. And it's I'm not even like, I'm not even
know what's going on, right? And you really want it
to be its own separated, isolated process. Yeah,
MCX stuff was really cool. I really like, I really
dug it. I was super excited about it. And just
how easy it was. And now that I've dog deep friends
actually using it a bunch, it's definitely worth
checking out. And there's like definitely some
smart defaults. I think the problem is like if you
have sandboxing and then you have like, there's
a few things, right? So for example, there's,
there's like sandboxing to say, Hey, listen,
I want you to run all my terminal commands in a
sandbox. That is one thing versus a network, disk
access, things like that. Because the problem we
run into as app developers is, Hey, I need to do
a donnet build. Oh, donnet's not installed in the
sandbox. Yeah, right. That's a problem. Whereas
what I'm talking about is more permission based
sandboxing for accessing locally on your machine.
So I do think when we think sandbox is a pretty
loaded term. But what I was at least setting
up as my default for MXC. As far as like,
file and network access, which networks access
for me is pretty much like, yeah, it's fine. But
you could imagine an enterprise, you could just
say, I only want to access like local things. I
don't want you to go out to the internet that
would probably like import for them. But there's
this and one of the cool parts is like, it also like
proxies the access through that. So for example,
I had this demo that was, Hey, can you go get
me a list of like my GitHub issues or whatever.
And it was going to use the GitHub CLI to go do it.
But the GitHub CLI is running inside of the
sandbox with the sandbox networking, which then
blocked it because it can't access GitHub.
Because I turned off network access outside of it
on this project. So kind of interesting in that
regard. So I think it's just fine line of like,
okay, what am I, what things am I doing versus like,
I just want to build a project, like just run it
on my machine because I have all the stuff on my
machine and not in the sandbox. And that's what
kind of is right thing versus agent versus like,
developer build tools, like those two worlds seem to kind of like.
Right. But I feel like we're the first beta
testers of all this agent stuff. Yeah.
All the under other industries, they're still in
the chat world. You know, they're still doing
everything at the chat prompt level. We're the
ones doing the agent and craziness, let it take
over my computer stuff. And that line,
it still is a shaky line. Like just yesterday,
it couldn't build my app. And I'm like,
why can't you build that app? They're like,
dotnet restore is stalling. I'm like, why would dotnet restore? So they're like,
because I don't have network access. I'm like,
oh, okay, why did it take a why to take me three
wise to find out you don't have network access?
And B, I'm like, why don't you just enable network
access? And it's like, great, I will. And then the
Mac dialogue pops up saying, Hey, do you want to
give it network access? Mac OS 27 changed that
dialogue box. So it took me a while to figure out
how that dialogue box works. Good. Finally,
I granted access. And it's like, Hey, you have to
restart the harness in order to do this. And I'm
like, but I'm in the middle of a session. So I'm
just deep breath. That's like the security stuff
is a little bit. So let's just hold them down.
So that's good. We'll work it out.
We'll figure this stuff out. How hard can it be?
Well, the next thing that they talked about beyond
that, right? And also, like at the very end,
we'll talk about the other hardware that I'm more
excited about. So there's the another piece of
hardware. But we'll talk about every single
announcement here as we get into like some deeper
Windows stuff. But one of the things that they
really focus on was sort of like local and hybrid
AI model usage. And one of the things that Kayla
showed off, because on my team, which also
worked on the Windows terminal team, just to be on
the Windows terminal team, is basically frontier
coding models, like local and your machine.
So there's a few things is like on this machine,
what they were talking about is like, Hey, taking
me I code one dot one flash with Nematron models,
also deep seek V4 flash, like that is a 137 billion
per parameter. You know, total 6.8 billion active
and video is a 70 billion parameter, two
bit quantization down to 20 gigs on memory.
Deep seek V4 flash is a 284 billion parameter model.
So there's all these things that are going to
run locally on the RTX spark, which is crazy.
Obviously, I'd imagine, depending on what RAM you
have, you want all the RAM, by the way. And they're
optimized and tunes to basically run on these
machines. Now, of course, you can run those
Lego machines, but you also need great tooling
to run things locally. And this is where the
brand new local model integration in GitHub
co-pilot comes in. And there's two things
specifically. One is that Kayla showed off like
auto local routing and specifically that was
turning into efficiency mode and the model.
The router can route to a local model on disk
automatically. And this efficiency mode has a
little like, a little leaf icon, which is very cute.
But the thing that I think is even more cool
is the hydrophusion support, which you did an
entire podcast on. And hydrophusion can then
decide to delegate out different tasks to
different local models on disk and make decisions
optimizing your token usage across the board.
And this thing is coming to like everything, basically,
out of the box, along with all that sandboxing
things that happen. And Kayla had a really good
demo, which was, hey, like, I've done this repo,
and then I'm doing these investigations. And I
might be using like a frontier model in the cloud.
And then it may decide to then delegate locally,
you know, accordingly for that, those different tasks can run off subsections.
She also showed off, I think she also showed
off automations. And actually, this is actually
really genius. And I was talking to Kayla about
her usage of local models on her machine. And she
said a lot of what she's doing is, yeah, she
does have, if she's like, hey, I'm about to do
something that is going to take a while.
And I don't actually care about, like, I mean, time to
first token is important, like, you know, and then
token, like, coming back. But if she's like, hey,
listen, I know that my local models are never going
to run as fast currently as like, if I'm running
like Luna to go do a thing. However, if I'm like,
hey, I'm about to jump in a call and have a half
an hour and an hour or whatever it is, or overnight
to run long processes, what do I care if it's
running locally on my machine or if it's running in
the cloud, right? Besides that I'm going to save
a bunch of money. And then she said, she actually
has a bunch of automation set up that run every
single night at midnight when her computer
just sitting there with nothing else to do,
and is then crunching the local models to do
a bunch of issue triage bug triage support,
like kind of like these workflows automatically
that can then fire off new sessions or do a
bunch of computation. And she's like, now
that's all just free for me, right? And then,
she also showed, you know, like what she was showing
in the hydrophusion examples, it may start with
a cloud model, local model, go back to clicking,
go back and forth, but it's going to figure out
what it needs to do and what local models you have
readily available, which is like super crazy to
think about. So as she I don't know if she showed
it, but we were really showing is like, you now
could basically for all intents and purposes,
like just boot up something like this, turn on
airplane mode and go, right? So yeah, it's so cool.
Your computer will get warm during that process,
though, I promise. So you can use it as a heater.
You know what it is, it's it's liberating. Maybe
you don't experience this because I think you
have infinite tokens. But for those of us with
limited tokens, I'm always worried about like,
I'm always watching the like, how much usage do I
have left? Am I over my budget for, you know, I,
I'm always like looking at the date, oh, it's the
10th, there's 30 days in the month, so I should be
at 30% and you know, I'm doing this like stupid
math in my head. But when you use a local model,
and even a stupid one, it's very liberating because
you're just like, oh, I have as much as I want,
I can just have this thing chugging and burning
whenever I want. There's there's no little background
clock in the back of my head. Yeah, I mean,
hey, don't do this task because you want to save
your tokens for that other task or things like that.
When you have a local model, you're just like,
I'm gonna use it and I'm gonna, yeah, even the best
models out there, I was just yelling at Astra and
Astra is using up my entire month of credit in
like two hours when I use it fully. And even
then I'm yelling at it because I'm like, yeah,
that's an ugly button, you made your button ugly,
it should be better than that. And I find I yell
at my dumb models and my local models just as much
as I yell at Astra. So I would rather be burning
just electricity here in my home and yelling
at the model than worrying about token counts
that I'm paying for cloud models and all that.
So if you haven't been lucky enough to experience
local models, I don't want to go and say like,
they're just as good as the big cloud ones because
they're not, they definitely aren't. But what they
are is liberating. You just have zero guilt. You're
just like, I'm just going to use this model.
Maybe it's on an airplane. I don't fly that much anymore.
So it's fine. I don't care about the airplane
scenario. But what I care about is just not
worrying, not thinking about my quotas and any of that stuff.
We're sitting there doing these demonstrations and
when a demo or was was booting up one of these
models and just like let it rip for like 30
minutes and after 30 minutes, I had like a full
video editor. And I was just like, cool, here it is.
And there was this whole thing was an airplane.
But I was like, really kind of crazy. And like
you said, very liberating and like, you know,
token used to zero. I think you need to,
it's a blend, right? You need to use the right
model at the right time. And then, you know,
especially with those like long running processes,
like I had a session that ran for 100 hours
fully, you know, I was cranking and testing some
things out and just go into town, right? And this
is like a blend between frontier models, local
models, all these other things. I'm bopping back
and forth between them. And I think like that's
a really unique scenario where I was just like
set in, forget it. You know, I'm going to be in
San Francisco, just like, I'm not even, I'm not
even having to have this computer, my computer is
at home. What am I doing? I'm not even checking
in on it. I was like autopilot, Yolo, I'll come
back on it. It's sandbox. It's good to go. And I
think that's like a very intriguing scenario.
I mean, the other thing that they also announced,
but actually didn't like show, I mean, they showed,
but they actually have these like AI super
computer, like tower things like that. I don't know.
I don't know if it's, I don't know if it's
not to call them like a token factory or something.
I forgot what he said, something like that.
Well, basically there's these huge like basically
servers that, you know, kind of server towers,
you know, DGX stations, which are these huge
stations that they're talking about running
windows now. And these are apparently using a GB
300 grace Blackwell ultra chip DGX workstation.
It says it can unlock power to run frontier models from Loma for Maverick,
Kimmy, K to six deep seek for V for pro and more
than a trillion parameters on these things.
Oh my god. That is a great are the professional
grade model runners. Maybe someday I'll get a
10 year old one that would be nice. I think
there was a cute little afterwards during the
presentation where they were just reminiscing about
like the computer industry. And they're like,
do you remember when chat GPT was trained on a
DGX one? Do you remember that DGX one? I'm like,
guys, that was like three years ago. It's not
that long ago. Like, I would still kill for that.
And they're acting like it's all old hardware.
They're like, Oh, we just have those. We put those
out the pasture, shot them and left them for the
carry in. Like guys, I'll take your old DGX.
Just Satya, Frank will take your old DGX. It's cool.
It's cool. Yeah. Well, there's a few other
things in like windows, windows itself.
So there's a lot of things around performance,
search optimization, making search like the most
amazing thing in the entire world. I think they
really focus a lot on here the improvements that
were making to core windows, how we're making a
sort of an agent first operating system.
And I thought that was really cool how like agents are
deeply integrated. And also I think how they're
evolving co-pilot. And specifically around this,
whole idea that you have this home code and
autopilot and home is really like, I am on my
machine. I have the context that is my in this case,
like I'm in now consumer mode, right? Like,
I want to write documents. I want to do stuff.
And I want to be safe. And I want to not just be,
you know, you know, yellow on it out to wherever
context, but basically you can, you know,
co-pilot can understand relevant content on the PC,
including files, recent activity, local context.
And the cool part about accessing these is it
can also tap into local actions. So it can take
actions on your behalf. So it can organize
files, diagnose device, troubleshoot issues,
it can obviously code, it can carry out workloads,
it basically do anything you want. And I'm thinking
like this kind of like, what's the next general
of like shortcut type of thing, actions. But then
it all goes so can tap into local models too,
running on your device. So when it's accessing
these things, it's able to access the same models
that you might be using for coding or these other
things actually do this work, which is really
cool. So they're really talking about this
hybrid intelligent MXC support built in.
And then the idea that you're going to have these raw
long running personal agents too. And to your
point of like, what are people doing? Like,
actually, the thing that I'm doing at work often is,
you know, inside of co-pilot, like a Microsoft
co-pilot is I am for all intents and purposes,
having it gather a bunch of my work intelligence.
Like I do this is a thing that I do. Like I'm
planning right now a bunch of our co-pilot,
a get up co-pilot like advocacy efforts, right?
What are we doing for the rest of H1 and in H2?
And what I needed to do is I needed to tap into
my emails, to my messages, into my files on disk,
and into these conversations. And I needed to tap
into other planning docs, coerce things together,
look at latest news on the internet, pull things in,
and then take this and then transform it into
docs and reviews and these other things, and then
messages and send things and do follow-ups and
then that's the thing that I'm actually doing at
work, which I wasn't really doing all that much,
like six to 10 months ago. But I've been trying to
do a lot more of those processes as these tools
gotten better. And I actually find myself doing
this too often is in the co-pilot app, what I do
end up doing today is I'm doing a lot of
house projects, right? And I'm doing a lot of
contractor reviews and all these other things.
And right now I'm dragging and dropping files
into a thing and doing like, I would love to just
be able to do a lot of that integration into
Excel and into these other things in this workspace.
So I'm really excited about how that's evolving
and how these, you know, agent harnesses are
running. I'm not sure if I'm there yet. Like,
I'm not an open claw Hermes person at this point.
Like, I use them in some regard, but I'm starting to,
I think once you start to see how other people
work that I'm interested, like, I'm not really
interested in giving meta all my data.
Like everyone's like really excited for a muse. I have zero
interest in giving meta all of my data, right?
Like, I'm not interested in that at all. I am
really interested in local models being able to
do computation over myself, which I think of
Apple intelligence, right? Those models that are
running local and your device like, you know,
Apple, when the local model trend is there and
you know, they've been trying to evolve that in
the latest devices and the leading operating
systems. And I do like this idea of hybrid. I
think this is, I don't know if I don't, I've
been tried Apple intelligence and in Mac OS 27
yet on my machine, which I have here. But from
my experience of what I saw when I saw people
trying to use Siri was it would ask you to question
and say, Oh, I can't do this right here. I'm going
to go to the cloud. That's almost hybrid, right?
Except for you're doing these actions. In this
case, it's automatic hybrid. It's going able
to be able to do these smart things locally,
remotely and configure this for you. Now, I think
that the biggest thing that Apple got right,
then I'll be interested in this experience is
the seamless one button click, turn it on.
Like, I think that's really important.
Like inside of Xcode, inside of your Mac OS machine,
inside of your iPhone, you say, do I want to use Apple intelligence and you say,
yes. And then that's all you don't have to worry
about what model am I using? What am I doing?
XYZ? That's what I would like to see from this experience is like, if you want to go
bits and bop and pick your models, go for it.
But we're going to give you a bunch of defined
models that are just really good at doing certain
things that you're going to be doing locally
anyways, like doing text or doing image stuff or doing like profile processing.
Yeah, absolutely. And you're really hitting the
nail on the head. And I got to say, I don't think
either operating system is quite there yet
because ideally in the future, in the same way,
I can pick which search engine I want to use in my
browser, I can just say use Google or whatever,
use Bing. I would like to be able to under Syrian
Apple intelligence or Syria or under Windows ML
just say like, here's a model I downloaded from
hugging face use that for everything. That's the
model I want you to use. I want you to use it locally.
Don't log it to anything. This is the model.
But sadly, right now we're still in the case
where Apple releases one model. It's the model
that comes with the operating system. The model
that comes with Mac OS 27 iOS 27 is different than
the model that came with 26. And I feel like
Windows is going to kind of be there right now too.
Plus Windows is in a really good position because
I got to tell you Apple's dropped the ball a bit
still with Siri. I've been using it for a while.
I am not amazed. I was arguing with it all afternoon
where I would keep asking a questions.
And I wish it would say the thing like you said of,
it'll say, hey, I'm going to go use Chetchi PT or
whatever right now. No, all I kept saying was,
I didn't quite catch that. Do you want to try again?
Like, no, you obviously got the text. I can see
it. You printed the text right there. You got it.
You're just not acting on it. So for whatever
reason. And it doesn't make any sense to me because
I know I use the local model. It's like it can
do this stuff. So I don't know. There's there's
Apple's dropping the ball. I think this is a perfect
opportunity for I'm in Windows to catch up but
Windows to get ahead quite a bit because the demo
they had for the start button and search like
they were saying it's faster than you can type.
Now I know some fast type is out there.
So I'm curious how just how fast it is but like the
demo looks great. I know it's easy to make a demo
look good but you know time will tell when people
get this installed on their machines but I think
they have like Apple should have the lead in the
same way Google should have the lead in AI.
All these people who should have the lead don't.
So this is a great opportunity for Microsoft to
like really make some headroads. And I know they've
gotten a lot of flak like co-pilot this and
that and the brand is getting a little bit
overwhelmed and trying to put AI and everything
but like the Windows search box is a perfect place
to impress people. If you start impressing people
on that box you're going to get a lot of converts
and like they're just going to get used to it.
You're going to set a new bar, a new standard for
what your computer should be able to do and help
someday Mac will catch up to them at least to that
demo. Well and I think if the users are starting
to get used to this aspect like when you Google or
you Bing or you like sort something into some
box and you start to get these AI summaries and
you start to follow up with AI you start to get used
to this and you almost want that as the default
experience. So it's almost like looking forward
okay or people are people there now right. That's
the problem is you try too early or your first
one of the first there you know it's going to
take a lot of revs to get it right and you're
going to maybe make a lot of people mad and
you're going to make some people happy and then
you know maybe you all mock it and hopefully so.
But let's talk about the last thing. There's
plenty more things that we could cover but
a thing that I am most excited about in
this moment is not a laptop. Frankly. Okay.
It is a PlayStation 2. No it is a brand new it is
a brand new it's a brand it kind of looks like
a hybrid between like an Xbox and a PlayStation
2 like slim which is the Surface RTX Spark Dev
box and I'm pretty sure they showed this app build.
I think I'm pretty sure it was there and if not
well something was there but okay right I think it
was there but this for all intents and purposes
is very similar to the ultra however there's one
configuration you get one you get one that's it
one price one configuration and this really
is I believe it has a default out of the box
Windows 11 experience for developers like it is
it is a special image that is made for developers
I don't know what that means but they said
it on there from like what it's supposed to
I'm pretty sure it's what it is but it is a
the same 6144 core GPU 20 core CPU the biggest
it is 128 gigs of LPDDR5X Unified Memory and it
is the biggest SSD the two terabyte which is
removable by the way in this puppy and this thing
looks just gorgeous it is a big ish not super big
but big ask basically dust on PC and it starts at
6000 lives yeah 559 yeah add us tax to that you
owe the dollars I've held it it's beautiful
it's fantastic and yeah it's it's what I want
I think like I know it doesn't have a screen
Frank but also our Mac mini zone have a screen and
I I tweeted after the event that it is official
that my Mac mini is officially for sale so I am
yeah traitor I'm sure I'm done you know it's bigger
it's bigger though it's not many I want to call
it many and so it looks like it's about the size
of the laptop have you held one have you seen it
in person oh I have yeah the I have held it in my
hand the question is are the are the technical
specification right sure sure yeah don't worry
okay yeah they well they have it in photographs they
have it next to the laptop and it looks roughly
the size of the laptop it looks like a giant
pancake it's it's definitely what I appreciate
is they're not ripping off like the Mac studio
design or anything this is a pretty unique design
it looks more like a game console than anything
else your PlayStation 2 reference is actually
pretty pretty spot on I would say my biggest
question is where are the fans on this thing
because again I use my GPU as a heater I need
fans on it to distribute the heat throughout the
whole room so that while I'm training neural networks
I can also be heated this winter that's a very
important feature so I'm curious how this thing uh
thermally conducts it's thermal yeah I was just
because um it's pretty thin you know it's it's kind
of like a two-story um it's got a smaller bottom
than a in a bigger top it's a little mushroom
shake and it was a little bit confusing like where
they're putting all the fans and everything on it
it's gorgeous I love it um it reminds me of like
the old sun microsystems boxes I don't know if
you remember any of those from the early internet
days okay no never mind um my biggest complaint
because I was telling you before we started recording
is for roughly the same price it's almost cost as
much as the same configuration as the laptop but
you're not getting the screen you're not getting
a keyboard you're not getting a touchpad uh what
you are getting is about 25 more watts of power
I should say that the laptop is 140 watt power
supply and this puppy is 165 watt power supply I
still think that that's a little bit on the low
end from my personal taste because my 3090 right
now is literally taking up 350 watts of power so
it's already running double the amount of power
um that this thing can produce now obviously this
is a more modern chip it's gonna do better it's a
it's more efficient chip than my guy but you're
still stuck within that power envelope so it's good
to know that um with the laptop you're in the 140
watt power envelope with this guy you're in the
165 watt power envelope and these numbers matter
like physics matters you've got to shove electricity
into these things in order to make them go fast
um so it's it's pretty cool to have this box out
I don't know who's gonna buy one um I hope that
you're able to make a lot of money on your Mac mini
I'm gonna be so sad that you're not gonna be my
Mac mini brother anymore I'm just gonna be out here
Mac miniing by myself but I hope you're able to
get at least 10 grand for your Mac mini so you
can buy two of these things I did look I did look
and I did see that I should be able to make a
few hundred dollars off of you really yes okay I
don't know did you see did Miguel say he's gonna
get the six pack of these you know I don't know I
didn't I didn't see I should a six pack of these
uh dev workstations that's gonna be pretty awesome
yeah I'm I'm pretty jagged I can't I can't argue
it's just a lot of money it's a lot of money I'm uh
I think you know for me obviously we'll keep my
Mac book air because I do iOS development I need
a Mac around like you cannot get around it and
the m1 still it's still great device I don't know
I know it's still the rumored like OLED um Apple
laptops coming out in a couple weeks oh which by
the way a correction for last week so the current
rumor is they are touchscreen by the way so oh I
don't know did they say whether the new ultra is
a touchscreen or the ultra is a touchscreen yeah
yeah I figured as much because Microsoft laptops
have they're all they're all the touch sense yeah
ultra whatever it is so um I think I said on the
podcast that it wasn't going to be a touchscreen
Mac and it hasn't been announced so this is not
official but the current rumor mill is it is
actually a touchscreen Mac so I was like maybe it
is time for me to upgrade but then like look at
my cute little m1 I'm like you're such a trooper
little m1 I don't think I could ever sell you or
get rid of you like if I ever drop you down
a six-story building or something like that
accidentally then maybe I will upgrade and fight
you're right yeah it's hard to get rid of the
thing it's been so much value for the buck yeah
I'm I'm super jazzed on it this my current plan
and you know what I really do like my my my Mac
mini and it's really great I just actually don't
even know if I've taken as much advantage of it
as I as I should then no no I know if I will take
advantage of the the dev box or not but we'll see
at least and I'm excited to go through this journey
of where I'm at so the the interesting part is
just like I went through tons of transitions
with Thunderbolt 5 and this and that okay
well I go through all this again with all my
crazy stuff I will say this so I'm also reducing
my plan is to reduce my ultra wide I'm getting
rid of my ultra wide okay I'm convinced by John
Galloway that this $200 Dell 4k monitor that he
got on big day deals was what I should get because
he said it was good and I said I don't have Amazon
Prime so you're gonna need to buy it for me wink
free free monitor um so he said he said so so it's
it's come into the house which is good which is
nice and getting rid of that I want to like redo
my setup yet again obviously but um it just it's
really tricky okay I don't I have never really
felt like like I think when I was building my
custom built tower desktop PCs because I was
designing what it was I was understanding the ports
I was preparing in it like it was perfect I don't
know if I have rather perig monitor setup but I
do feel when I had two dual screens it like felt
correct like a dual screen monitor setup it felt
correct for a while but I do that yeah but I do
feel like I really want to trim it down that being
said Frank Ruger I could be convinced into an
ultra because it has the other monitor right I
would have another monitor that's there that I
could use and it coordinates with this new 27 inch
4k monitor side by side like I think about it you
know what I mean yeah I don't have to get a stand
for it I gotta do something I just think that that
that box will look so sleek just sitting there
we'll see I don't know who knows what I'll do
in general but that's right at least where I'm at
right now and I'm real excited I feel like you
know I've always been a Windows person and when
the Mac mini came out the size of it the power of
it spoke to me and so does this that box it really
speaks to me as a as a aesthetics developer mini
PC that I want and I think I'm gonna get it so
we'll see oh boy well I know I'm gonna send you a
bunch of models to try on it because I'm gonna be
curious how fast it is so I'm gonna make you test
the few things some remote desktop access into
your own private account that's something nice
make it into a proper workstation server yeah
oh we didn't mention the most important part of
the laptop before we go I just have to say don't
call it make safe but they added make safe to the
laptop it is a wild weirdest little USB-C connector
you've ever seen in your life and no one believes
it works but I've seen video online of people
fake tripping over it it seems to work fine
come on apple catch up because it's pretty
cool it's a pretty cool design they did there
with the USB-C because it's USB-C and it's make
safe but it's not safe copyright trademark
apple it's a it's the world's first magnetic
USB-C I have one right here yeah boom there it is
and it's over yet yeah have you proven it I have
shipped over it many times and it does work it
does work yes and it snaps right in it's it's very
very nice so nice yeah it's absolutely delightful
all right that's gonna do a people for this
week's emerge conflict go spend all your money
asking me service devices I am super stoked
super good but that's gonna do for this one so
until next time jay's on to magnow and i'm frank
rueger thanks for watching and listening peace