Skip to content
Episode 537

537: Going ULTRA with RTX Spark and New Surface Devices

October 19th, 2026

57 mins 10 secs

Your Hosts

About this Episode

Microsoft’s new Surface Laptop Ultra pairs an ARM chip with NVIDIA RTX graphics and unified memory, promising workstation-class power for creators and local AI—at a premium price. The conversation explores why CUDA support and Windows ML matter, how local and cloud models can work together, and how Microsoft’s new container-based isolation aims to keep autonomous coding agents safer. A look at ambitious hardware and the software ecosystem needed to make it useful.

https://blogs.windows.com/windowsexperience/2026/10/07/building-windows-for-hybrid-intelligence/

Follow Us

⭐⭐ Review Us ⭐⭐

Machine transcription available on http://mergeconflict.fm

Support Merge Conflict

Transcript

Welcome back everyone to Merch Confiture Weekly Developer Podcast.

Talking about all things in the world of

software development, AI, and new hardware.

I'm one of your co-hosts who works at Microsoft James Montemagno,

and with me, the one and only who used to work

at Microsoft, Frank Riggas, and gone buddy.

Oh, it's been a while. Do they still call it Microsoft? Is that still the name?

Is that trademark registered? Yeah. Okay.

MSO, still the ticker.

Tick tick.

Hi, James. It's wonderful to be here on your show.

This is one of my favorite weeks where we are going to talk about hardware spoilers.

I know we like to ease into our topics,

Read Full Transcript Hide Full Transcript

like talk about random stuff for 15 minutes,

but I feel like we have a lot to cover this week because

Microsoft's in the hardware game, you know, like they put their toes in,

and then they step out of the water. They're like, "Who's that?"

It's a little cold, and then they get back in,

and then they're like, "Eh, I'll try it again."

This is one of their, "We're just going to cannonball into the water."

And I don't know what you want to talk about.

What I want to talk about is this big, cool laptop

they just released, and I just want to dive into that puppy.

James, have you heard about this laptop that a little MSFT company has released?

Well, a little company out of Redmond, Washington.

I have one sitting in front of me, so yes, yes, I know.

Okay. Well, aren't you rich because, man,

the starting price of this thing is just as big

as its beautiful screen.

Well, no, there's a big hardware event,

but also a software event and a local, like,

a model event too. It was kind of... It was

like a windows and hardware event, and to me,

it was also kind of also somewhat a developer

event, a little bit too, I want to say.

I mean, we're getting all the attention right now,

right? Because anytime you say AI, it's like,

"Okay, business people do something with it.

I'm really not sure what business people do.

I assume they just chat with it all day or

something like that. Maybe it fills out their

Excel sheets or something, but I think you and I

have the most experience with agents and everything

as developers, as I'm sure everyone listening,

and we're kind of the crowd that all the

agency stuff is really catered to at the moment.

So I feel like anytime anyone's talking to AI,

there's a little bit of our club talking, at least

for now. It's taken over the world, for sure,

but we're the ones deepest into this fiasco

called AI. That's whatever it's called.

Well, there was two devices. One got the most

attention. The second one is the one that I'm

also extremely excited about, but I am excited

that you're excited for Microsoft's hardware. I've

been a huge fan of the Surface line of devices

for a long time. They're beautiful hardware.

Frank, I'll let you lead, because I know all the things about this device.

So got the sale you from an outsider's perspective,

what I noticed, I noticed Microsoft released

and partnered with NVIDIA to create a new RTX

spark, whatever the heck that is, Surface laptop

Ultra, the ultra being very important because

there's a million surface things out there,

laptop being important, the ultra meaning it's fast.

This is an arm chip, and you will believe

how many web pages I had to look through to find

out what the architecture of the chip actually was.

This is an arm chip, everyone, and it's an

arm chip bolted to a giant GPU, or it's a GPU

bolted to a small arm chip. Not really sure which

one it is. All crammed inside of a laptop case,

and speaking as an Apple Dweeb, this is

definitely a big competitor to the M5 and the big

big powerful Pro Max that have been coming out lately.

This is Microsoft stepping into the ring

NVIDIA joining them because we need GPUs and they're

the only ones that can afford RAM these days.

So they're all getting in there releasing this

big beautiful monster of a machine with a monster

price tag. I mean, not forget to keep saying that,

but it's a beautiful machine. Like you can't argue.

The video they used to introduce it was very

Apple-esque, the machine itself is very Apple-esque.

It's a big beautiful Pixel Sense 200 and let's

just go with 50 pixels per inch. It's high res.

That's all that really matters. It's an HDR

monitor, and it's got a ridiculous number of

NVIDIA cores in it. And as someone who does a lot of

ML still, like I train my own models and everything,

it's really cool to have actual NVIDIA hardware on.

Basically, it's a personal computer, I suppose.

It's more of a workstation crammed into an laptop,

but it's cool to have that NVIDIA hardware because

as much as I love my apples, the metal ecosystem

just isn't nearly as advanced as the CUDA NVIDIA

ecosystem. The kernels are just a little bit different.

Some of them are buggy. A lot of them are much

slower than the CUDA world. Right now, my room is

20 degrees warmer than it ought to be because I

have my RTX 3090 crushing away on some numbers,

doing tensor math, using all the flops in the world,

tops. I don't even know what they are anymore.

That's all NVIDIA hardware. That's CUDA. That's

my happy place. You can run Python. Any model you

download from the internet is going to happily run

on CUDA hardware very efficiently, and that's

what they stuffed into this laptop case. So for

$2,600, you can get a giant GPU somehow attached to

a monitor and a keyboard and a trackpad all glued

into a very fine-looking case. It's a big price

tag, but gosh, it's a beautiful machine. How can

you not be excited about it? You'd have to totally hate tech to not be excited by it.

No, I mean, you're right. It's a big, beautiful

machine. The starting price is $2,700. It goes up

to almost $6,000. Now, I do agree with you.

The real core is actually the 32 giga-ram

unified, so that's going to get you basically,

I don't know, 3300 or whatever. There's a bunch

of different tweaks on here. But yes, yeah, I mean,

you're right. It's the RTX Spark. I'm looking

over here so I can get the tech specs. N1x,

it has either, there's two models for the laptop.

One is 5,120 core GPU with an 18 core CPU or a

6,144 core GPU and a 20 core CPU. There's a lot

of cores, Franker, and it has, I don't know if

you've heard of this thing called unified memory.

Can you just describe what that means? Unified memory.

That's why it's an arm. I should have known

just from the unified memory that it was armed

because I don't know Intel's still not doing it

for whatever reason. Unified memory means the GPU

and the CPU are sharing the same memory bank,

which is a wonderful thing, A, as programmers,

because when I'm doing development, I have to

first get my data into system RAM, and then it has

to be uploaded to the GPU. That's because I have

an Intel system with non-unified memory. It's a

pain in the butt because there are two different

speeds, honestly. DDR4, usually as your CPU RAM,

some people have modern computers, GDR6, 5, 6,

but it's usually quite a speed difference between

the two, and in the past, it was graphics RAM.

Graphics RAM was more expensive and faster

than stuff running inside the GPUs. Nowadays,

it's getting a little bit weird. You always had

this very awkward programming model of you would

have to really write pretty sophisticated

software to keep feeding data into your GPU,

to keep the GPU hot, spinning so that your room

increases by 20 degrees whenever you're in it.

That's very important. But it's kind of a pain

in the butt because you're constantly having to

synchronize these two, basically there's two different computers in your computer,

and you're having to synchronize the memory through

it as programmers. They would try to hide it from

you, but like all bad abstractions, it was a

leaky abstraction. And if you wanted to get high

performance out of training or inference, either

side of the ML world, you had to get really good

at memory transfers. You had to fully understand

all the DMA channels and all that. Unified memory

obviously simplifies all that. I think it

premiered on the Xbox one, Xbox, maybe just Xbox.

Apple's been doing it forever. The beautiful

thing is we don't have to do that synchronizing

visualization as developers. The second beautiful

thing is when you increase the RAM of the computer,

it's all just available to the graphics card.

I love my RTX 3090, but it only has 24 gigabytes

of RAM. And the machine has 16 gigabytes of CPU RAM.

So technically I have 40 gigabytes of RAM,

but can I use that for training? No, James, I cannot.

I am stuck with 24. And just two hours ago,

my training crashed because I ran out of memory.

How app or po? Oh, wow. Unified is great for two

things. A, you get more. B, it simplifies the

programming model. That's why we all like unified

memory. It's important. It's the future.

Some would say the past if you're an Apple user.

Yeah. And I think that's one thing that Apple's called out a lot in the M series,

the unified memory between the CPU and GPU and

it's kind of the architecture that is kind of like

moving forward enables a lot of scenarios.

And yeah, so this thing goes anywhere from 24 gigs of

LPDDR 5X unified memory up to 128.

And it also goes up to, it says a removable.

You know, that's the dev box is a removable.

Is it is the surface laptop?

One removable. I think the laptop had screws for the hard drive also. If I was,

I was, I couldn't tell because I had a really

smooth video where the whole machine explodes.

And I couldn't tell what you're allowed to explode

yourself and what you're not. But it looks like

you could unscrew that hard drive from it says

removable says removal, solid state drive, Gen

five SSD, the Gen four for the 512, but Gen

five for the one terabyte and two terabyte.

Yeah, the resolution of the screen is 32 70 by

2180, 262 PPI on it. And three by two ratio,

like all the other service devices, H.R. all that stuff that's on there.

But I think in general, yeah, it's a, it's a,

as you saw, like a super gorgeous device.

And it's a really high end device. I think what,

Pavan was saying was you need device for creators

and the kind of like Harkins to what Apple

originally had like with the pro model of devices. And now they saw the, obviously,

the pro and the studio models, but then they

delineate them with the specific like chips,

like your pros and your max chips that are out

there. And, you know, by the name of this, the

ultra, you know, that is more than what the

surface laptop and the studio would be is like,

they're saying, this is the ultra, this is the top

top to the top end that you can get for creators.

And they showed a lot of different scenarios,

specifically of running a lot of local models.

And we'll talk about local models and training

and things, but a lot of different developer

workflows, creator workflows. There was one

specifically that they showed about an artist,

that does a lot of like different artwork, you know, kind of like mock up artwork of

space and stuff that you and I, you know,

gel with those kind of very like Firefly ask.

Serenity asks, but like that type of that world

is kind of like, you know, illustrations.

And he was running it through all of these different models that once he had drawn,

drew, draw the base of it, then he could run

it through all these models, like locally,

like five different local models, some cloud

models, do a whole do a whole screen, go through

blender and do all this work all locally.

And he was like, things that used to take me like

weeks now take me like a day to do like all on

this device, like with the power of it. And

it's not just these models, but also the, they're

talking about super nemotron. And then also the

MAI code one flash model, which is 120 gig, 120

billion perimeter model running locally on these

devices. And I think that's like super crazy impressive.

I'm still running, having R. Dem running

any models on my, you know, M4 Pro 64 GBS. I can do it.

It's just a struggle. That's all I'm saying.

It's going to be a struggle. I mean, except for MAI,

it's going to be a struggle with everything.

Running a local model sadly is a little more

complicated than it needs to be. But you hit the

nail on the head because you can make the most

beautiful machine in the world, but you need to

have good software to run it. And that's kind of what I was saying about the, the,

CUDA and video world. That's the one nice thing

about that world is there's a lot of AI software

out there tuned to it, but it's not just that.

It's video production software is tuned to it. And video

not only is a large profitable company, but it

turns out they've taken over a lot of the software

ecosystem. So a lot of your high end plugins for

things take advantage of all these Nvidia chips.

We shouldn't forget gaming. This is going to be

a monster gaming device, but I don't know what

gamers are going to spend the way they showed gears

running on it while they were doing all this stuff.

Yeah, definitely more for the creator crowd.

If you're doing like a lot of CAD, if you're doing

movie production, all that stuff, things that can

take advantage of these ridiculous GPUs they're

putting in there. But yeah, building up that

software ecosystem is difficult. So being within

video, they're lucky. They're just inheriting a

bunch of software that's just going to work with it.

But then they made a bunch of enhancements to

Windows also that's going to take advantage of it.

The one that I found kind of interesting was,

and I'm you're going to have to help me here,

because I could never quite remember its name.

But there's machine learning on Windows. It's kind

of the equivalent of core ML on the Apple side

where you can kind of recompile your models to run

nicely on Windows. And I believe like, on X models,

ONNX models, which is an interchange format,

has like native drivers to run officially on

Windows. So it seemed like a big part of this

announcement wasn't just this hardware, but all

these software upgrades to Windows specifically

for being able to run these more advanced scenarios

more easily. Because you could always do it like

I'm saying, you could always get torch up in Python

and get these things cranking. But it wasn't easy.

It wasn't fun. And so I think, and you've had that

experience on Mac. So getting these things to be

easy to run is important. Yeah, and that's what it

was specifically. So there's a big blog post on

the Windows experience blog, which is, is late

to power this. So Windows ML is an AMVIT.

Thank you.

It is a scalable runtime to deploy model across

GPU MP and CPU. And then they announced support

for llama CPP, giving developers more open source choices and also, pie torch.

As well for Windows ML. So pie torch and also

llama CPP, bringing those directly into it,

which is pretty rad. So in general, what that

looks like, they have a diagram on this website is

like your native app or web app. There's like a

custom open source model for on X, pie torch or

GGUF goes into Windows ML and then interlaces with

the Windows Silicon environment, whether it's

Intel, Qualcomm, AMD or NVIDIA. And then

that accesses the GPU, MPU and the CPU too.

Yeah. And let me kind of break this down as like

a practitioner of ML. You're going to use your

pie torches and NVIDIA CUDA stuff to develop

ML stuff. And it doesn't even have to be large

language models. It can be all sorts of things.

It can be the imaging stuff. It can be audio stuff.

It can be video stuff. But that's going to be like your development environment.

llama C++ is very nice for the nerds out there

who are building their own harnesses and

development environments and everything that

are going to be executing these models. Because

a lot of the open source models out there come in llama C++ native formats GGUF.

And so you're just going to need that file

format and something that can execute that file

format. That's why it's a big deal that llama C++

comes out. But as app developers, if you actually

want to deploy the stuff, you really honestly don't

want to use either of those. Because deploying

torch and Python is gross and annoying. llama C++

is just hard to keep up with. It's a moving target.

So that's where the nice Windows ML. It's nice

to have a stable deployment target. And in the

past, it was onyx. But onyx has onyx is kind of

just the file format. So there can be different

engines that execute those onyx models.

So as developers, if you want to add custom ML stuff to

your software, you want to be deploying with

something stable and platform approved. And that's

where like Windows ML comes into play. So it's all

those different areas like from development up to

nerdy stuff or you're willing to get your hands

a little dirty up to this is the clean polished

rosy path for deploying your things. And it says

here, you know, they're contributing back to llama

CPP and in partnership with NVIDIA to your point

here in the broader community. They're delivering

a bunch of performance improvements to llama

CPP and GUF support in the format through CUDA

kernel optimization, kernel fusion improves

CPU, GPU scheduling, weight repacking, and

kutigrafts that work also adds Eagle three, MTP

and D flash two, speculative decoding, multi-GPU

execution, NVF P4, new model architectures and

back end samplings, all sorts of stuff. I don't

know what any of that means, but sounds cool.

I could tell you every single one of those, but all

you need to know is that's the modern stuff invented

to make these things fast. We're just trying to

go fast. That's all we're trying to do. It's easy

to train these things, but making them fast. But

it's funny, that's not actually how the presentation

started. They didn't start with fast. No, what

did they start with? Do you remember? Well,

we started mostly with I think I think Kayla was the

first demo. Was she the first demo or the second

demo? Even earlier, I think it was safety and

security James. It was actually, yeah. I think

Satya said safety was actually the very first thing

that he mentioned. I'm not really with him.

I say throw caution to the wind, let these things

have full access to your computer, give it every

login you have, allow it to use all the memory in

your house. Just put RAM inside the computer and

turn the crank until it has enough RAM. That's

what I think. But who am I to argue with Satya?

What they announced was the Windows container

support. I don't know, was it in a beta or something?

Because they made a big deal. It sounded like

this was almost an official release or something.

Was that what happened? It was in preview a build.

We announced the Microsoft execution containers MXC.

MXC specifically works across Windows, Mac,

and Linux. What it does is it gives process and

session isolation. Obviously, on the different

operating systems, it's going to use different

container systems that are there. Obviously,

on Windows, because it's the stack that they own,

they can go a lot deeper on it. On Windows,

it has a lot deeper process and session isolation

through MXC locally with these one-off process

isolated sessions. It now supports WSLC, virtual

machines, and Windows 365 for agents, which is

really cool. I was demoing this myself the day

before. I'm very in tune with it. Actually,

by demoing it, I'm in love with this more than ever.

It was cool that it was the first thing.

I had been -- this is just sandboxing people.

We heard about sandboxing before in VS code and

the Co-Pilot app and the CLI. MXC now is

rolled out across codex, Co-Pilot, OpenClaw,

Replit, LM Studio, Unscent AI, and a bunch more

that are releasing support now that this thing

is GA, which is super big because the standardized

support gives us developers. It's actually just

an open source library that you can integrate into

your application to provide this functionality,

like everyone else has. Then as users, we get

this really rich features and capabilities.

So you can think of it as you're able to control

disk access, network access, clipboard access,

all these other things that you're able to really get fine green control on.

And what the first thing I did was I'm like,

why ever what I want to give, for example,

the CLI, VS code, or the app access to my

OneDrive, my work OneDrive. So you can just

say block this folder. Just like, you never need

to go to my OneDrive. That's my business stuff,

right? So you're there. Now, a lot of these

sandboxing are really nice. So it'll be like,

hey, I'm trying to access this file. It's in a

sandbox. It's a shik did. Am I allowed to? And

you could break out of the sandbox. If you

need to, there's always those gates just like,

do you approve this thing or not? Now auto approval

on this thing, it needs this human interaction

to do that. And it would like turn it off with

that call and then turn it back on and back and

forth, which is fascinating. But that's up to the

client, that's up to the app developer to enable

that functionality. Now, the real cool part about

it is that most of the apps today are doing process

isolation. So that means for most intense purposes,

and that can be, that could be like a five second

boost up to like they could have a process that's

running that's isolated. So when you boot up

codecs, like every session has its own process that

is then isolated, but you could make it so you

make a request an agent and as soon as that that

turn is done, you kill it, right? You could do

that as an option. Normally, even with the CLI

or VS code has a process that's running that's

sitting there for each of the sessions.

And those are in isolated process there. However,

the thing that's even the cooler than that is

session isolation. And session isolation is cool.

And because Open Claw has it integrated today.

And what session isolation does is it actually

gives a full like user isolation for that application running. So the Open Claw

that is running, the agent that is running in

Open Claw has its own basically user account

and its own user folder that you can peek into,

but it can't poke out to, which means every

process that is spun up in the session of the

app, like the entire app is running as its own

separate user account, which is 100% then isolated

from you. And that's like built into this thing.

And that is like super crazy cool, actually.

I thought that was like really, really neat.

And I've just like really fallen in love with

it, funnily enough, because I was demoing it

nonstop and I was like, oh, crap, this is like

really cool, you know, in general, you know,

and then you can have like a allow list and

denial list and read only, write only the only

things. The only problem is you really got to

set it up. You know, that's the main problem

in general. However, what I'm really thinking

is, okay, let me have some standard defaults.

Such as maybe don't be able to do this stuff in

my one drive folder, your drop box or whatever,

you know what I mean? So just some like default

standards, but you could imagine in the enterprise,

this is like crazy important. And they showed a

bunch of other windows features of like really

deeply integrating these things were really agents

almost really have their own identities on their

on your machine, which is like so cool. Which is

right, you know, Unix invented this in the 70s.

Yes, user accounts are good and they can be used

for other things are than users. That's just a

good isolation mechanism. So it's good to see it

actually extending. Honestly, I have a little bit

of mixed feelings here because I think with any

new security model, there is that time to figure

out what should be the defaults and whatnot.

You know, so every time I open a session in VS code,

it has to ask me permission to like read something

from Microsoft. I'm like, you can you can use

the Microsoft.com website. It's fine.

It feels like no matter how many times I say allow,

a new security model gets introduced and that

allow is turned off or something like that. So,

as a user, I get a little bit frustrated with

there's a little bit of theater with our security

and there's a little bit of real security with

our security. And I understand those differences,

but it can still be frustrating to find that

fine line between them. And I just totally

understand from the enterprise standpoint why

they want it. But some of that stuff coming back

onto real users, it's a bit frustrating. I mean,

talking about Windows Vista, like that we were

the first ones to add the annoying pop up dialogues.

Hey, do you really want to do this? Do you want

to give permission to do this? And it took quite

a few versions to get those dialogues refined.

And I feel like we're still just in that annoying

stage. But hopefully with MCX coming out and

everyone kind of consolidating on it, we'll have

some a better defaults and be just, I can just say,

hey, yeah, here's a whole set of directories on

the computer. Go have fun. I don't care what you

do over there. Exactly. And I do think that this

is going to be ever important as these agents are

sort of like autonomous, they're just running on

machines separately. Like you said, you know,

it's like, yeah, this thing is just running over

there. And it's I'm not even like, I'm not even

know what's going on, right? And you really want it

to be its own separated, isolated process. Yeah,

MCX stuff was really cool. I really like, I really

dug it. I was super excited about it. And just

how easy it was. And now that I've dog deep friends

actually using it a bunch, it's definitely worth

checking out. And there's like definitely some

smart defaults. I think the problem is like if you

have sandboxing and then you have like, there's

a few things, right? So for example, there's,

there's like sandboxing to say, Hey, listen,

I want you to run all my terminal commands in a

sandbox. That is one thing versus a network, disk

access, things like that. Because the problem we

run into as app developers is, Hey, I need to do

a donnet build. Oh, donnet's not installed in the

sandbox. Yeah, right. That's a problem. Whereas

what I'm talking about is more permission based

sandboxing for accessing locally on your machine.

So I do think when we think sandbox is a pretty

loaded term. But what I was at least setting

up as my default for MXC. As far as like,

file and network access, which networks access

for me is pretty much like, yeah, it's fine. But

you could imagine an enterprise, you could just

say, I only want to access like local things. I

don't want you to go out to the internet that

would probably like import for them. But there's

this and one of the cool parts is like, it also like

proxies the access through that. So for example,

I had this demo that was, Hey, can you go get

me a list of like my GitHub issues or whatever.

And it was going to use the GitHub CLI to go do it.

But the GitHub CLI is running inside of the

sandbox with the sandbox networking, which then

blocked it because it can't access GitHub.

Because I turned off network access outside of it

on this project. So kind of interesting in that

regard. So I think it's just fine line of like,

okay, what am I, what things am I doing versus like,

I just want to build a project, like just run it

on my machine because I have all the stuff on my

machine and not in the sandbox. And that's what

kind of is right thing versus agent versus like,

developer build tools, like those two worlds seem to kind of like.

Right. But I feel like we're the first beta

testers of all this agent stuff. Yeah.

All the under other industries, they're still in

the chat world. You know, they're still doing

everything at the chat prompt level. We're the

ones doing the agent and craziness, let it take

over my computer stuff. And that line,

it still is a shaky line. Like just yesterday,

it couldn't build my app. And I'm like,

why can't you build that app? They're like,

dotnet restore is stalling. I'm like, why would dotnet restore? So they're like,

because I don't have network access. I'm like,

oh, okay, why did it take a why to take me three

wise to find out you don't have network access?

And B, I'm like, why don't you just enable network

access? And it's like, great, I will. And then the

Mac dialogue pops up saying, Hey, do you want to

give it network access? Mac OS 27 changed that

dialogue box. So it took me a while to figure out

how that dialogue box works. Good. Finally,

I granted access. And it's like, Hey, you have to

restart the harness in order to do this. And I'm

like, but I'm in the middle of a session. So I'm

just deep breath. That's like the security stuff

is a little bit. So let's just hold them down.

So that's good. We'll work it out.

We'll figure this stuff out. How hard can it be?

Well, the next thing that they talked about beyond

that, right? And also, like at the very end,

we'll talk about the other hardware that I'm more

excited about. So there's the another piece of

hardware. But we'll talk about every single

announcement here as we get into like some deeper

Windows stuff. But one of the things that they

really focus on was sort of like local and hybrid

AI model usage. And one of the things that Kayla

showed off, because on my team, which also

worked on the Windows terminal team, just to be on

the Windows terminal team, is basically frontier

coding models, like local and your machine.

So there's a few things is like on this machine,

what they were talking about is like, Hey, taking

me I code one dot one flash with Nematron models,

also deep seek V4 flash, like that is a 137 billion

per parameter. You know, total 6.8 billion active

and video is a 70 billion parameter, two

bit quantization down to 20 gigs on memory.

Deep seek V4 flash is a 284 billion parameter model.

So there's all these things that are going to

run locally on the RTX spark, which is crazy.

Obviously, I'd imagine, depending on what RAM you

have, you want all the RAM, by the way. And they're

optimized and tunes to basically run on these

machines. Now, of course, you can run those

Lego machines, but you also need great tooling

to run things locally. And this is where the

brand new local model integration in GitHub

co-pilot comes in. And there's two things

specifically. One is that Kayla showed off like

auto local routing and specifically that was

turning into efficiency mode and the model.

The router can route to a local model on disk

automatically. And this efficiency mode has a

little like, a little leaf icon, which is very cute.

But the thing that I think is even more cool

is the hydrophusion support, which you did an

entire podcast on. And hydrophusion can then

decide to delegate out different tasks to

different local models on disk and make decisions

optimizing your token usage across the board.

And this thing is coming to like everything, basically,

out of the box, along with all that sandboxing

things that happen. And Kayla had a really good

demo, which was, hey, like, I've done this repo,

and then I'm doing these investigations. And I

might be using like a frontier model in the cloud.

And then it may decide to then delegate locally,

you know, accordingly for that, those different tasks can run off subsections.

She also showed off, I think she also showed

off automations. And actually, this is actually

really genius. And I was talking to Kayla about

her usage of local models on her machine. And she

said a lot of what she's doing is, yeah, she

does have, if she's like, hey, I'm about to do

something that is going to take a while.

And I don't actually care about, like, I mean, time to

first token is important, like, you know, and then

token, like, coming back. But if she's like, hey,

listen, I know that my local models are never going

to run as fast currently as like, if I'm running

like Luna to go do a thing. However, if I'm like,

hey, I'm about to jump in a call and have a half

an hour and an hour or whatever it is, or overnight

to run long processes, what do I care if it's

running locally on my machine or if it's running in

the cloud, right? Besides that I'm going to save

a bunch of money. And then she said, she actually

has a bunch of automation set up that run every

single night at midnight when her computer

just sitting there with nothing else to do,

and is then crunching the local models to do

a bunch of issue triage bug triage support,

like kind of like these workflows automatically

that can then fire off new sessions or do a

bunch of computation. And she's like, now

that's all just free for me, right? And then,

she also showed, you know, like what she was showing

in the hydrophusion examples, it may start with

a cloud model, local model, go back to clicking,

go back and forth, but it's going to figure out

what it needs to do and what local models you have

readily available, which is like super crazy to

think about. So as she I don't know if she showed

it, but we were really showing is like, you now

could basically for all intents and purposes,

like just boot up something like this, turn on

airplane mode and go, right? So yeah, it's so cool.

Your computer will get warm during that process,

though, I promise. So you can use it as a heater.

You know what it is, it's it's liberating. Maybe

you don't experience this because I think you

have infinite tokens. But for those of us with

limited tokens, I'm always worried about like,

I'm always watching the like, how much usage do I

have left? Am I over my budget for, you know, I,

I'm always like looking at the date, oh, it's the

10th, there's 30 days in the month, so I should be

at 30% and you know, I'm doing this like stupid

math in my head. But when you use a local model,

and even a stupid one, it's very liberating because

you're just like, oh, I have as much as I want,

I can just have this thing chugging and burning

whenever I want. There's there's no little background

clock in the back of my head. Yeah, I mean,

hey, don't do this task because you want to save

your tokens for that other task or things like that.

When you have a local model, you're just like,

I'm gonna use it and I'm gonna, yeah, even the best

models out there, I was just yelling at Astra and

Astra is using up my entire month of credit in

like two hours when I use it fully. And even

then I'm yelling at it because I'm like, yeah,

that's an ugly button, you made your button ugly,

it should be better than that. And I find I yell

at my dumb models and my local models just as much

as I yell at Astra. So I would rather be burning

just electricity here in my home and yelling

at the model than worrying about token counts

that I'm paying for cloud models and all that.

So if you haven't been lucky enough to experience

local models, I don't want to go and say like,

they're just as good as the big cloud ones because

they're not, they definitely aren't. But what they

are is liberating. You just have zero guilt. You're

just like, I'm just going to use this model.

Maybe it's on an airplane. I don't fly that much anymore.

So it's fine. I don't care about the airplane

scenario. But what I care about is just not

worrying, not thinking about my quotas and any of that stuff.

We're sitting there doing these demonstrations and

when a demo or was was booting up one of these

models and just like let it rip for like 30

minutes and after 30 minutes, I had like a full

video editor. And I was just like, cool, here it is.

And there was this whole thing was an airplane.

But I was like, really kind of crazy. And like

you said, very liberating and like, you know,

token used to zero. I think you need to,

it's a blend, right? You need to use the right

model at the right time. And then, you know,

especially with those like long running processes,

like I had a session that ran for 100 hours

fully, you know, I was cranking and testing some

things out and just go into town, right? And this

is like a blend between frontier models, local

models, all these other things. I'm bopping back

and forth between them. And I think like that's

a really unique scenario where I was just like

set in, forget it. You know, I'm going to be in

San Francisco, just like, I'm not even, I'm not

even having to have this computer, my computer is

at home. What am I doing? I'm not even checking

in on it. I was like autopilot, Yolo, I'll come

back on it. It's sandbox. It's good to go. And I

think that's like a very intriguing scenario.

I mean, the other thing that they also announced,

but actually didn't like show, I mean, they showed,

but they actually have these like AI super

computer, like tower things like that. I don't know.

I don't know if it's, I don't know if it's

not to call them like a token factory or something.

I forgot what he said, something like that.

Well, basically there's these huge like basically

servers that, you know, kind of server towers,

you know, DGX stations, which are these huge

stations that they're talking about running

windows now. And these are apparently using a GB

300 grace Blackwell ultra chip DGX workstation.

It says it can unlock power to run frontier models from Loma for Maverick,

Kimmy, K to six deep seek for V for pro and more

than a trillion parameters on these things.

Oh my god. That is a great are the professional

grade model runners. Maybe someday I'll get a

10 year old one that would be nice. I think

there was a cute little afterwards during the

presentation where they were just reminiscing about

like the computer industry. And they're like,

do you remember when chat GPT was trained on a

DGX one? Do you remember that DGX one? I'm like,

guys, that was like three years ago. It's not

that long ago. Like, I would still kill for that.

And they're acting like it's all old hardware.

They're like, Oh, we just have those. We put those

out the pasture, shot them and left them for the

carry in. Like guys, I'll take your old DGX.

Just Satya, Frank will take your old DGX. It's cool.

It's cool. Yeah. Well, there's a few other

things in like windows, windows itself.

So there's a lot of things around performance,

search optimization, making search like the most

amazing thing in the entire world. I think they

really focus a lot on here the improvements that

were making to core windows, how we're making a

sort of an agent first operating system.

And I thought that was really cool how like agents are

deeply integrated. And also I think how they're

evolving co-pilot. And specifically around this,

whole idea that you have this home code and

autopilot and home is really like, I am on my

machine. I have the context that is my in this case,

like I'm in now consumer mode, right? Like,

I want to write documents. I want to do stuff.

And I want to be safe. And I want to not just be,

you know, you know, yellow on it out to wherever

context, but basically you can, you know,

co-pilot can understand relevant content on the PC,

including files, recent activity, local context.

And the cool part about accessing these is it

can also tap into local actions. So it can take

actions on your behalf. So it can organize

files, diagnose device, troubleshoot issues,

it can obviously code, it can carry out workloads,

it basically do anything you want. And I'm thinking

like this kind of like, what's the next general

of like shortcut type of thing, actions. But then

it all goes so can tap into local models too,

running on your device. So when it's accessing

these things, it's able to access the same models

that you might be using for coding or these other

things actually do this work, which is really

cool. So they're really talking about this

hybrid intelligent MXC support built in.

And then the idea that you're going to have these raw

long running personal agents too. And to your

point of like, what are people doing? Like,

actually, the thing that I'm doing at work often is,

you know, inside of co-pilot, like a Microsoft

co-pilot is I am for all intents and purposes,

having it gather a bunch of my work intelligence.

Like I do this is a thing that I do. Like I'm

planning right now a bunch of our co-pilot,

a get up co-pilot like advocacy efforts, right?

What are we doing for the rest of H1 and in H2?

And what I needed to do is I needed to tap into

my emails, to my messages, into my files on disk,

and into these conversations. And I needed to tap

into other planning docs, coerce things together,

look at latest news on the internet, pull things in,

and then take this and then transform it into

docs and reviews and these other things, and then

messages and send things and do follow-ups and

then that's the thing that I'm actually doing at

work, which I wasn't really doing all that much,

like six to 10 months ago. But I've been trying to

do a lot more of those processes as these tools

gotten better. And I actually find myself doing

this too often is in the co-pilot app, what I do

end up doing today is I'm doing a lot of

house projects, right? And I'm doing a lot of

contractor reviews and all these other things.

And right now I'm dragging and dropping files

into a thing and doing like, I would love to just

be able to do a lot of that integration into

Excel and into these other things in this workspace.

So I'm really excited about how that's evolving

and how these, you know, agent harnesses are

running. I'm not sure if I'm there yet. Like,

I'm not an open claw Hermes person at this point.

Like, I use them in some regard, but I'm starting to,

I think once you start to see how other people

work that I'm interested, like, I'm not really

interested in giving meta all my data.

Like everyone's like really excited for a muse. I have zero

interest in giving meta all of my data, right?

Like, I'm not interested in that at all. I am

really interested in local models being able to

do computation over myself, which I think of

Apple intelligence, right? Those models that are

running local and your device like, you know,

Apple, when the local model trend is there and

you know, they've been trying to evolve that in

the latest devices and the leading operating

systems. And I do like this idea of hybrid. I

think this is, I don't know if I don't, I've

been tried Apple intelligence and in Mac OS 27

yet on my machine, which I have here. But from

my experience of what I saw when I saw people

trying to use Siri was it would ask you to question

and say, Oh, I can't do this right here. I'm going

to go to the cloud. That's almost hybrid, right?

Except for you're doing these actions. In this

case, it's automatic hybrid. It's going able

to be able to do these smart things locally,

remotely and configure this for you. Now, I think

that the biggest thing that Apple got right,

then I'll be interested in this experience is

the seamless one button click, turn it on.

Like, I think that's really important.

Like inside of Xcode, inside of your Mac OS machine,

inside of your iPhone, you say, do I want to use Apple intelligence and you say,

yes. And then that's all you don't have to worry

about what model am I using? What am I doing?

XYZ? That's what I would like to see from this experience is like, if you want to go

bits and bop and pick your models, go for it.

But we're going to give you a bunch of defined

models that are just really good at doing certain

things that you're going to be doing locally

anyways, like doing text or doing image stuff or doing like profile processing.

Yeah, absolutely. And you're really hitting the

nail on the head. And I got to say, I don't think

either operating system is quite there yet

because ideally in the future, in the same way,

I can pick which search engine I want to use in my

browser, I can just say use Google or whatever,

use Bing. I would like to be able to under Syrian

Apple intelligence or Syria or under Windows ML

just say like, here's a model I downloaded from

hugging face use that for everything. That's the

model I want you to use. I want you to use it locally.

Don't log it to anything. This is the model.

But sadly, right now we're still in the case

where Apple releases one model. It's the model

that comes with the operating system. The model

that comes with Mac OS 27 iOS 27 is different than

the model that came with 26. And I feel like

Windows is going to kind of be there right now too.

Plus Windows is in a really good position because

I got to tell you Apple's dropped the ball a bit

still with Siri. I've been using it for a while.

I am not amazed. I was arguing with it all afternoon

where I would keep asking a questions.

And I wish it would say the thing like you said of,

it'll say, hey, I'm going to go use Chetchi PT or

whatever right now. No, all I kept saying was,

I didn't quite catch that. Do you want to try again?

Like, no, you obviously got the text. I can see

it. You printed the text right there. You got it.

You're just not acting on it. So for whatever

reason. And it doesn't make any sense to me because

I know I use the local model. It's like it can

do this stuff. So I don't know. There's there's

Apple's dropping the ball. I think this is a perfect

opportunity for I'm in Windows to catch up but

Windows to get ahead quite a bit because the demo

they had for the start button and search like

they were saying it's faster than you can type.

Now I know some fast type is out there.

So I'm curious how just how fast it is but like the

demo looks great. I know it's easy to make a demo

look good but you know time will tell when people

get this installed on their machines but I think

they have like Apple should have the lead in the

same way Google should have the lead in AI.

All these people who should have the lead don't.

So this is a great opportunity for Microsoft to

like really make some headroads. And I know they've

gotten a lot of flak like co-pilot this and

that and the brand is getting a little bit

overwhelmed and trying to put AI and everything

but like the Windows search box is a perfect place

to impress people. If you start impressing people

on that box you're going to get a lot of converts

and like they're just going to get used to it.

You're going to set a new bar, a new standard for

what your computer should be able to do and help

someday Mac will catch up to them at least to that

demo. Well and I think if the users are starting

to get used to this aspect like when you Google or

you Bing or you like sort something into some

box and you start to get these AI summaries and

you start to follow up with AI you start to get used

to this and you almost want that as the default

experience. So it's almost like looking forward

okay or people are people there now right. That's

the problem is you try too early or your first

one of the first there you know it's going to

take a lot of revs to get it right and you're

going to maybe make a lot of people mad and

you're going to make some people happy and then

you know maybe you all mock it and hopefully so.

But let's talk about the last thing. There's

plenty more things that we could cover but

a thing that I am most excited about in

this moment is not a laptop. Frankly. Okay.

It is a PlayStation 2. No it is a brand new it is

a brand new it's a brand it kind of looks like

a hybrid between like an Xbox and a PlayStation

2 like slim which is the Surface RTX Spark Dev

box and I'm pretty sure they showed this app build.

I think I'm pretty sure it was there and if not

well something was there but okay right I think it

was there but this for all intents and purposes

is very similar to the ultra however there's one

configuration you get one you get one that's it

one price one configuration and this really

is I believe it has a default out of the box

Windows 11 experience for developers like it is

it is a special image that is made for developers

I don't know what that means but they said

it on there from like what it's supposed to

I'm pretty sure it's what it is but it is a

the same 6144 core GPU 20 core CPU the biggest

it is 128 gigs of LPDDR5X Unified Memory and it

is the biggest SSD the two terabyte which is

removable by the way in this puppy and this thing

looks just gorgeous it is a big ish not super big

but big ask basically dust on PC and it starts at

6000 lives yeah 559 yeah add us tax to that you

owe the dollars I've held it it's beautiful

it's fantastic and yeah it's it's what I want

I think like I know it doesn't have a screen

Frank but also our Mac mini zone have a screen and

I I tweeted after the event that it is official

that my Mac mini is officially for sale so I am

yeah traitor I'm sure I'm done you know it's bigger

it's bigger though it's not many I want to call

it many and so it looks like it's about the size

of the laptop have you held one have you seen it

in person oh I have yeah the I have held it in my

hand the question is are the are the technical

specification right sure sure yeah don't worry

okay yeah they well they have it in photographs they

have it next to the laptop and it looks roughly

the size of the laptop it looks like a giant

pancake it's it's definitely what I appreciate

is they're not ripping off like the Mac studio

design or anything this is a pretty unique design

it looks more like a game console than anything

else your PlayStation 2 reference is actually

pretty pretty spot on I would say my biggest

question is where are the fans on this thing

because again I use my GPU as a heater I need

fans on it to distribute the heat throughout the

whole room so that while I'm training neural networks

I can also be heated this winter that's a very

important feature so I'm curious how this thing uh

thermally conducts it's thermal yeah I was just

because um it's pretty thin you know it's it's kind

of like a two-story um it's got a smaller bottom

than a in a bigger top it's a little mushroom

shake and it was a little bit confusing like where

they're putting all the fans and everything on it

it's gorgeous I love it um it reminds me of like

the old sun microsystems boxes I don't know if

you remember any of those from the early internet

days okay no never mind um my biggest complaint

because I was telling you before we started recording

is for roughly the same price it's almost cost as

much as the same configuration as the laptop but

you're not getting the screen you're not getting

a keyboard you're not getting a touchpad uh what

you are getting is about 25 more watts of power

I should say that the laptop is 140 watt power

supply and this puppy is 165 watt power supply I

still think that that's a little bit on the low

end from my personal taste because my 3090 right

now is literally taking up 350 watts of power so

it's already running double the amount of power

um that this thing can produce now obviously this

is a more modern chip it's gonna do better it's a

it's more efficient chip than my guy but you're

still stuck within that power envelope so it's good

to know that um with the laptop you're in the 140

watt power envelope with this guy you're in the

165 watt power envelope and these numbers matter

like physics matters you've got to shove electricity

into these things in order to make them go fast

um so it's it's pretty cool to have this box out

I don't know who's gonna buy one um I hope that

you're able to make a lot of money on your Mac mini

I'm gonna be so sad that you're not gonna be my

Mac mini brother anymore I'm just gonna be out here

Mac miniing by myself but I hope you're able to

get at least 10 grand for your Mac mini so you

can buy two of these things I did look I did look

and I did see that I should be able to make a

few hundred dollars off of you really yes okay I

don't know did you see did Miguel say he's gonna

get the six pack of these you know I don't know I

didn't I didn't see I should a six pack of these

uh dev workstations that's gonna be pretty awesome

yeah I'm I'm pretty jagged I can't I can't argue

it's just a lot of money it's a lot of money I'm uh

I think you know for me obviously we'll keep my

Mac book air because I do iOS development I need

a Mac around like you cannot get around it and

the m1 still it's still great device I don't know

I know it's still the rumored like OLED um Apple

laptops coming out in a couple weeks oh which by

the way a correction for last week so the current

rumor is they are touchscreen by the way so oh I

don't know did they say whether the new ultra is

a touchscreen or the ultra is a touchscreen yeah

yeah I figured as much because Microsoft laptops

have they're all they're all the touch sense yeah

ultra whatever it is so um I think I said on the

podcast that it wasn't going to be a touchscreen

Mac and it hasn't been announced so this is not

official but the current rumor mill is it is

actually a touchscreen Mac so I was like maybe it

is time for me to upgrade but then like look at

my cute little m1 I'm like you're such a trooper

little m1 I don't think I could ever sell you or

get rid of you like if I ever drop you down

a six-story building or something like that

accidentally then maybe I will upgrade and fight

you're right yeah it's hard to get rid of the

thing it's been so much value for the buck yeah

I'm I'm super jazzed on it this my current plan

and you know what I really do like my my my Mac

mini and it's really great I just actually don't

even know if I've taken as much advantage of it

as I as I should then no no I know if I will take

advantage of the the dev box or not but we'll see

at least and I'm excited to go through this journey

of where I'm at so the the interesting part is

just like I went through tons of transitions

with Thunderbolt 5 and this and that okay

well I go through all this again with all my

crazy stuff I will say this so I'm also reducing

my plan is to reduce my ultra wide I'm getting

rid of my ultra wide okay I'm convinced by John

Galloway that this $200 Dell 4k monitor that he

got on big day deals was what I should get because

he said it was good and I said I don't have Amazon

Prime so you're gonna need to buy it for me wink

free free monitor um so he said he said so so it's

it's come into the house which is good which is

nice and getting rid of that I want to like redo

my setup yet again obviously but um it just it's

really tricky okay I don't I have never really

felt like like I think when I was building my

custom built tower desktop PCs because I was

designing what it was I was understanding the ports

I was preparing in it like it was perfect I don't

know if I have rather perig monitor setup but I

do feel when I had two dual screens it like felt

correct like a dual screen monitor setup it felt

correct for a while but I do that yeah but I do

feel like I really want to trim it down that being

said Frank Ruger I could be convinced into an

ultra because it has the other monitor right I

would have another monitor that's there that I

could use and it coordinates with this new 27 inch

4k monitor side by side like I think about it you

know what I mean yeah I don't have to get a stand

for it I gotta do something I just think that that

that box will look so sleek just sitting there

we'll see I don't know who knows what I'll do

in general but that's right at least where I'm at

right now and I'm real excited I feel like you

know I've always been a Windows person and when

the Mac mini came out the size of it the power of

it spoke to me and so does this that box it really

speaks to me as a as a aesthetics developer mini

PC that I want and I think I'm gonna get it so

we'll see oh boy well I know I'm gonna send you a

bunch of models to try on it because I'm gonna be

curious how fast it is so I'm gonna make you test

the few things some remote desktop access into

your own private account that's something nice

make it into a proper workstation server yeah

oh we didn't mention the most important part of

the laptop before we go I just have to say don't

call it make safe but they added make safe to the

laptop it is a wild weirdest little USB-C connector

you've ever seen in your life and no one believes

it works but I've seen video online of people

fake tripping over it it seems to work fine

come on apple catch up because it's pretty

cool it's a pretty cool design they did there

with the USB-C because it's USB-C and it's make

safe but it's not safe copyright trademark

apple it's a it's the world's first magnetic

USB-C I have one right here yeah boom there it is

and it's over yet yeah have you proven it I have

shipped over it many times and it does work it

does work yes and it snaps right in it's it's very

very nice so nice yeah it's absolutely delightful

all right that's gonna do a people for this

week's emerge conflict go spend all your money

asking me service devices I am super stoked

super good but that's gonna do for this one so

until next time jay's on to magnow and i'm frank

rueger thanks for watching and listening peace

Download Transcript

Episode Comments