Intel Mac Pros max out at 1.5TB of RAM, and these are max 192GB. Seems like a large drop. I've heard that Apple Silicon is more memory-efficient, maybe because of efficiencies from the tight integration with macOS. Is that enough to make up for the difference?
Apple Silicon isn't more efficient in terms of memory capacity, there's really no way to do that unless you go for some crazy memory architecture that nobody wants to program for.
Apple Silicon has a lot more memory bandwidth though, maybe that's what you heard?
I think the parent comment meant there isn't a way to make memory more efficient when it comes to app data. And code takes up so much less memory that x86 vs ARM code size isn't going to make a measurable difference.
In any case, 1.5TB is only going to help a few app niches. It is hard to even reliably predict which kinds. Even, say, editing 8K video could be well-enough accommodated with much less memory if file system bandwidth is high enough.
Unlikely. 1.5TB of RAM is the kind of crazy amount that you'd use if you were handling some immense workload and you need to hold a huge amount of data in memory. You couldn't just cut the amount of available RAM down to a fraction and expect things to continue to work.
One thing I was surprised to not see any comments from Apple on is whether GPUs will still be supported in the PCIe slots. It'd require work from either AMD or Nvidia to get an ARM64 macOS driver so maybe that's the holdup there. It's hard to compete with max builds which have many GPUs in raw power without the ability to go beyond the built in GPU. That said, 192 GB of 800 GB/s VRAM is pretty crazy at the price point.
This looks like a product meant to serve a very niche market.
During lockdown I had the Intel Mac pro in my home office and my electric bill doubled with that thing on. It also pushed out so much heat, but at least the chassis design was whisper quiet. I don't know, by the end of it all, I was so happy to get that heavy thing out of my house.
Sure, it was definitely the fastest thing to enable the tools I needed back in 2020 and I was extremely happy to have it. It also became obsolete once the M1 had native tooling ready a couple of years later. Soooo yea.
I guess with all that excessive space in that chassis they can overclock the system and squeeze out every bit of performance. The PCI expansion could be good for production studios that need it. At that cost level though, the system will have to outperform anything you can find on the windows side.
> This looks like a product meant to serve a very niche market.
The Mac Pro always was a niche-market device for professionals with big compute needs in a workstation and money to burn.
> At that cost level though, the system will have to outperform anything you can find on the windows side.
You can count on that. It's Apple. Not only is the M2 designed to outperform any other desktop-class chip out there, but they've also bought out almost all of TSMC's entire 3nm process capacity, shutting Intel, AMD, and other ARM vendors out of the highest tier of CPU performance entirely.
I don't know what kind of work you do, but in some ways it is a niche market. It's a professional workstation that is usually meant for compute intensive tasks like scientific computing, 3d rendering and animation, etc. It's certainly overkill for most software engineers, for example.
Is it overkill? I use all 32 of my CPU cores when building C++ or Go, and while fuzz testing. I'd take 64 if I could get them. There is no reason why I'd ever want fewer cores, even if it doesn't make Chrome or Emacs any faster. Latency is crucial during the edit/compile/debug cycle; making compile as close to 0 as possible is always a productivity win.
Finally if you think Intel power saving is bad, wait until you see AMD's. My Threadripper system idles at 270W. This is exceedingly wasteful. Before upgrading to a Threadripper I had an i7-6950X which idled under 100W. The difference is quite noticeable.
I too subscribe to the Tim "The Toolman" Taylor ideology of more power. My only point was that most developers don't "need" a Mac Pro, even if they neeeeed it.
Indeed, I read "macbook". Still, going from 30W average over 40 hours a week to 60W continuous[1] doesn't change the calculus that much, and if anything a 3Y amortization is probably too short for a workstation. Really my point was that this was a bit too far down the hyperbole hole. There are reasons to like new efficient hardware, but "electricity bills" aren't one of them.
[1] Measured average consumption of my 64-core Zen 2 box, which is surely going to be more than that Mac.
It does feel a little like a product they made because people kept asking for it... As to who actually needs it, I have no idea.
I was hoping it would have expandable memory. Not because I need the Mac Pro, or insane amounts of memory, I just wanted to see how they'd deal with it.
Technically I suppose Nvidia could write the drivers, not sure if they can justify the cost though. Seems like most people would just get a Linux box.
I wonder how much of the mainboard is essentially blank? (Just a plane or essentially traceless) With unified memory, that certainly simplifies the “desktop” format considerably.
I'm a bit naive when it comes to how interconnects like PCI work, but other than OS level drivers being available, is there any technical reason why CPU architecture could restrict PCI device support?
That is to say, if the drivers were signed and supported on macOS, could you theoretically take any modern PCIe device from a PC and run it in an ARM-based PC, Mac, whatever?
There's also the I/O ports (PCI has three address spaces: configuration, memory, and I/O, with the I/O space being a legacy x86 thing which does not exist on ARM and most other CPU architectures), which some cards might require; and the PCI address window (BAR space), which might be too small for some cards (usually GPUs; see for instance https://www.jeffgeerling.com/blog/2020/external-gpus-and-ras... for a couple of examples where it was too small.
I did not expect them to recycle the old Mac Pro enclosure. I get it, if it works it works, it just looks hilariously empty and unwieldy compared with the Mac Studio that could house the same chip (Yes I know the expansion slots need room).
I’d assume you can add any expansion card with drivers for Apple Silicon macos. Now that this config exists I guess some companies will make the necessary drivers for their products.
I think CXL devices in general will become more and more common given that other players on the hardware side are now supporting CXL 1.1 / 2.0 protocols (Sapphire Rapids / 4th Gen EPYC[2]) and it's a way to scale memory past the limitations of a single machine and the number of memory slots / DIMM slots.
For now it's PCIe 5.0, but given that PCIe 6.0[2] will come around with PAM4 signalling, FEC and FLIT mode on CXL 3.0 spec it may be not be as bad or slow with this future configuration.
Given that macOS has memory compression, it already has two "classes" of memory pages in a storage-hierarchy relationship, with active promotion/eviction. Just mount the CXL "external memory" to a different part of the address space, and default to writing compressed/cold pages there.
> So Apple wasn't able to increase performance (or even ram capacity) beyond the Ultra series chips. Thats disappointing.
Did we watch the same presentation? The M1 Ultra maxed out at 128GB of RAM, the M2 Ultra can handle 192GB. The CPU cores go from 20 to 24. GPU cores go from 64 to 76.
These are minor improvements over the M1 series in line with the other released M2 chips. For a 1 year delay to the original transition timeline and skipping the first generation it's reasonable to expect something more than the high end Ultra chip in a new form factor.
How about a 4x, 8x and 16x M2 Max chiplet design which would have truly been next level...
I never mentioned a comparison with M1 Ultra. I was referring to a step above the Ultra series. From the moment the M2 chip was announced with increased RAM, CPU and graphics cores it was obvious all these benefits would come to Pro/Max and Ultra series.
The point is that Ultra remains the top of the line chip. There's no Mac Pro exclusive chip that lives on another level of performance.
This is true, but I also don’t see why it matters. In practice most users who want the highest performance level will choose the Mac Studio form factor, which is far more convenient.
The Pro chassis is an alternative enclosure for use cases that require PCI expansion cards for niche applications.
In the pre Apple Silicon days the highest performance was always exclusive to the Mac Pro.
It’s disappointing that Apple wasn’t able to make something better than an Ultra chip and the RAM limit for the new Mac Pro has decreased and is no different to the studio.
I think what the parent meant is that, once Apple decided that the Mac Pro should have the M2 chips, they must have targetted feature parity with the intel versions, aka in the amount of RAM. But presumably, they couldn't get there despite the extra year.
Its true that Hector Martin said they didn't have the ability to add another interconnect like they did to double the Max to get Ultra.
But that ignores the entirely different chip design T6500 which was assumed to be for the new Mac Pro. No technical details of it ever leaked but its reasonable to assume it would offer higher performance than the T6002 chips which became M1 Max/Ultra
> But that ignores the entirely different chip design T6500 which was assumed to be for the new Mac Pro.
The thing about leaks is that they aren't confirmed and might, literally, not exist. In which case, you might be upset at Apple for not releasing a chip that they literally never made. Anyone can make up a fake chip number, create a Chinese account, and make a fake leak for clicks or the lulz, or reach out to a legitimate leaker and mislead them.
But let's say it did exist. You are assuming that it can be mass-produced, even in small quantities, without immense waste or excessive resources, or at a R&D cost or tooling cost so large that it will never break even. Some things are just experiments.
OK but I asked "Is this now the best hardware (or at least pareto frontier) for local LLMs?" and it sounds like you're mostly talking about software issues.
Hmmm... theoretically maybe? You can't argue with 192GB hooked up to a single GPU, even if the die is split. I'm not sure if it behaves more like a single GPU or 2 NVLink GPUs when running or training llms.
But again, at that price point and with theoretical software support, you are starting to look at AMD MI300s, Intel GPU max/gaudi boxes, 4x 4090 boxes and such, so I am not sure.
If you need RAM more than you need compute maybe? The M1 Ultra was much slower than the 3090, and by all accounts the 4090 made a significantly bigger step forward in performance over the 3090 than the M2 Ultra does over the M1 Ultra.
If the M2 Ultra can double the performance of an M2 Max it will be on par with the 4070ti in compute, or slightly ahead of a 3090. But we'll have to wait for benchmarks to see.
> Mac Pro with M2 Ultra features a 24-core CPU, up to 76-core GPU and 192 GB RAM. It also features two HDMI ports, dual 10-gigabit Ethernet, and a 32-core Neural Engine for machine learning tasks.
