The end of the AArch64 desktop experiment
45 points by Vaelatern
45 points by Vaelatern
I'm a bit sympathetic to this. On my Raptor Talos II, IBM keeps breaking PowerNV. First it was amdgpu, now it's the SATA card. I've been stuck on a 6.14 kernel because they keep faffing around with the PCIe for their non-baremetal systems.
What Linux distribution are you running? We have work servers running Ubuntu 26.04 with 7.0 kernels that have been well supported.
I've been wanting one since they came out but now they are quite old and maybe there will be a Power 11 version.
Fedora. Right now it's F44 but with an F42 kernel, or it locks up hard. My contact at Red Hat is going to get me some bleeding edge ones to try.
I also suspect you may not be running bare metal (only PowerNV has gotten bustage). Are those LPARs?
I know 6.18 works as an LPAR on PowerVM on P8, but I did run into an issue with a bogus patch to PAPR drivers. That did get fixed by Gentoo anyways.
It's a rackmount version of the Raptor Talos II, using PowerNV / OPAL firmware. No LPARs from what I can see in /proc/cpuinfo.
This has been my experience as well - I've been trying to run NixOS on a Thinkpad X13S. It works well enough (though I had to use an Ubuntu ISO to install Nix because there aren't any working bootable aarch64 UEFI NixOS images I could find), but the latest kernel upgrade broke my ability to boot and I've run out of energy to spend on just getting my system to work.
Even on Ubuntu, which has some level of support for the X13S built-in, quite a few things which would be taken for granted on a more traditional x86_64 machine just simply don't work. Suspend isn't available at all, TPM support is limited, graphics are quirky, and I'm sure there are many others I just didn't get around to testing.
And that's not even counting the many arm devices which only have an out of date image provided (like many emulation handhelds by Anbernic or other companies) or require non-standard kernel patches because of clever uses/abuses of hardware (I'm looking at you Clockwork uConsole). Both of those end up with software that never gets upstreamed and leads to hardware with an OS that can't be updated.
I want arm to work well on Linux, and I've spent quite a bit of time attempting it on multiple computers, but I'm a bit stuck. Nothing I've tried (outside the Raspberry Pi) just works and I don't know enough of the hardware/kernel side to make any meaningful difference here.
I got a X13S to use (the size and weight seemed perfect). I did the same and spent hours trying to get NixOS installed. Did the same installed from Ubuntu and sorta got it working but it was so fragile it really wasn't working.
I really really wanted it to work, but it seemed abandoned by Linux folks.
Actually just ended up sucking up running nixos in a VM on a Apple Macbook Air.
I've got the X13s too; the only distro I've tried is Fedora and that has I/O hangs when you start the installer. Not ideal! I haven't bothered trying another distro (not terribly passionate abou Ubuntu), since Windows at least works OK enough.
The newer SoCs have a lot less quirks (i.e. you don't have to type a paragraph worth of kernel cmdline), but they don't make an X Elite 2 version of the X13s.
I wonder if the new Nvidia RTX Spark laptops are going to get official Linux support. After all, they're basically the same platform as the DGX Spark which runs a Ubuntu derivative. So far the signs don't look promising.
I'm going to give the opposite experience... been running a Radxa Rock5bPlus (16gigs, with nvme), using u-boot (mainline) and EFI version of fedora rawhide (mainline kernel) for about 4 months. The work collabora has done on getting rk3588 support mainline, is basically incredible.
There are a few bits I'm still hopefully waiting for, HDMI 2.0, or more (for 4k@60), and USBC DP, but honestly, hardware wise, its probably got less quirks than my XPS 13 9370, where audio out just stopped working round about 5.14.
I'm owning a OrangePi 5 Plus, and i've just seen that HDMI capturing is now merged.
Still without HDCP support, but i guess it's time to return to an old experiment of mine:
Low latency 1080p HDMI OSD
I had it running with 3 frames latency (theoretical minimum is 2), and overlaying a Chromium Embedded over the HDMI signal, which gave me OSD superpowers in my home media setup.
Biggest issue was instability, the whole thing crashed the OrangePi kernel regularly.
Could you elaborate? Sounds intriguing.
Usage:
the technical solution was an application which:
This gave me the ability to basically overlay arbitrary software over my HMDI signal.
I had two issues back then:
In theory, you're nowadays able to craft your own HDCP device keys, at least for HDCP 1, which would've allowed me to also pass through protected content. The issue is: It's a legal issue.
Because i could've also streamed a BlueRay to ffmpeg with this setup easily.
Fun fact:
I've found an HDMI splitter which "passes" the HDMI signal by just removing HDCP protection rofl.
Wait, you're just using it to build an HDMI switch UI? Interesting.
(I'm obsessed with how much using TVs suck nowadays. My main obsession is plugging something powerful enough into a TV so its UI doesn't suck, but I hadn't though about what you're doing.)
This seems to be more of an indictment of the current state of Linux, where only hardware that's popular and profitable gets to have support in the kernel, and the current state of binary drivers, which has always been and continues to be hell.
It's interesting to see how people chase Linux to get something working on their hardware, then end up with old kernels or end up needing to maintain and rebase patches themselves, instead of running an OS that supports older hardware without the need for this.
To me it sounds like they're still trying to figure out how to get this faulty hardware to work well with AMD GPUs:
Per Altra family erratum, PCIE_65 may cause invalid addresses to be generated on PCIe mmio writes, impacting certain device types, notably AMD GPUs, and thus the Altra family is not generally compatible with those device types. We have provided a proof-of-concept software patch under the GPL v2 that will allow experimental and development work to proceed.
I can see why an OS might not want to accept patches that just PoC.
It's interesting to see how people chase Linux to get something working on their hardware, then end up with old kernels or end up needing to maintain and rebase patches themselves, instead of running an OS that supports older hardware without the need for this.
There are lots of Linux kernel forks with support for some particular bit of hardware, and this is a shame. These forks have often have invasive or experimental commits, that would also require more work to be accepted by the mainline Linux kernel.
Are there other OSs that have a different story here? What do they do to make upstream contributions easier, while making maintenance of architecture/SoCs/boards manageable?
Are there other OSs that have a different story here? What do they do to make upstream contributions easier, while making maintenance of architecture/SoCs/boards manageable?
NetBSD does a pretty good job of maintaining support for old, odd or problematic hardware while not allowing or minimally allowing that support to impact newer hardware. For instance, the NetBSD Wii U port has some clever tricks to allow full use of all three processor cores without the coherency issues that Linux suffers. NetBSD has support for old earmv4 (hpcarm) devices, StrongARM, earmv5 (like PogoPlug, SheevaPlug), et cetera, and none of that keeps support for earmv6, v7, +hf, +eb, or aarch64, +eb, held up.
But support for those old processors and machines doesn't just rot and go away. Sure, as NetBSD moves to newer versions of gcc and llvm, code will need to be updated, but the amount of work that requires is vastly overstated by the corporate shills at other open source projects.
Interesting!
I'm setting up a Netgear RN102 (Marvell Armada 370; armv7) and have some context on the Linux patches that a community fork maintains to get decent network throughout, and some of the challenges with that fitting in mainline Linux.
I see NetBSD has Armada 370 support. I will install it, and see how it fares.
I prefer NixOS on my machines, but for a 32-bit ARMv7 remote NAS that just serves NFS, I'm keen to try NetBSD.
Ah well. That saves me the trouble of trying it, which I was planning to do soon... Hopefully identifying the pain points will help in the long run.
I thought I was the only one suffering! I had similar spec for a development server. Most of the issues come from Python dependencies with native code (I remember some optimization packages and even Matplotlib). I tried all regular pip and uv but ended up neeed to compile them from sources. I switched back to x86 for good. And to be frank, the performance wasn’t that great even with that many cores.
I tried all regular pip and uv but ended up neeed to compile them from sources.
Do you mean in the sense of needing to go outside of pip and build the packages by hand?
No, this is a compilation step within pip/uv. It is not usually a blocker except time wasted. But, I do occasional run into compilation issues requiring installing other OS level dev dependencies that are painful to resolve.
Yeah if you want a Linux desktop that actually works for gaming you need an Nvidia card with the binary drivers, and you need to avoid things like Flatpak that don't know how to play nice with that. That's been the case for decades on any architecture, to the point that I'd consider it common sense.
I have an AMD GPU and gaming works just fine on Linux. Add to that that the general hickups are less than with my previous nvidia and their stupid closed driver blob.
I will second this counterexample. The open-source amdgpu drivers have treated me great, whether using flatpak, steam proton, or native games. Examples: Hytale, Subnautica 2, Sins of a Solar Empire 2, Helldivers.
We can have better than fucking NVidia.
AMD has been the preferred GPU for Linux gaming for quite a few years unless you absolutely need Nvidia for something else like Cuda.
What are you on about? AMD GPUs work just fine for gaming, and so does e.g. Steam inside Flatpak. Sure, in the case of Steam the Flatpak won't give you access to your Steam controller but beyond that it works just fine.
If you're going to make such claims at least provide some data to back it up, other than "trust me bro".
It used to be true, that nvidia with proprietary drivers was just better. AMD caught up though.
I’ve actually had better experiences with my AMD based Steam Deck, a AMD APU (5750GE, so 8 cores of zen 3 + vega 8 GPU) based mini PC, and an Intel arc b580 in recent years than I have had with an NVIDIA 3090 in the same timeframe.
In such a kernel-patch sensitive setup I wouldn't bother with distribution's kernel package at all. Just build and boot my own kernel outside the package system, update it at my own pace.
Oh wow, I had followed this thread of posts a bit and I was surprised it worked for so long, always sounded a bit "trying to make it work". Sad to read it though.
Normally I would start to bisect the kernel to find out where the problem is. But I was running a tainted kernel due to PCIE65 patches so who knew where the problem actually was… Let’s get Nvidia
If you call yourself an Arm developer that shouldn't've been an obstacle for debugging ;)
I run my dev branch off of linux-next, with about 150-200 patches on top, on various Snapdragon devices (including my actual daily driver laptop and phone). Have debugged various issues, including in totally unrelated parts of the kernel (like twice it was FUSE related heh) no problem.