#arc750 — Public Fediverse posts

Running #llama (latest as of now) on my Intel #Arc750 GPU with the new qwen35-9B model using SYCL (Intel's thing):

- Needs a special environment to compile (setvars.sh, icc compiler, ...)
- Only achieves ~80% compute throughput (nvtop)
- Produces gibberish output
- Bench: pp=131t/s tg=9.4t/s

So I tried using the independent Vulkan backend:

- No special env required
- Achieves 100% compute
- Reasonable output produced
- Bench: pp=393t/s tg=9.5t/s

What gives?

Running #llama (latest as of now) on my Intel #Arc750 GPU with the new qwen35-9B model using SYCL (Intel's thing):

- Needs a special environment to compile (setvars.sh, icc compiler, ...)
- Only achieves ~80% compute throughput (nvtop)
- Produces gibberish output
- Bench: pp=131t/s tg=9.4t/s

So I tried using the independent Vulkan backend:

- No special env required
- Achieves 100% compute
- Reasonable output produced
- Bench: pp=393t/s tg=9.5t/s

What gives?

It's wild how effective hardware encoding is.

I needed to convert a 1080p 2 hour lecture down to a reasonable size (so that I can view it via slow mobile). I encoded it to #AV1 using VAAPI on my #Arc750. I'm getting 20x realtime encode speed 🤯

I'm happy to get 0.2x using software encoding with libsvtav1.

It's wild how effective hardware encoding is.

I needed to convert a 1080p 2 hour lecture down to a reasonable size (so that I can view it via slow mobile). I encoded it to #AV1 using VAAPI on my #Arc750. I'm getting 20x realtime encode speed 🤯

I'm happy to get 0.2x using software encoding with libsvtav1.

And apparently I've misunderstood the work size limits in the OpenCL documentation (1024 in my case). That limit only applies to the *local work size*, not the global work size.

So indeed my sample "add" kernel is indeed just `c[gid] = a[gid] + b[gid]` (no for-loops required inside kernel, even if global size is 1GiB) and I'm getting reasonable looking "gflops" numbers (but still orders below the theoretical maximum for my #Arc750 ).

And apparently I've misunderstood the work size limits in the OpenCL documentation (1024 in my case). That limit only applies to the *local work size*, not the global work size.

So indeed my sample "add" kernel is indeed just `c[gid] = a[gid] + b[gid]` (no for-loops required inside kernel, even if global size is 1GiB) and I'm getting reasonable looking "gflops" numbers (but still orders below the theoretical maximum for my #Arc750 ).

Hardware accelerated (vaapi) #av1 10-bit encoding using #ffmpeg :

$ ffmpeg -vaapi_device /dev/dri/renderD128 -i <input> -vf '<any other filters you might need, e.g. scale/crop/drawtext>,format=p010,hwupload' -c:v av1_vaapi -b:v <bitrate> output.mp4

The 10-bit part is "format=p010", for 8-bit encode use "format=nv12".

I don't know why this works but it produces a 10bit AV1 encoded video on my #Intel #Arc750 so I'm happy.

Hardware accelerated (vaapi) #av1 10-bit encoding using #ffmpeg :

$ ffmpeg -vaapi_device /dev/dri/renderD128 -i <input> -vf '<any other filters you might need, e.g. scale/crop/drawtext>,format=p010,hwupload' -c:v av1_vaapi -b:v <bitrate> output.mp4

The 10-bit part is "format=p010", for 8-bit encode use "format=nv12".

I don't know why this works but it produces a 10bit AV1 encoded video on my #Intel #Arc750 so I'm happy.

I challenged myself to implement a code completer for #TextAdept editor in #Lua based on #llama 3.2. It ended up being <100 LOC total.

It feels pretty fast considering that it's running on modest #Arc750. And it's interesting to see the different "code" it makes up every time it's run.

I challenged myself to implement a code completer for #TextAdept editor in #Lua based on #llama 3.2. It ended up being <100 LOC total.

It feels pretty fast considering that it's running on modest #Arc750. And it's interesting to see the different "code" it makes up every time it's run.

Clearly a problem with the driver for the #Arc750, fortunately the music is still playing in the background to let me know that the computer is responsive over SSH.

Now to figure out how to do a kernel backtrace...

Clearly a problem with the driver for the #Arc750, fortunately the music is still playing in the background to let me know that the computer is responsive over SSH.

Now to figure out how to do a kernel backtrace...

OMG, so many ML/NN frameworks/runtimes!

Fortunately exactly *one* supports my #Intel #Arc750 GPU, so that makes choosing one very easy...

Model: yolov8m
Format: time (ms/image)
PyTorch: 228.85
TorchScript: 268.37
ONNX: 217.97
OpenVINO: 16.17 <== GPU
TensorFlow: SavedModel: 275.35
TensorFlow: GraphDef: 565.08
TensorFlow: Lite: 783.01
PaddlePaddle: 431.24
NCNN: 290.61

#yolo

OMG, so many ML/NN frameworks/runtimes!

Fortunately exactly *one* supports my #Intel #Arc750 GPU, so that makes choosing one very easy...

Model: yolov8m
Format: time (ms/image)
PyTorch: 228.85
TorchScript: 268.37
ONNX: 217.97
OpenVINO: 16.17 <== GPU
TensorFlow: SavedModel: 275.35
TensorFlow: GraphDef: 565.08
TensorFlow: Lite: 783.01
PaddlePaddle: 431.24
NCNN: 290.61

#yolo

Did dome napkin math, and the #intel #Arc750 GPU I installed in my always-on computer (#diy #NAS + occasional gaming) really shifted the decision towards a dedicated NAS.

The issue is that with the GPU, the power draw is ~120W idle (not good). If I can bring it down to ~20W, over the span of a year that's ~150€ in electricity savings.

So I have ~150€ to spend on a NAS solution (I already have the disks) and it's all savings after 1 year.

https://widget.uk/@burtyb/112214159765245262 looks really tempting...

Did dome napkin math, and the #intel #Arc750 GPU I installed in my always-on computer (#diy #NAS + occasional gaming) really shifted the decision towards a dedicated NAS.

The issue is that with the GPU, the power draw is ~120W idle (not good). If I can bring it down to ~20W, over the span of a year that's ~150€ in electricity savings.

So I have ~150€ to spend on a NAS solution (I already have the disks) and it's all savings after 1 year.

https://widget.uk/@burtyb/112214159765245262 looks really tempting...

So I went ahead and compiled the Intel OpenCL driver with debug symbols, and opened it up in GDB.

The init fails here: https://github.com/intel/compute-runtime/blob/e44ac2a0017434b2af6fdf5601d98975640e781e/shared/source/os_interface/linux/drm_memory_manager.cpp#L102 since the computed GPU address space is 281474976645119 which Wolfram Alpha tells me is a prime, and is nothing that the rest of the driver recognizes.

Looking at the binary pattern, I suspect a bitflip. Faking the "correct" value in the driver source gives me a working OpenCL device / platform 😕

Enough "fun" for today.

#linux #intel #Arc750

So I went ahead and compiled the Intel OpenCL driver with debug symbols, and opened it up in GDB.

The init fails here: https://github.com/intel/compute-runtime/blob/e44ac2a0017434b2af6fdf5601d98975640e781e/shared/source/os_interface/linux/drm_memory_manager.cpp#L102 since the computed GPU address space is 281474976645119 which Wolfram Alpha tells me is a prime, and is nothing that the rest of the driver recognizes.

Looking at the binary pattern, I suspect a bitflip. Faking the "correct" value in the driver source gives me a working OpenCL device / platform 😕

Enough "fun" for today.

#linux #intel #Arc750

I've spent the past few hours debugging why #OpenCL no longer picks up my #Intel #Arc750 GPU, and I got nowhere.

$ sudo clinfo
Number of platforms 0

Not even the CPU is picked up. Why oh #Arch why are you doing this to me? I guess it's a sign to go to sleep...

I've spent the past few hours debugging why #OpenCL no longer picks up my #Intel #Arc750 GPU, and I got nowhere.

$ sudo clinfo
Number of platforms 0

Not even the CPU is picked up. Why oh #Arch why are you doing this to me? I guess it's a sign to go to sleep...

Ever since I installed the #Intel #Arc750 in my computer, it experiences hard lockups - nothing responds anymore, and I have to hard reboot the computer.

It's annoying since it also acts as my Nextcloud for phone photo backups et al, and extra annoying if I'm not home and want to access older photos.

I guess I'll just migrate all of the server stuff to a #raspberrypi or some other #sbc.

Ever since I installed the #Intel #Arc750 in my computer, it experiences hard lockups - nothing responds anymore, and I have to hard reboot the computer.

It's annoying since it also acts as my Nextcloud for phone photo backups et al, and extra annoying if I'm not home and want to access older photos.

I guess I'll just migrate all of the server stuff to a #raspberrypi or some other #sbc.

Does anyone have experience with diagnosing / changing memory modules on #Nvidia GPU?

I have a semi-broken #1080ti laying around, and I'd like to #diy #repair it to make it useful again: https://toot.kottman.xyz/@michal/105216236032017261

I've not been very satisfied with my #Arc750 purchase, ever since I installed it, I'm experiencing roughly one unrecoverable lock-up per week which requires a hard reboot.

Does anyone have experience with diagnosing / changing memory modules on #Nvidia GPU?

I have a semi-broken #1080ti laying around, and I'd like to #diy #repair it to make it useful again: https://toot.kottman.xyz/@michal/105216236032017261

I've not been very satisfied with my #Arc750 purchase, ever since I installed it, I'm experiencing roughly one unrecoverable lock-up per week which requires a hard reboot.

I've been waiting for #AV1 hardware encoder for a very long time, and now that I have the #Arc750 I realize that software encoding has its merit.

I've been wondering if I can encode "computer content" (screen recording) at higher resolutions at the same bitrate as Youtube's 360p (~55kbps) and still keep details.

SW: SVT-AV1 has no problem encoding 480p, even 720p at 55k while producing crisp screen recording.

HW: Arc produces blurry gobs of artifacts at 480p and outright bus errors at 720p.

I've been waiting for #AV1 hardware encoder for a very long time, and now that I have the #Arc750 I realize that software encoding has its merit.

I've been wondering if I can encode "computer content" (screen recording) at higher resolutions at the same bitrate as Youtube's 360p (~55kbps) and still keep details.

SW: SVT-AV1 has no problem encoding 480p, even 720p at 55k while producing crisp screen recording.

HW: Arc produces blurry gobs of artifacts at 480p and outright bus errors at 720p.

I finally made #llama.cpp to on my #Intel #Arc750 using #OpenCL with the CLBlast !

https://github.com/ggerganov/llama.cpp#clblast

Unfortunately the CPU happens to be ~2x faster than the GPU (250ms/token vs. 490ms/token) 😕

I finally made #llama.cpp to on my #Intel #Arc750 using #OpenCL with the CLBlast !

https://github.com/ggerganov/llama.cpp#clblast

Unfortunately the CPU happens to be ~2x faster than the GPU (250ms/token vs. 490ms/token) 😕

Mythbusters taught me that the difference between "fun" and "science" is writing things down.

So I started #Satisfactory on my #Arc750, chose the lowest graphics settings on 1080p, and changed settings one by one, writing down the FPS.

This way I am running the "nicest settings" without sacrificing too much. For science of course.

Mythbusters taught me that the difference between "fun" and "science" is writing things down.

So I started #Satisfactory on my #Arc750, chose the lowest graphics settings on 1080p, and changed settings one by one, writing down the FPS.

This way I am running the "nicest settings" without sacrificing too much. For science of course.

More #Steam games on the #Intel #Arc750 at 1080p:

#BeamNG - 50 fps at "high/ultra" settings, 60 fps at "normal".

#BattleBit - does not start, nothing shown in the console

#Satisfactory - now starts with patch v0.8.3.2, yay! But it's just 20 fps on "ultra", 35 fps on "medium", I think it was faster on my old 1060. "Merry Christmas you filthy animal" to you yoo, CoffeeStain.

#RiseOfTheTombRaider - capped at 60fps at highest settings in the benchmark.

More #Steam games on the #Intel #Arc750 at 1080p:

#BeamNG - 50 fps at "high/ultra" settings, 60 fps at "normal".

#BattleBit - does not start, nothing shown in the console

#Satisfactory - now starts with patch v0.8.3.2, yay! But it's just 20 fps on "ultra", 35 fps on "medium", I think it was faster on my old 1060. "Merry Christmas you filthy animal" to you yoo, CoffeeStain.

#RiseOfTheTombRaider - capped at 60fps at highest settings in the benchmark.

I am capturing timelapses on a DSLR camera and converting them to #AV1 videos using #ffmpeg.

I bought the #Arc750 with hope to speed up the encoding, but in my workflow it's actually not speeding up much.

I added some simple "filter-graph profiling" to ffmpeg and independently verified using `perf record` that actually ~70% of the time is spent in decoding the JPEG image, scaling it down to 1080p and deflickering the sequence.

Amdahl's law came to bite me for not checking before buying...

I am capturing timelapses on a DSLR camera and converting them to #AV1 videos using #ffmpeg.

I bought the #Arc750 with hope to speed up the encoding, but in my workflow it's actually not speeding up much.

I added some simple "filter-graph profiling" to ffmpeg and independently verified using `perf record` that actually ~70% of the time is spent in decoding the JPEG image, scaling it down to 1080p and deflickering the sequence.

Amdahl's law came to bite me for not checking before buying...

Apparently I need to tweak the BIOS settings to make sure the Arc GPU has lower power consumption when idle.

Weird, I didn't need to do anything with the Nvidia 1060...

https://www.intel.com/content/www/us/en/support/articles/000092564/graphics.html

#Intel #Arc750

Apparently I need to tweak the BIOS settings to make sure the Arc GPU has lower power consumption when idle.

Weird, I didn't need to do anything with the Nvidia 1060...

https://www.intel.com/content/www/us/en/support/articles/000092564/graphics.html

#Intel #Arc750

This is not fun. I'm having trouble getting #OpenCL working on the #Intel #Arc750 - even clinfo hangs in "futex_wait". I have all the packages I supposedly need installed and reinstalled, rebooted multiple times.

I found a promising thread about it: https://bbs.archlinux.org/viewtopic.php?id=281546

The resolution was: "in the end I sold the card :("

This is not fun. I'm having trouble getting #OpenCL working on the #Intel #Arc750 - even clinfo hangs in "futex_wait". I have all the packages I supposedly need installed and reinstalled, rebooted multiple times.

I found a promising thread about it: https://bbs.archlinux.org/viewtopic.php?id=281546

The resolution was: "in the end I sold the card :("

One thing I did not anticipate is power draw. When I had the trusty old 1060 in, it drew ~60W at idle, which means I was OK with running this 24/7 also acting as a server.

The same setup, just with the1060 replaced with #Arc750, now draws 120W at idle. This is at the lowest power state (clocked down to 20MHz, confirmed by intel_gpu_top).

I wonder how much of that is the LED logo 🤔

One thing I did not anticipate is power draw. When I had the trusty old 1060 in, it drew ~60W at idle, which means I was OK with running this 24/7 also acting as a server.

The same setup, just with the1060 replaced with #Arc750, now draws 120W at idle. This is at the lowest power state (clocked down to 20MHz, confirmed by intel_gpu_top).

I wonder how much of that is the LED logo 🤔