Replies: 6 comments 11 replies
-
|
did you figure this out? |
Beta Was this translation helpful? Give feedback.
-
|
When searching photos I see - I guess this is still pending. |
Beta Was this translation helpful? Give feedback.
-
|
it seems to work for me: problem is I get load on my CPU and running or even shows no activty. how do I confirm it's actually using HW accel? my setup: orangepi 5 Joshua-Riek ubuntu I downloaded and installed |
Beta Was this translation helpful? Give feedback.
-
|
Look forward to any progress from you. |
Beta Was this translation helpful? Give feedback.
-
|
it took some time for me to get things going, but I'm also seeing the "Power transition timed out unexpectedly" issue. :( forgot to mention I'm running rock pi 5a |
Beta Was this translation helpful? Give feedback.
-
|
Rockchip boards tend to be sensitive to the power block and cable you use. I had to try like 4 different things before I found something that worked reliably. Check what others have used with success for your model. A "dumb", non-PD charger with the right voltage and amperage is more likely to work well. |
Beta Was this translation helpful? Give feedback.
-
|
@mertalev Thank you for your suggestion. Unfortunately I can't find the power supply and cable that suits. tried a few high power rated cable and various power supply, including the Radxa 30w power adapter. |
Beta Was this translation helpful? Give feedback.
-
|
Finally! after another few hours of fiddle around and got this working. |
Beta Was this translation helpful? Give feedback.
-
|
Do you mind giving a step by step guide on how you did this? I just installed Immich on my Orange Pi 5 Plus so looking to try it out with hardware acceleration. |
Beta Was this translation helpful? Give feedback.
-
|
@RikudouGoku have you follow the steps in the doc? https://immich.app/docs/features/ml-hardware-acceleration btw.. there was an update of driver a few hours ago at https://github.com/tsukumijima/libmali-rockchip/releases |
Beta Was this translation helpful? Give feedback.
-
|
Also want to note, I have not able to get any clip model working except for the default "ViT-B-32__openai"... |
Beta Was this translation helpful? Give feedback.
-
|
I did check the docs. "/dev/mali0", "/usr/lib/libmali.so" and "/lib/firmware/mali_csffw.bin" exists. Here is how my docker-compose.yml looks under the ml area: immich-machine-learning: |
Beta Was this translation helpful? Give feedback.
-
|
I have upgraded to When running text searches I see - When forcing a facial recognition, I see - I guess I'll try with the 'g13p0' version |
Beta Was this translation helpful? Give feedback.
-
Ah. I need the g6p0 version (presumably some binary compatibility). Specifically, I'm currently trying I'm not seeing errors with smart search admin job - clip model appears to be loaded and used. Accelerated text searches not supported. dmesg is still very noisy ... GPU reset many times. I've no idea if this stops acceleration being used. |
Beta Was this translation helpful? Give feedback.
-
|
I haven't seen similar errors. However, straight after the server start, you can run mine looks like this
|
Beta Was this translation helpful? Give feedback.
-
|
Hi @plord12 I was curious if you got this working? For me I am facing a slightly different issue with mounting the *.so and .bin files with docker-compose. (yes, it's Device Stats: Ubuntu 24.04.1 LTS (GNU/Linux 6.1.0-1025-rockchip aarch64)
$: cat /etc/orangepi-release
BOARD=orangepi5plus
$: uname -a
Linux ubuntu-gpu0 6.1.0-1025-rockchip #25-Ubuntu SMP Mon Aug 26 23:01:14 UTC 2024 aarch64 aarch64 aarch64 GNU/LinuxHelper script to simplify the #!/bin/bash
echo "Display session type: ${XDG_SESSION_TYPE}"
# Install Mali GPU driver
#? https://github.com/immich-app/immich/discussions/11056
install() {
local version=$(gh release -R tsukumijima/libmali-rockchip list | awk 'NR==1{print $1}')
echo "$FUNCNAME(): Installing Mali GPU driver..."
echo "Current location: ${PWD}"
mkdir -p ~/tmp/libmali
pushd ~/tmp/libmali
# sudo apt-get install -y libmali-sunxi-utgard0-r6p0
gh release -R tsukumijima/libmali-rockchip download $version -p '*-g610-g13p0*'
sudo dpkg -i libmali-valhall-g610-g13p0-gbm_${version}_arm64.deb
# sudo dpkg --info --simulate libmali-valhall-g610-g13p0-gbm_1.9-1_arm64.deb
popd
echo "Mali GPU driver installed."
}
purge() {
echo "$FUNCNAME(): Purging Mali GPU driver..."
sudo dpkg -P libmali-valhall-g610-g13p0-gbm
echo "Mali GPU driver purged."
}
list_kernel_modules() {
local modules=${1:-''}
echo "$FUNCNAME(): Listing kernel (active/loaded) modules..."
ls -R /lib/modules/$(uname -r)/kernel/$modules
# lsmod | grep mali
}
list_kernel_drivers() {
local drivers=${1:-''}
echo "$FUNCNAME(): Listing kernel drivers..."
list_kernel_modules drivers/gpu
}
install_glmark2() {
echo "$FUNCNAME(): Installing glmark2..."
sudo apt install glmark2-es2-x11
}
install_clinfo() {
echo "$FUNCNAME(): Installing glmark2..."
sudo apt install clinfo
}
find_firmware_csf() {
echo "$FUNCNAME(): Finding Mali GPU CSF firmware..."
# find /lib/modules/$(uname -r)/kernel/drivers/gpu/drm/ -name '*csf*'
#? expect -- /usr/lib/firmware/mali_csffw.bin
sudo find / -name mali_csffw.bin
}
find_shared_obj() {
echo "$FUNCNAME(): Finding Mali GPU libmalidrm..."
# find /lib/modules/$(uname -r)/kernel/drivers/gpu/drm/ -name '*libmalidrm*'
#? expect -- /usr/lib/aarch64-linux-gnu/libmalidrm.so
sudo find / -name libmali*.so*
}
$@
x-armnn-og: &armnn-og
#? For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
#* Example tag: ${IMMICH_VERSION:-release}-cuda
image: &armnn-image ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-armnn
environment: &armnn-env
MACHINE_LEARNING_ANN: "true" #? Enable ARM-NN hardware acceleration if supported
# MACHINE_LEARNING_ANN_FP16_TURBO: "false" #? Enable ARM-NN FP16 Turbo mode if supported
# MACHINE_LEARNING_ANN_TUNING_LEVEL: "2" #? ARM-NN GPU tuning level (1: rapid, 2: normal, 3: exhaustive)
devices: &armnn-devices
- /dev/mali0:/dev/mali0
restart: &armnn-restart always
expose: &armnn-expose
- 3003
#? Reference: https://docs.docker.com/engine/storage/bind-mounts/
volumes: &armnn-og-volumes
# - /lib/firmware/mali_csffw.bin:/lib/firmware/mali_csffw.bin:ro # Mali firmware for your chipset (not always required depending on the driver)
# - /usr/lib/aarch64-linux-gnu/libmali.so:/usr/lib/libmali.so:ro # Mali driver for your chipset (always required)
- type: volume
source: model-cache
target: /cache
- type: volume #? Mali firmware for your chipset (not always required depending on the driver)
source: firmware
target: /lib/firmware
- type: volume #? Mali driver for your chipset (always required)
source: usr-lib
target: /usr/lib
services:
immich-machine-learning:
# env_file: .env
container_name: immich_machine_learning
hostname: immich_machine_learning.docker
image: *armnn-image
# extends: #? uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
#* file: hwaccel.ml.yml
#* service: cpu #? set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
environment:
<<: *armnn-env
devices: *armnn-devices
restart: *armnn-restart
expose: *armnn-expose
volumes:
*armnn-og-volumes
networks:
immich_frontend:
aliases:
- immich_machine_learning.docker
- immich_ml
healthcheck:
disable: false
volumes:
model-cache:
usr-lib:
external: true
name: usr-lib
#* driver: local
#* driver_opts:
#* type: bind
#* device: /usr/lib/aarch64-linux-gnu
#* o: bind,ro
firmware:
external: true
name: lib-firmware
#* driver: local
#* driver_opts:
#* type: bind
#* device: /lib/firmware
#* o: bind,ro
networks:
immich_frontend:
external: true
name: cloudflare-tunnelI was able to execute Smart Search and use For youDespite us having different boards are you sure that I originally was trying with Ubuntu 24.10 using ubuntu-rockchip. I wasn't sure about the For meI keep getting silly errors about setting volumes being docker: Error response from daemon: error while creating mount source path '/usr/lib/aarch64-linux-gnu/libmali.so': mkdir /usr/lib/aarch64-linux-gnu/libmali.so: read-only file system.and have tried not docker run -v $(readlink -f /usr/lib/aarch64-linux-gnu/libmali.so):/usr/lib/libmali.so -v /lib/firmware/mali_csffw.bin:/lib/firmware/mali_csffw.bin:ro -it --rm alpine /bin/ash
docker: Error response from daemon: error while creating mount source path '/usr/lib/aarch64-linux-gnu/libmali.so.1.9.0': mkdir /usr/lib/aarch64-linux-gnu/libmali.so.1.9.0: read-only file system.Anyway if you have a chance try out my suggestion to at least get it working.. in the meantime I presume that the docker snap can't see Happy New Year :-) |
Beta Was this translation helpful? Give feedback.
-
|
Using snap to install docker is not recommended and is probably the cause of those mount issues. It's best to use the docker apt repos as detailed in https://docs.docker.com/engine/install/ubuntu/. |
Beta Was this translation helpful? Give feedback.
-
Hi All,
Now that #11019 is resolved, anyone got acceleration working with maili GPU ?
Firstly I found it hard to find a definitive source for libmali ... eventually I used the deb packages at https://github.com/tsukumijima/libmali-rockchip/releases ( I chose libmali-valhall-g610-g6p0-gbm_1.9-1_arm64.deb ).
I updated hwaccel.ml.yml :
and docker-compose.yml :
Now when I click on the smart search job I see :
So it seems libmali is being picked up. But not much info on the failure.
On the host I see :
I've no clue what this means.
Thanks!
Beta Was this translation helpful? Give feedback.
All reactions