This is all stated when i thought to try Ollama running on Intel iGPU on laptop. This took me into the rabbit hole and back.

Installation and running in GPU mode Link to heading

I initially got “no GPU detected” from Ollama, So I had to dig deeper into the source. Based on the code, I needed to install libze_intel_gpu.so from the following packages.

sudo apt install libze-intel-gpu-dev libze-intel-gpu1

Then, enable Intel GPU env vars

export OLLAMA_INTEL_GPU=1

Finally, running the thing with debug enabled to see more messages

cmake -B build
cmake --build build
export OLLAMA_DEBUG=1
go run . serve

The next step was problem detecting memory of intel GPU. After googling around, it seems like known issue with Intel iGPU.

discovered 0 Level-Zero memory modules
releasing oneapi library

Possible Solution 1 Link to heading

There is proposed PR at 4 trying to work around that but i don’t think it never got in.

Possible solution was to try this guys repo and I had to set export OLLAMA_FORCE_ENABLE_INTEL_IGPU=1

git clone https://github.com/zhewang1-intc/ollama/
git checkout support_intel_igpus

That didn’t work out. It seems this repo/branch was far behind the Ollama repo.

Possible Solution 2 Link to heading

I tried this guy repo which seems like Docker image and using something called ipex-llm

git clone https://github.com/mattcurf/ollama-intel-gpu
cd ollama-intel-gpu
docker compose up

Ollama walkthrough Link to heading

The reason i am here is I wanted to run Ollama on the embedded Intel GPU. Running the vanilla Ollama, I got these messages.

time=2025-01-30T18:49:26.982Z level=INFO source=gpu.go:217 msg="looking for compatible GPUs"
time=2025-01-30T18:49:26.991Z level=INFO source=gpu.go:377 msg="no compatible GPUs were discovered"

Which pointed to this snippet in Ollama where GPU are being discovered

			oHandles = initOneAPIHandles()
			if oHandles != nil && oHandles.oneapi != nil {
				for d := range oHandles.oneapi.num_drivers {
					if oHandles.oneapi == nil {
						// shouldn't happen
						slog.Warn("nil oneapi handle with driver count", "count", int(oHandles.oneapi.num_drivers))
						continue
					}
					devCount := C.oneapi_get_device_count(*oHandles.oneapi, C.int(d))
					for i := range devCount {
						gpuInfo := OneapiGPUInfo{
							GpuInfo: GpuInfo{
								Library: "oneapi",
							},
							driverIndex: int(d),
							gpuIndex:    int(i),
						}
						// TODO - split bootstrapping from updating free memory
						C.oneapi_check_vram(*oHandles.oneapi, C.int(d), i, &memInfo)
						// TODO - convert this to MinimumMemory based on testing...
						var totalFreeMem float64 = float64(memInfo.free) * 0.95 // work-around: leave some reserve vram for mkl lib used in ggml-sycl backend.
						memInfo.free = C.uint64_t(totalFreeMem)
						gpuInfo.TotalMemory = uint64(memInfo.total)
						gpuInfo.FreeMemory = uint64(memInfo.free)
						gpuInfo.ID = C.GoString(&memInfo.gpu_id[0])
						gpuInfo.Name = C.GoString(&memInfo.gpu_name[0])
						gpuInfo.DependencyPath = []string{LibOllamaPath}
						oneapiGPUs = append(oneapiGPUs, gpuInfo)
					}
				}
			}
		}

		rocmGPUs, err = AMDGetGPUInfo()
		if err != nil {
			bootstrapErrors = append(bootstrapErrors, err)
		}
		bootstrapped = true
		if len(cudaGPUs) == 0 && len(rocmGPUs) == 0 && len(oneapiGPUs) == 0 {
			slog.Info("no compatible GPUs were discovered")
		}

Looking at initOneAPIHandles for One API (Intels GPU API apparently). And I can see it looks for libze_intel_gpu.so. That’s Why i install the libraries above.

// Note: gpuMutex must already be held
func initOneAPIHandles() *oneapiHandles {
	oHandles := &oneapiHandles{}

	// Short Circuit if we already know which library to use
	// ignore bootstrap errors in this case since we already recorded them
	if oneapiLibPath != "" {
		oHandles.deviceCount, oHandles.oneapi, _, _ = loadOneapiMgmt([]string{oneapiLibPath})
		return oHandles
	}

	oneapiLibPaths := FindGPULibs(OneapiMgmtName, OneapiGlobs)
	if len(oneapiLibPaths) > 0 {
		var err error
		oHandles.deviceCount, oHandles.oneapi, oneapiLibPath, err = loadOneapiMgmt(oneapiLibPaths)
		if err != nil {
			bootstrapErrors = append(bootstrapErrors, err)
		}
	}

	return oHandles
}

var OneapiGlobs = []string{
	"/usr/lib/x86_64-linux-gnu/libze_intel_gpu.so*",
	"/usr/lib*/libze_intel_gpu.so*",
}