It started as it often does: a customer writes an email with five topics, three questions and two new ideas. The whole thing ends up in the ticket system as a confusing collective ticket.
What if a local LLM dissects these messages automatically - directly in its own infrastructure perimeter, without the cloud, without data outflow?
That’s exactly what I implemented. And this article shows how local AI can be practically integrated into real workflows.
From experiment to productive task
The previous article “Local LLM – AI language models without the cloud” was primarily about a technical experiment: What can a language model do locally? What does offline inference feel like? How much control do you regain?
This follow-up article describes the specific use of a local LLM in my everyday life:
- I run Redmine as a ticket system for customer communication.
- Collective tickets that contain several topics are often created, especially via the email interface.
- The aim was therefore an automated breakdown of these messages into structured individual tickets.
- And that without any form of data transmission to external services.
An ideal use case for a locally operated LLM, integrated into an existing workflow.
Operating environment and objectives
Before we start implementing, take a look at the environment.
The prototype from the exploration phase should go into a container. Not as a security measure (container insulation is not hardening), but for practical reasons:
- File system isolation – cleanly separated runtime without side effects
- Resource Control – because the model shares the home or branch server with other services
- No real-time requirements – ticket processing can easily run asynchronously and with a time delay
Here, containerization is an operational and organizational tool, not a security feature. It ensures that the AI environment can be operated in a reproducible, maintainable and clearly defined manner, without encroaching on the infrastructure of other components.
A proof of concept in Go should then be created that uses both the Redmine API and the Ollama API to map the entire task end-to-end: Read tickets, pass description to the AI model, extract subtasks, create subtickets and automatically advance the main ticket.
The goal is a minimalist but robust solution that shows that local AI can be integrated into existing workflows productively, comprehensibly and confidently with reasonable effort.
Ollama container build
First clone the Ollama repository:
git clone https://github.com/ollama/ollama.git
cd ollama
A Dockerfile is already included - build attempt:
ollama/ollama [ podman build -t ollama -f Dockerfile main ] 04:49
WARN[0000] missing "CGO_CFLAGS" build argument. Try adding "--build-arg CGO_CFLAGS=<VALUE>" to the command line
WARN[0000] missing "CGO_CXXFLAGS" build argument. Try adding "--build-arg CGO_CXXFLAGS=<VALUE>" to the command line
[1/21] STEP 1/7: FROM --platform=linux/amd64 rocm/dev-almalinux-8:6.3.3-complete AS base-amd64
Error: creating build container: short-name "rocm/dev-almalinux-8:6.3.3-complete" did not resolve to an alias and no unqualified-search registries are defined in "/etc/containers/registries.conf"
The registry error is easy to fix:
sudo nano /etc/containers/registries.conf
unqualified-search-registries = ["docker.io"]
CGO flag warnings can be suppressed using explicit null values:
podman build -t ollama -f Dockerfile --build-arg TARGETARCH=amd64 --build-arg FLAVOR=amd64 --build-arg CGO_CFLAGS="" --build-arg CGO_CXXFLAGS="" .
The build takes time - and that’s no wonder:
- various Rocky Linux images
- additional YUM repositories
- proprietary SDKs (e.g. CUDA)
- final runtime in an Ubuntu image
The supply chain is long and complex. For productive use, a technical security audit is required - especially to check the entire software supply chain and its integrity. More background information in the article: “Making software supply chains visible”
TARGETARCH and FLAVOR had to be added to the build - otherwise it breaks (at least in my setup).
After a longer term:
Successfully tagged localhost/ollama:latest
e53dcbfcbecefedcffb00862e8202623de88b4d0f3e98823b0ab8914fec53a7b
Optional push into your own registry:
podman tag e53dcbfcbece registry.ext.d2ux.net/ollama:latest
podman login registry.ext.d2ux.net
podman push registry.ext.d2ux.net/ollama:latest
Testing
Before an AI model is put into the real workflow, it should first be thoroughly tested. It’s less about content and more about understanding the interfaces, testing functionality and gaining initial experience. This allows you to assess early on how stable the API is, how different models behave and what performance your own hardware delivers. Only then is it worth looking at details such as quality, speed or specific application scenarios.
API container test launch:
podman run -p 11434:11434 -it registry.ext.d2ux.net/ollama
Download a model:
curl http://localhost:11434/api/pull -d '{
"name": "phi3"
}'
First ticket splitting test:
curl http://localhost:11434/api/generate -d '{
"model": "phi3",
"stream": false,
"prompt": "Erfasse die Einzelaufgaben dieser Email und gib sie als Stichpunkte aus: ..."
}'
Test different models
Models have different numbers of parameters, which – in addition to the actual weights – has a noticeable effect on the processing time and the quality of the results. Larger models typically provide better decompositions and understand context more deeply, but require significantly more CPU resources. Smaller models respond faster, but are less precise or miss implicit tasks. Based on my tests, the following relative and partly subjective assessment results:
| Model | Parameters | CPU speed | Quality task decomposition | Memory requirement (q4_K_M) | Assessment |
|---|---|---|---|---|---|
| Qwen2.5 1.5B Instruct | 1.5B | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ | ~1.6GB | Ideal for fast batch analysis |
| Qwen2.5 3B Instruct | 3B | ⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ~2.8GB | Very good ratio of speed & precision |
| Qwen2.5 1.5B Coder | 1.5B | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ~1.6GB | Stronger with technical texts |
| Qwen2.5 7B Instruct | 7B | ⭐⭐ | ⭐⭐⭐⭐⭐ | ~4.8GB | Best overall model for task detection on CPU |
| Llama 3.2 3B Instruct | 3B | ⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ~2.9GB | Very stable output, good structuring |
| Llama 3.2 1B Instruct | 1B | ⭐⭐⭐⭐⭐ | ⭐⭐ | ~1.1GB | For very small systems, quality limited |
| Mistral Nemo 2B | 2B | ⭐⭐⭐⭐ | ⭐⭐⭐ | ~2.0GB | Fast, good, but a little less precise than Qwen |
| Phi-3 Mini 3B Instruct | 3B | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ~2.2GB | Extremely fast, very efficient, good disassembly for its size |
Idea & Specification: The Taskbot
| step | Description | Details |
|---|---|---|
| 1. Receipt of tickets | Tickets are created in the inbox | project New support requests or form messages are automatically created as issues. |
| 2. Selection of relevant tickets | Filter: Status new (ID: 1) + no parent | Only independent main tickets without a higher-level task will be processed further. |
| 3. Handover to AI model | Description text is sent to Ollama | Systemprompt forces clean output: one task per line, each with “-” as a prefix. |
| 4. Task extraction | AI returns a structured to-do list | Output is a pure list; no introductions, explanations or other text. |
| 5. Generation of sub-tickets | A subticket is created for each recognized task | Subtickets reference the main ticket as parent_issue_id. Content is taken 1:1 from the AI output. |
| 6. Main ticket update | Main ticket is set to in progress (ID: 2) | Signals that the ticket is moving to the next processing step and has already been disassembled. |
Example call
To run the taskbot, all you have to do is set the access data for Redmine and configure the desired processing. The bot then reads all matching tickets, extracts the tasks using the local AI model and automatically creates the corresponding sub-tickets.
export REDMINE_URL="https://redmine.example.com"
export REDMINE_TOKEN="1234567890abcdef"
./taskbot \
-project=inbox \
-from-status-id=1 \
-to-status-id=2 \
-ollama-url="http://192.168.2.250:11434" \
-ollama-model="qwen2.5:3b-instruct-q4_K_M"
The call filters all new tickets (status ID 1) from the project inbox, passes their description to the AI model and creates a separate subticket for each recognized task. Once the decomposition is complete, the main ticket is automatically transferred to the status in progress (Status ID 2).
Performance evaluation
For this use case, a classic multi-role standard server is used - specifically an HP MicroServer in the version that is common in many small locations, branches and edge environments. Such systems often run as all-in-one servers for file storage, container runtime and light automation tasks - and are therefore well suited as a reference platform for purely local AI processing.
Hardware base:
- Intel Xeon E-2314
- 16GB RAM
- SSD storage
Risk analysis
The models I tested in Ollama are exclusively GGUF models. This eliminates a significant portion of the usual risks that arise with cloud or remote AI - in particular the possible leakage of sensitive data. Processing is completely local and the models themselves have no functions to transmit information externally.
What remains are two manageable risk areas: On the one hand, the content behavior of the model. Language models can misinterpret tasks, miss unclear passages, or occasionally produce hallucinations. However, for the use case of task decomposition, the risk can be easily controlled because the AI only generates structuring suggestions and does not make any safety or process-critical decisions.
On the other hand, there is the risk of a compromised supply chain. Models should therefore only be loaded from trustworthy sources and checked via hash. If you only use official repositories such as Ollama, HuggingFace or ModelScope and avoid models from non-transparent sources, this risk can be largely minimized.
The locally isolated operation - model, processing and taskbot in the same security perimeter - creates an overall very high level of protection that is more than sufficient for the intended purpose.
GGUF is a pure storage format for language models. It only contains model weights, tokenizer data and static metadata - but no executable code. A GGUF model can therefore:
- do not establish network connections
- do not send telemetry
- do not reload resources
- do not execute logic or transfer data
The model is completely passive and is interpreted exclusively by the local runtime environment (e.g. Ollama).
Conclusion
A real, practical benefit becomes apparent immediately:
- Tickets are structured automatically
- better prioritization and transparency
- cleaner processing chains
- no external data flows
- confident, fully local AI usage
Local AI works — simple, secure, confident.