常见问题

• Error: Precompiled module at /opt/.kindling/ is not found
• Error: Prometheus cannot match error
• kindling probe init err: xxxxxx: Permission denied

Error: Precompiled module at /opt/.kindling/ is not found #

Problem description and logs #

If the kindling pod is not running normally, you can use the following command to check the logs.

kubectl logs --tail=100 -f kindling-agent-xxxx  -c kindling-agent -n kindling

* Mounting debugfs
* Failure to find a BPF probe, try to load kernel probe
* Unloading kindling-falcolib-probe, if present
* Trying to find precompiled kindling-falcolib-probe for 4.19.1-1.el7.elrepo.x86_64
* Error: Precompiled module at /opt/.kindling/ is not found, and the agent will not work as expected
Start kindling probe...
KINDLING_PROBE_VERSION: v0.1-2021-1221
Unable to load the driver
kindling probe init err: error opening device /host/dev/kindling-falcolib0. Make sure you have root credentials and that the kindling-falcolib

Solution #

1. You must install the kernel headers manually first.

For Debian-style distributions, run the command:

apt-get -y install linux-headers-$(uname -r)

For RHEL-style distributions, run the command:

yum -y install kernel-devel-$(uname -r)

Of course, you can also download RPM files manually to install the header files.

2. Rebuild the Kindling Agent image.

bash -c "$(curl -fsSL https://k8s-bpf-probes-public.oss-cn-hangzhou.aliyuncs.com/recompile-module.sh)"

The product of this step is an image. Please ensure that the image can be accessed by the k8s cluster node. Maybe you can upload the image to a private harbor, or compress it and distribute it to each node of the k8s cluster. (You should set the imagePullPolicy of kindling-agent to ifNotPresent when compress and distribute image.)

3. Finally, modify the image name of Kindling Agent.

kubectl set image ds/kindling-agent kindling-agent=kindlingproject/kindling-probe:bymyself -n kindling

Error: Prometheus cannot match error #

Problem description and logs #

After executing the script install.sh. The following error occurred:

error: unable to recognize "kindling-prometheus-servicemonitor.yml":
no matches for kind "ServiceMonitor" in version"monitoring.coreos.com/v1"

Solution #

If you have not installed Prometheus, please visit [Prometheus-Operator](https://github.com/prometheus-operator/prometheus-operator { target="_blank" })to install it.

If you have installed Prometheus without operator, please ignore this error and manually configure the exposed metrics endpoint of Kindling Agent in Prometheus. The URL is http://localhost:9500/metrics.

kindling probe init err: xxxxxx: Permission denied #

Problem description and logs #

kindling probe init err: can't create map: Permission denied

or

insmod error:cloud not insert xxx.ko:Permission denied

Solution #

• Confirm that the kindling agent has enabled the privileged mode.
• Confirm that SELinux is turned off on the node where the kindling agent is located.
• Confirm that CRI is running with root privileges.