Debugging the kernel using Ftrace

AmigaOne X1000 platform specific issues related to Linux only.
Post Reply
User avatar
xeno74
Posts: 7357
Joined: Fri Mar 23, 2012 7:58 am

Debugging the kernel using Ftrace

Post by xeno74 »

For example debugging Mac-on-Linux/KVM with Ftrace.

Ftrace shows the events that lead up to a crash. It gives a better chance of finding exactly what caused it and can help the developer in creating the correct solution.
  1. First you need the kernel 3.14.4 with ftrace support. Download: vmlinux-3.14.4-KVM-PR-FTRACE-AmigaOneX1000.tar.bz2
  2. After the start of the new kernel 3.14.4, the directory /sys/kernel/debug will be created. To mount
    this directory, you can add to your /etc/fstab file:

    Code: Select all

    debugfs       /sys/kernel/debug          debugfs defaults        0       0
    
    Or you can mount it at run time with:

    Code: Select all

    mount -t debugfs nodev /sys/kernel/debug
    
  3. Change to the directory /sys/kernel/debug/tracing

    Code: Select all

    cd /sys/kernel/debug/tracing/
    
  4. Activate the function graph tracer:

    Code: Select all

    echo function_graph > current_tracer
    
  5. Start Mac OS X with startmol -X
  6. Show the kernel KVM events:

    Code: Select all

    cat trace | head -20
    
For example ftrace of Mac-on-Linux/KVM:
ftrace_mac-on-linux.txt.gz
(4.63 KiB) Downloaded 252 times
man-on-linux_boot.png
http://www.amigalinux.org
http://www.supertuxkart-amiga.de

Running Linux on AmigaONEs can require some tinkering.
User avatar
xeno74
Posts: 7357
Joined: Fri Mar 23, 2012 7:58 am

Re: Debugging the kernel using Ftrace

Post by xeno74 »

Problem Mac OS X with MOL on KVM: Sometimes I get a shell in the single user mode. But the cpu load rise up to 100% and I can't enter any commands.

A detailed ftrace from Mac-on-Linux/KVM:
  1. Code: Select all

    modprobe kvm-pr
  2. Code: Select all

    mount -t debugfs nodev /sys/kernel/debug
  3. Code: Select all

    echo 1 > /sys/kernel/debug/tracing/events/kvm/enable
  4. Code: Select all

    echo 1 > /sys/kernel/debug/tracing/events/kvm_pr/enable
  5. Code: Select all

    cat /sys/kernel/debug/tracing/trace_pipe > /dev/shm/trace &
  6. Code: Select all

    startmol -X
    (Mac OS X Tiger guest in the single user mode from the DVD ISO)
  7. Code: Select all

    cat /dev/shm/trace
  8. Code: Select all

    startmol -X
    (Mac OS X Jaguar guest in the single user mode from the disk image)
  9. Code: Select all

    cat /dev/shm/trace
Here are the traces:

trace_mol_mac_os_x_tiger.txt.tar.gz
trace_mol_mac_os_x_jaguar.txt.tar.gz

Screenshots:
man-on-linux_boot3.png
man-on-linux_boot2.png
Note: This is a Linux thread. Mac-on-Linux is only an example for tracing the Linux kernel. Please do not ask a question about Mac OS X.
http://www.amigalinux.org
http://www.supertuxkart-amiga.de

Running Linux on AmigaONEs can require some tinkering.
User avatar
xeno74
Posts: 7357
Joined: Fri Mar 23, 2012 7:58 am

Re: Debugging the kernel using Ftrace

Post by xeno74 »

I have compiled the kernel 3.14.4 with KVM DEBUG_MMU.

Code: Select all

diff --git a/arch/powerpc/kvm/book3s_32_mmu.c b/arch/powerpc/kvm/book3s_32_mmu.c
index 93503bb..939b14f 100644
--- a/arch/powerpc/kvm/book3s_32_mmu.c
+++ b/arch/powerpc/kvm/book3s_32_mmu.c
@@ -27,8 +27,8 @@
#include <asm/kvm_ppc.h>
#include <asm/kvm_book3s.h>

-/* #define DEBUG_MMU */
-/* #define DEBUG_MMU_PTE */
+#define DEBUG_MMU
+#define DEBUG_MMU_PTE
/* #define DEBUG_MMU_PTE_IP 0xfff14c40 */

#ifdef DEBUG_MMU 
Now I get KVM messages over dmesg:

Code: Select all

watch -n 0 "dmesg" > dmesg_mol_mac_os_x_tiger.txt

Code: Select all

watch -n 0 "dmesg" > dmesg_mol_mac_os_x_jaguar.txt
dmesg_mol_mac_os_x_tiger.txt.tar.gz
(73.33 KiB) Downloaded 251 times
dmesg_mol_mac_os_x_jaguar.txt.tar.gz
(101.48 KiB) Downloaded 223 times
Note: This is a Linux thread. Mac-on-Linux is only an example for debugging the Linux kernel. Please do not ask a question about Mac OS X.
http://www.amigalinux.org
http://www.supertuxkart-amiga.de

Running Linux on AmigaONEs can require some tinkering.
Post Reply