dev.nlited.com

>>

DbgOut11 on Overo

<<<< prev
next >>>>

2016-03-03 03:20:51 chip Page 1581 📢 PUBLIC

Mar 2 2016


W A R N I N G

What follows is the "raw dump" of how the sausage was made, and it contains a lot of ugly hacks that will (or should have been) cleaned up before it is used in any serious production project. My goal at this point is to just get it working as quickly as possible. This is stage one of the typical 3-stage process.

  1. Get it working. (Exploration)
  2. Get it working right. (Refinement)
  3. Get it working well. (Optimization)

DbgOut11 does not build with the new Overo kernel. I need to update the build, update any API changes, fix any new bugs. At the same time, I need to be careful not to break compatibility with the existing BLIS platform.

I have built a working Overo boot disk. This provides both the boot disk and the Linux source tree that I need to build my kernel modules.


Mistake icon

I am always too quick to commit changes, and neglected to create a new branch before making any modifications. This will require changes to DbgOut11 for both the Overo/yocto build environment and Linux 3.18. The DbgOut11 on Overo changes are in the "DbgOut318" branch. The "master" branch should still build for the BLIS/Linux 3.2 platform.

virtual/kernel vs gumstix-console-image

The bitbake recipe to create the complete boot disk is gumstix-console-image. This can also be used to create a devshell for cross-compiling programs to run on the gumstix. Do not use this for compiling kernel modules.

The recipe to create the devshell for compiling kernel modules is virtual/kernel.

KSRC vs KBUILD_OUTPUT

The latest Overo yocto build environment behaves a bit differently than the BLIS version. The kernel sources are now split into two locations, one for the original kernel sources (KSRC) and another for the kernel sources used to build the binaries (KBUILD_OUTPUT). Neither location is complete, both are needed to build kernel modules. KBUILD_OUTPUT is part of the yocto environment and is set automatically. KSRC needs to be set manually from the devshell.


devshell: cd /src/yocto/overo . setenv.sh bitbake -c devshell virtual/kernel export KSRC=`pwd` cd ~/Src/HQ/Dev/SB/Chip/DbgOut/1102/ . SetLnx.sh make -s -f Linux.mak

The KBUILD_OUTPUT directory contains a symlink source that can also be used to set KSRC.

Building DbgOut11

Enable Host Folder Sharing on OveroVM

11:40> Install VMware tools on Overo. Reduce the memory size of Overo VM to 1.5GB so I can run both Overo and VS12 on Pogo (8GB total). Restart. Share C:\Share.
11:54> The vmhgfs module is not loading.

Trying vmtools again. Still not working. I found this post which led to this. I applied the patch to file.c, it compiled but then the build failed in link.c. Clearly, VMware's folder sharing does not work with Ubuntu 14.04 (Linux 4.2). :(

This presents a BFP. The inability to easily copy files between the Overo VM and the host will be a significant inconvenience. Building a new Overo VM using Ubuntu 12.04 will set me back at least 8 hours and make finishing the demo very difficult. The IP networking is operational, so I can pull files through git and web sites. I might be able to transfer files through the VS12 nginx webserver...

Enable Git on OveroVM

Copy my git ssh private key (githqchip.rsa) to Overo VM. I can use text copy/paste for this.
~> mkdir ~/.ssh vi ~/.ssh/githqchip.rsa copy private key contents chmod 400 ~/.ssh/githqchip.rsa vi ~/.ssh/config Host aws.chip4.net Hostname aws.chip4.net. User git-dev Port 22 IdentityFile ~/.ssh/githqchip.rsa IdentitiesOnly yes

Git DbgOut on OveroVM

13:00> Fetch the DbgOut project to the Overo VM using git. I want to avoid copying the entire Chip repo, if possible. I configure a sparse repo:
mkdir -p ~/Src/HQ/Dev/SB/Chip/ cd ~/Src/HQ/Dev/SB/Chip/ git init git remote add -f origin ssh://hq-chip@aws.chip4.net/git/src/HQ/Dev/SB/Chip git config core.sparseCheckout true vi .git/info/sparse-checkout DbgOut/1102/ git pull origin master
13:18> This worked, just pulling the DbgOut project. It was still a 275MB download.

Walk the dogs.

Build DbgOut11.ko

15:30> Create an overo devshell:
build> bitbake -c devshell gumstix-console-image

The devshell uses "arm-poky-linux-gnueabi-" as the cross-coompiler prefix, so I need to change SetLinux.sh.

cc1: fatal error: /include/generated/autoconf.h: No such file or directory
autoconf.h is found in
/src/yocto/overo/build/tmp/work/overo-poky-linux-gnueabi/linux-gumstix/3.18-r0/linux-overo-standard-build/include/generated
However, the standard Linux headers seem to be in an entirely different location with a different kernel version:
./cortexa8hf-vfp-neon-poky-linux-gnueabi/linux-libc-headers/3.19-r0/linux-3.19/include/linux/module.h

OK... The kernel organization seems to have changed somewhere along the line. I now need TWO directories: $KSRC and $KBUILD_DIR. KSRC is the directory that is used by devshell and contains the "generic" kernel sources and contains module.h. KBUILD_DIR is set by yocto and points the kernel sources that were used to build the specific machine and includes autoconf.h. Neither is complete!

I will need slightly different compiler options depending on the kernel version. It also appears that some of the fundamental kernel structures have changed, and the source code will need to change to match.

LnxNew.mak: # Remove the machine-specific options and use the CC macro defined by yocto. 73 # CPU options 74 # ARM currently supported 75 #CPUopt += -marm -march=armv7-a -mcpu=cortex-a8 -msoft-float 76 #CPUopt += -mabi=aapcs-linux -mno-thumb-interwork 77 #CPUopt += -mlittle-endian -Uarm 78 #CPUopt += -fno-dwarf2-cfi-asm ... 87 # Include paths, required. Order is important. 88 INC += -include $(KSRC)/include/linux/kconfig.h 89 INC += -include $(KBUILD_OUTPUT)/include/generated/autoconf.h 90 INC += -I$(KBUILD_OUTPUT) 91 INC += -I$(KBUILD_OUTPUT)/include 92 INC += -I${KBUILD_OUTPUT}/include/generated 93 INC += -I${KSRC}/arch/arm/include 94 INC += -I${KBUILD_OUTPUT}/arch/arm/include/generated 95 INC += -I${KSRC}/include 96 INC += -I${KSRC}/arch/arm/mach-omap2/include 97 INC += -I${KSRC}/arch/arm/plat-omap/include 98 INC += -I${KSRC}/drivers/media/video 99 INC += -I. -I./LnxKrn

Fix DbgOut11.ko For Linux 3.18

17:00> Update the DbgOut server to build against the new kernel sources.

create_proc_entry() has been replaced with proc_create().

I also needed to update the modpost step to replace KSRC with KBUILD_OUTPUT.

17:53> I have finally compiled and linked DbgOut11.ko.

Load DbgOut11.ko on Overo

18:00> Try to load DbgOut11.ko on the gumstix Overo...
[23718.104919] DbgOut11: version magic '3.18.18-custom SMP mod_unload modversions ARMv7 p2v8 ' should be '3.18.18-custom SMP mod_unload modversions ARMv6 p2v8 ' insmod: ERROR: could not insert module DbgOut11.ko: Invalid module format
Easy fix in LnxServer.mak:
Try again...
root@overo:~# insmod DbgOut11.ko [23885.060882] DbgOut11 kernel has arrived... [23885.065399] DbgOut11 Server has arrived! [23885.069549] DbgOut11 1.2.220 root@ubuntu 20160302175724 [23885.087615] DbgOutServer: Device 'DbgOutServer' created. [23885.093231] BUG: FP instruction issued in kernel mode with FP unit disabled [23885.100585] Internal error: Oops - undefined instruction: 0 [#1] SMP ARM [23885.107604] Modules linked in: DbgOut11(O+) ctr ccm rfcomm ecb hci_uart bnep bluetooth arc4 wl18xx wlcore mac80211 cfg80211 rfkill snd_soc_omap_twl4030 snd_soc_omap_mcbsp snd_soc_omap snd_pcm_dmaengine wlcore_sdio snd_soc_twl4030 snd_soc_core snd_compress snd_pcm snd_timer snd mt9v032 soundcore v4l2_common videodev twl4030_madc media industrialio ipv6 [23885.140594] CPU: 0 PID: 1969 Comm: insmod Tainted: G O 3.18.18-custom #1 [23885.148712] task: dd0cc440 ti: dd58a000 task.ti: dd58a000 [23885.154479] PC is at _ZN4Fifo7Create2Ev+0x8/0x110 [DbgOut11] [23885.160461] LR is at _ZN4Fifo6CreateEPP8Handle_s+0x40/0x88 [DbgOut11] [23885.167236] pc : [] lr : [] psr: a00f0013 [23885.167236] sp : dd58bdec ip : 00000000 fp : bf3aaed4 [23885.179290] r10: dd67f608 r9 : dd67ff00 r8 : c09041a0 [23885.184783] r7 : 00000000 r6 : bf3a2000 r5 : dd58a030 r4 : dd1a4d00 [23885.191650] r3 : 00000000 r2 : 00000000 r1 : fffffff0 r0 : dd1a4d00 [23885.198486] Flags: NzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user [23885.205993] Control: 10c5387d Table: 9d3c4019 DAC: 00000015 [23885.212036] Process insmod (pid: 1969, stack limit = 0xdd58a240) [23885.218353] Stack: (0xdd58bdec to 0xdd58c000)

18:00> OK, believe it or not that is real progress! My driver is now accepted into the castle walls. I am fairly sure the crash is in the proc file stuff.

18:55> I disabled Fifo::Create() and the module is now able to load.

root@overo:~# insmod DbgOut11.ko [ 2935.164947] DbgOut11 kernel has arrived... [ 2935.169464] DbgOut11 Server has arrived! [ 2935.173614] DbgOut11 1.2.221 root@ubuntu 20160302180534 [ 2935.191497] DbgOutServer: Device 'DbgOutServer' created. [ 2935.197357] Created 8 trace blocks of 4096 bytes, 32768 total. [ 2935.203491] Trace flush level set to 2620 events. [ 2935.208526] Trace:Create2: DBG_TRACE [ 2935.212310] DbgOut Server creation is complete! root@overo:~#

Fix /proc/DbgOut11/Status.txt

The /proc/DbgOut11/ directory was created, but the Status.txt file was not visible. The call to remove_proc_entry() blew up while unloading, although the driver was still able to unload successfully. I need to do some research on the new proc API.

Branch DbgOut

19:30> Create a new branch: DbgOut318 for work on compatibility with the Linux 3.18 kernel.

The Strangest Bug...

23:45> I spent over three hours figuring out the strangest bug. This code would always crash:

ItemFifo.cpp: 363 int Fifo::Create2(void) { 364 int Err= DBGERR_OK; 365 UINT BlockMax; 366 BLOCK *pBlk; 367 Debug(0,"Fifo:Create2()\n"); 368 BlockMax= gCfg->ItemFifoBlockCount; 369 BlockSz= gCfg->ItemFifoBlockSize; 370 if(BlockMax > BLOCK_MAX) 371 BlockMax= BLOCK_MAX; 372 Debug(0,"Fifo:Create2: Creating %d blocks of %d bytes...\n",BlockMax,BlockSz); 373 if(DbgErr(Err= OsCriticalCreate(&hCritical,"ItemFifo"))) 374 Err= Error(DBGERR_SYSCREATE,"Fifo:Create2: Unable to create critical section."); 375 for(BlockCt=0;BlockCtSignature= SIGNATURE_FIFO_BLOCK; 379 pBlk->DataSz= BlockSz; 381 pBlk->StreamPos= 0; //BUG: This line crashes! fstd d8,[r0,#8] 386 pBlk->WrPtr= 0; 387 BlkList[BlockCt]= pBlk; 388 } 389 Debug(0,"Fifo:Create2: Created %d blocks of %d bytes, %d total\n",BlockCt,BlockSz,BlockCt*BlockSz); 390 if(BlockCt) 391 IsEnabled= 1; 392 return(Err); 393 }

This is kernel code, so I had to rely on old-school printf debugging to isolate the line that caused the crash. This was difficult for a number of reasons. I never expected the problem to be setting a value to zero, so I was looking hard at the OsCriticalCreate(), MemAlloc(), Debug(), etc. After I had narrowed it down to the loop, I put a break after each line until I narrowed it down to a single line: pBlk->StreamPos= 0;.

What was different about this line? StreamPos is a UINT64. Fortunately, my custom makefile generates assembly output and I was able to look at the actual machine code. This line was compiled as
473 .loc 1 380 0 discriminator 2 474 0234 028B80ED fstd d8, [r0, #8] @ int

fstd stores a floating point register. The compiler is optimizing by using a 64bit floating point register to clear a UINT64. This is a problem because the Linux kernel was compiled without floating point support, so the instruction triggers an exception and kernel panic.

I don't know how to fix this. I thought using the inherited CC macro from the devshell would solve my compatibility problems. But the CC expands to
arm-poky-linux-gnueabi-gcc -march=armv7-a -mfloat-abi=hard -mfpu=neon -mtune=cortex-a8 --sysroot=/src/yocto/overo/build/tmp/sysroots/overo
This enables the use of floating point registers. Using -mfloat-abi=soft would disable floating point registers, but I can't add this option without redefining CC without the "hard" option.

So the question is: Why is the kernel compiled without hardware floating point while the compiler is configured to use hardware floating point?

00:41> This is very confusing. As far as I can tell, everything is compiled with -mfloat-abi=hard. I don't know why fstd is triggering a fault. I have read that floating point is disallowed in the kernel by edict, it is simply too much trouble to maintain the floating point state for too little benefit.

So... the solution seems to be figuring out how to compile my code with the "-mfloat-abi=softfp" option.

01:30> I was able to build a version by defining all the compiler options in the makefile. The key was setting the --sysroot option. This creates a new problem, these options -- especially the sysroots path -- are now hard-coded in the makefile. I need to figure out how to derive these settings from the yocto environment.

On the other hand, these settings are completely under my control and I control the kernel that is being loaded. This is only a problem if I want the driver to be recompiled on someone else's system. I need to set this problem aside for now and concentrate on the demo. I only have one day left!

DbgOut User-Mode Components

11:00 (Day 3)> Build the user-mode DbgOut components.

The hard/soft configuration continues to vex me! I know for a fact that the gumstix-console-image recipes are configured for HARD floating-point. However, actually using floating point registers in the kernel is an instant BSoD. I configured the build environment to use SOFT floating-point and I was able to build and run the kernel module.

Now I am trying to build the user-mode components and I need to use HARD floating-point! The cross-compiler was built using the hard option and the soft option is not supported. Including stdio.h triggers a compiler error:
File.c: In file included from /src/yocto/overo/build/tmp/sysroots/overo/usr/include/features.h:389:0, from /src/yocto/overo/build/tmp/sysroots/overo/usr/include/stdio.h:27, from File.c:6: /src/yocto/overo/build/tmp/sysroots/overo/usr/include/gnu/stubs.h:7:29: fatal error: gnu/stubs-soft.h: No such file or directory # include ^ compilation terminated. make: *** [Out/Lnx_armsf_Dbg/File.obj] Error 1

I had bumped into this error last night and it confused me. I sidestepped it by removing the offending include, which fortunately was superfluous. Now it is required and I had to investigate further. I realized that "stubs-soft" refers to the software floating point emulation interface. There is a "stubs-hard.h" file, but no "stubs-soft.h" -- because this flavor of gcc was itself compiled using the hardware floating point option. Software emulation is not supported.

The user-mode make files need to use the CC options inherited from devshell, including the floating point options. This was fixed by simply commenting out the CPUopts in SetEnv.sh. The option I really need for the kernel build is "enable hardware floating point, but don't use it".

I just had a thought: I may be the only one bumping into this problem because no one else enables the optimizer for kernel modules. This problem is triggered by using the optimizer in a hard floating point environment where actually using the FP registers is forbidden. If I disabled the optimizer, which I loathe to do, the problem would be swept under the rug. I need to create a "HelloWorld" module using the normal build process and take a close look at the options. I expect to see "-O0" to disable the optimizer. If I hack the build to use "-O3" (maximum optimization), set a UINT64 to zero, and see it transcribed as "fstd" ... BSoD!

Client Library

Build the kernel-mode client library.

I am reliving all the build problems of the DbgOut11.ko module...

Comment out the CC and CP definitions in the make file, replace $(CP) with $(CPP).

Example Program

12:00> Build the example console app.

Mistake icon

I spent three hours chasing after unresolved externals only to finally realize I should have been invoking $(CC) instead of $(LD). Using $LD invokes a "bare-bones" linker where everything needs to be linked explicitly. Using $CC to link will automatically include intrinsic libraries that are specific to the gcc version, such as the C start-up and exit routines, C++ exception handlers, __aeabi_ functions, etc.


11:59:29 00:00:32 ExConsole> make -s -f LnxExConsole.mak Main.c: Updating version information... Reading file info from VerFile.h... Linking Out/Lnx_armsf_Dbg/ExConsole ... arm-poky-linux-gnueabi-ld: Out/Lnx_armsf_Dbg/Main.obj: undefined reference to symbol 'strncpy@@GLIBC_2.4' /src/yocto/overo/build/tmp/sysroots/overo/lib/libc.so.6: error adding symbols: DSO missing from command line make: *** [Out/Lnx_armsf_Dbg/ExConsole] Error 1

Fixed by adding "-lc" to include libc.

For some reason DbgLink.obj is silently being munged in libDbgOut11.a.
12:17:31 00:00:06 ClntUser> nm Out/Lnx_armsf_Dbg/DbgLink.obj nm: Out/Lnx_armsf_Dbg/DbgLink.obj: File format not recognized
Ahh... For some reason the .obj output from the .cpp sources is the preprocessor output, not the linker output. For some reason $CPP includes the "-E" option to write out the preprocessor output. I removed it.

Now I have the old long division bugbear:
12:27:21 00:00:06 ExConsole> make -s -f LnxExConsole.mak Updating version information... Reading file info from VerFile.h... Linking Out/Lnx_armsf_Dbg/ExConsole ... arm-poky-linux-gnueabi-ld: warning: cannot find entry symbol _start; defaulting to 0000000000008630 Out/Lnx_armsf_Dbg/Main.obj: In function `TimerReport': /home/chip/Src/HQ/Dev/SB/Chip/DbgOut/1102/Examples/Lnx/ExConsole/Main.c:278: undefined reference to `__aeabi_ul2d' Out/Lnx_armsf_Dbg/Main.obj: In function `TimerStop': /home/chip/Src/HQ/Dev/SB/Chip/DbgOut/1102/Examples/Lnx/ExConsole/Main.c:263: undefined reference to `__aeabi_ul2d' Out/Lnx_armsf_Dbg/Main.obj: In function `LongTest': /home/chip/Src/HQ/Dev/SB/Chip/DbgOut/1102/Examples/Lnx/ExConsole/Main.c:324: undefined reference to `__aeabi_uldivmod' /home/chip/Src/HQ/Dev/SB/Chip/DbgOut/1102/Examples/Lnx/ExConsole/Main.c:324: undefined reference to `__aeabi_idivmod' /home/chip/Src/HQ/Dev/SB/Chip/DbgOut/1102/Out/Lnx_armsf_Dbg/libDbgOut11.a(DbgLink.obj):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0' /home/chip/Src/HQ/Dev/SB/Chip/DbgOut/1102/Out/Lnx_armsf_Dbg/libDbgOut11.a(DbgLink.obj): In function `__static_initialization_and_destruction_0': /home/chip/Src/HQ/Dev/SB/Chip/DbgOut/1102/ClntLib/Lnx/ClntUser/DbgLink.cpp:36: undefined reference to `__dso_handle' /home/chip/Src/HQ/Dev/SB/Chip/DbgOut/1102/ClntLib/Lnx/ClntUser/DbgLink.cpp:36: undefined reference to `__dso_handle' /home/chip/Src/HQ/Dev/SB/Chip/DbgOut/1102/Out/Lnx_armsf_Dbg/libDbgOut11.a(TmpBuf.obj):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0' /home/chip/Src/HQ/Dev/SB/Chip/DbgOut/1102/Out/Lnx_armsf_Dbg/libDbgOut11.a(TmpBuf.obj): In function `__static_initialization_and_destruction_0': /home/chip/Src/HQ/Dev/SB/Chip/DbgOut/1102/ClntLib/Lnx/ClntUser/TmpBuf.cpp:21: undefined reference to `__dso_handle' /home/chip/Src/HQ/Dev/SB/Chip/DbgOut/1102/ClntLib/Lnx/ClntUser/TmpBuf.cpp:21: undefined reference to `__dso_handle' arm-poky-linux-gnueabi-ld: Out/Lnx_armsf_Dbg/ExConsole: hidden symbol `__dso_handle' isn't defined arm-poky-linux-gnueabi-ld: final link failed: Bad value make: *** [Out/Lnx_armsf_Dbg/ExConsole] Error 1

I fixed most of the __aeabi_ errors by explicitly including libgcc.a.

ExConsole.mak: Libs+= -L/src/yocto/overo/build/tmp/sysroots/overo/usr/lib/arm-poky-linux-gnueabi/4.9.2/ Libs+= -lgcc

The __dso_handle is some C++ weirdness. This is the best explanation I could find. The hack-fix is to declare it in Main.c. UPDATE: This was a red herring, and was later removed.

Main.c: void * __dso_handle= NULL;

This leaves just three unresolved:
Main.c: Updating version information... Reading file info from VerFile.h... Linking Out/Lnx_armsf_Dbg/ExConsole ... arm-poky-linux-gnueabi-ld: warning: cannot find entry symbol _start; defaulting to 00000000000085b8 /home/chip/Src/HQ/Dev/SB/Chip/DbgOut/1102/Out/Lnx_armsf_Dbg/libDbgOut11.a(DbgLink.obj):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0' /home/chip/Src/HQ/Dev/SB/Chip/DbgOut/1102/Out/Lnx_armsf_Dbg/libDbgOut11.a(TmpBuf.obj):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0' make: *** [Out/Lnx_armsf_Dbg/ExConsole] Error 1

__aeabi_cpp_unwind_pr0 was found in gcc/gnu/libgcc_eh.a (-lgcc_eh)

#*(&!!! It turns out a lot of my problems were caused by invoking $(LD) directly. I replaced $(LD) with $(CC) and all my linkage problems disappeared. I had to insert "-Xlinker" before some of the options.

ExConsole.mak: 16 #Libs+= -L/src/yocto/overo/build/tmp/sysroots/overo/usr/lib/arm-poky-linux-gnueabi/4.9.2/ 17 Libs+= -lDbgOut11 18 #Libs+= -lrt -lc 19 #Libs+= -lstdc++ -lgcc -lgcc_eh ... 73 $(OUT)/ExConsole: $(Objs) \ 74 $(DBGOUT_HOME)/$(OUT)/libDbgOut11.a 75 @echo Updating version information... 76 perl $(DBGOUT_HOME)/Bin/VerUpdate.pl builder="`whoami`@`hostname`" VerFile.h 77 $(CC) -c -I$(DBGOUT_HOME)/Include $(VER_OPT) -o $(OUT)/VerID.obj VerID.c 78 cat VerFile.h >> $(LOG) 79 @echo Linking $@ ... 80 echo $(CC) -Xlinker $(LinkOpt) -Xlinker -Map=$(OUT)/ExConsole.map -o $@ $(OUT)/VerID.obj $^ $(Libs) >> 81 $(CC) -Xlinker $(LinkOpt) -Xlinker -Map=$(OUT)/ExConsole.map -o $@ $(OUT)/VerID.obj $^ $(Libs) 82 # echo $(LD) $(LinkOpt) -Map=$(OUT)/ExConsole.map -o $@ $(OUT)/VerID.obj $^ $(Libs) >> $(LOG) 83 # $(LD) $(LinkOpt) -Map=$(OUT)/ExConsole.map -o $@ $(OUT)/VerID.obj $^ $(Libs)

14:17> I removed the __dso_handle declaration from Main.c. And it finally links...
14:11:10 00:00:08 ExConsole> make -s -f LnxExConsole.mak Main.c: Updating version information... Reading file info from VerFile.h... Linking Out/Lnx_armsf_Dbg/ExConsole ... Linux user-mode example build complete.

Example Kernel Module

Build Example/Linux/ExDriver

I copied the makefile configuration from Server/LnxKrnl. I need to use the devshell environment from virtual/kernel, not gumstix-console-image.

The API to create a /sys/ file needs to be updated.
Building ExDriver module... Building ExDriver.ko into Out/Lnx__armsf_Dbg DrvEntry.c: Initial module link... Generating Kernel linker source... Compiling Kernel linker object... In file included from /src/yocto/overo/build/tmp/work-shared/overo/kernel-source/include/linux/kobject.h:21:0, from /src/yocto/overo/build/tmp/work-shared/overo/kernel-source/include/linux/module.h:16, from Out/Lnx__armsf_Dbg/_ExDriver.mod.c:1: /src/yocto/overo/build/tmp/work-shared/overo/kernel-source/include/linux/sysfs.h: In function 'sysfs_get_dirent': /src/yocto/overo/build/tmp/work-shared/overo/kernel-source/include/linux/sysfs.h:457:37: warning: pointer targets in passing argument 2 of 'kernfs_find_and_get' differ in signedness [-Wpointer-sign] return kernfs_find_and_get(parent, name); ^ In file included from /src/yocto/overo/build/tmp/work-shared/overo/kernel-source/include/linux/sysfs.h:15:0, from /src/yocto/overo/build/tmp/work-shared/overo/kernel-source/include/linux/kobject.h:21, from /src/yocto/overo/build/tmp/work-shared/overo/kernel-source/include/linux/module.h:16, from Out/Lnx__armsf_Dbg/_ExDriver.mod.c:1: /src/yocto/overo/build/tmp/work-shared/overo/kernel-source/include/linux/kernfs.h:411:1: note: expected 'const char *' but argument is of type 'const unsigned char *' kernfs_find_and_get(struct kernfs_node *kn, const char *name) ^ Final Kernel link... -rw-rw-r-- 1 root root 94K Mar 3 19:57 Out/Lnx__armsf_Dbg/ExDriver.ko Linux kernel-mode example build complete.

These are only warnings, the example driver was built successfully.

DbgOutRelay

11:25 (Friday)> Build DbgOutRelay. This is the user-mode component that runs on the target and host (if they are different). On the target, DbgOutRelay fetches the data stream from the kernel module and relays it over the network to the DbgOutRelay running on the host. On the host, DbgOutRelay receives the data stream and writes to a file in the data store.

The make file needed to be modified a bit. The "-E" option in the devshell's $CPP is extremely annoying. Both $CC and $CPP refer to "-gcc", so I might be able to use $CC for both.

I added the "-Wno-delete-non-virtual-dtor" option to suppress some warnings. This creates a new warning if it is included in the options for a C file, so I had to create $CPopt for options that are specific to C++.

Build the Entire DbgOut Suite

I have fixed the build for all the individual components, now I need to run the full Linux build.

>

There are still some messy warnings, but the entire Linux project now builds.
11:36:04 00:00:07 1102> make -s -f Linux.mak REBUILD=1 Application output directory is Out/Lnx_armsf_Dbg Kernel output directory is Out/Lnx__armsf_Dbg Reading product info from Include/VerProd.h... Building Linux OS support library... Error.c: File.c: MemAlloc.c: New.cpp: OsError.c: StringA.c: StringW.c: Thread.c: Tick.c: Time.c: TmpBuf.cpp: WndUtil.c: Sync.cpp: Updating version information... Reading file info from VerFile.h... Linking Out/Lnx_armsf_Dbg/libLnxOsUsr.a ... Linux OS library build complete. Building DbgOut11 user-mode client library... DbgOut.c: TmpBuf.cpp: Create.c: Error.c: Memory.c: Print.c: DbgLink.cpp: Trace.c: Updating version information... Reading file info from VerFile.h... Linking Out/Lnx_armsf_Dbg/libDbgOut11.a ... Linux user-mode client library build complete. Building DbgOut11 kernel-mode client library... Building libDbgOutKrnl.a into Out/Lnx__armsf_Dbg DbgOutLink.c: Collating output library... -rw-rw-r-- 1 root root 60K Mar 4 11:36 Out/Lnx__armsf_Dbg/libDbgOutKrnl.a Linux kernel-mode example build complete. Building ExConsole application... Main.c: Updating version information... Reading file info from VerFile.h... Linking Out/Lnx_armsf_Dbg/ExConsole ... Linux user-mode example build complete. Building ExDriver module... Building ExDriver.ko into Out/Lnx__armsf_Dbg DrvEntry.c: Initial module link... Generating Kernel linker source... Compiling Kernel linker object... In file included from /src/yocto/overo/build/tmp/work-shared/overo/kernel-source/include/linux/kobject.h:21:0, from /src/yocto/overo/build/tmp/work-shared/overo/kernel-source/include/linux/module.h:16, from Out/Lnx__armsf_Dbg/_ExDriver.mod.c:1: /src/yocto/overo/build/tmp/work-shared/overo/kernel-source/include/linux/sysfs.h: In function 'sysfs_get_dirent': /src/yocto/overo/build/tmp/work-shared/overo/kernel-source/include/linux/sysfs.h:457:37: warning: pointer targets in passing argument 2 of 'kernfs_find_and_get' differ in signedness [-Wpointer-sign] return kernfs_find_and_get(parent, name); ^ In file included from /src/yocto/overo/build/tmp/work-shared/overo/kernel-source/include/linux/sysfs.h:15:0, from /src/yocto/overo/build/tmp/work-shared/overo/kernel-source/include/linux/kobject.h:21, from /src/yocto/overo/build/tmp/work-shared/overo/kernel-source/include/linux/module.h:16, from Out/Lnx__armsf_Dbg/_ExDriver.mod.c:1: /src/yocto/overo/build/tmp/work-shared/overo/kernel-source/include/linux/kernfs.h:411:1: note: expected 'const char *' but argument is of type 'const unsigned char *' kernfs_find_and_get(struct kernfs_node *kn, const char *name) ^ Final Kernel link... -rw-rw-r-- 1 root root 94K Mar 4 11:36 Out/Lnx__armsf_Dbg/ExDriver.ko Linux kernel-mode example build complete. Building DbgOutRelay application... Config.cpp: Main.c: Relay.cpp: LiveRelay.cpp: Lnx/Console.cpp: Lnx/Server.cpp: Lnx/Store.cpp: Lnx/WinMain.c: Lnx/LiveSkt.cpp: Updating version information... Reading file info from Lnx/VerFile.h... Linking Out/Lnx_armsf_Dbg/DbgOutRelay ... Linux Relay build complete. Building DbgOut11.ko kernel module for ... Building DbgOut11.ko into Out/Lnx__armsf_Dbg LnxKrn/DrvEntry.c: Client.cpp: Config.cpp: ItemFifo.cpp: Main.c: Main.c: In function 'DebugMem': Main.c:107:16: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] UINT64 Addr= (UINT64)pvMem; ^ Main.c:124:13: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] Addr= (UINT64)&pMem[n1]; ^ PrintfA.cpp: PrintfW.cpp: Trace.cpp: LnxKrn/StringA.c: LnxKrn/StringW.c: LnxKrn/Memory.c: LnxKrn/Sync.c: LnxKrn/Time.c: LnxKrn/Error.c: LnxKrn/Thread.c: TmpBuf.cpp: LnxKrn/Tick.c: LnxKrn/Device.c: LnxKrn/DevFile.c: LnxKrn/ProcFile.c: Status.cpp: LnxKrn/KrnlApi.c: LnxKrn/CfgStore.cpp: Initial module link... Reading file info from LnxKrn/VerFile.h... Generating Kernel linker source... Compiling Kernel linker object... Final Kernel link... -rw-rw-r-- 1 root root 583K Mar 4 11:36 Out/Lnx__armsf_Dbg/DbgOut11.ko Linux Server build complete. Linux build complete. 11:36:24 00:00:20 1102>

DbgOut on the Target Overo

11:40> It is time to see if all this ugly hacking created anything usable. Copy DbgOut11.ko, DbgOutRelay, ExConsole, and ExDriver to the Overo.

OveroVM: scp Out/Lnx_armsf_Dbg/DbgOutRelay root@192.168.0.213:/home/root/ scp Out/Lnx_armsf_Dbg/ExConsole root@192.168.0.213:/home/root/ scp Out/Lnx__armsf_Dbg/DbgOut11.ko root@192.168.0.213:/home/root/ scp Out/Lnx__armsf_Dbg/ExDriver.ko root@192.168.0.213:/home/root/

Sanity check to make sure everything loads. The printk output is visible only from a serial login session. It can be viewed from a telnet or ssh session using dmesg.

Overo: root@overo:~# insmod DbgOut11.ko [ 4785.496337] DbgOut11 kernel has arrived... [ 4785.500701] DbgOut11 Server has arrived! [ 4785.504821] DbgOut11 1.2.267 root@ubuntu 20160304113624 [ 4785.523712] DbgOutServer: Device 'DbgOutServer' created. [ 4785.529449] Fifo:Create1 [ 4785.532104] [ 4785.533660] Fifo:Create2 [ 4785.536376] [ 4785.537933] Fifo:Create2() [ 4785.540802] [ 4785.542358] Fifo:Create2: Creating 32 blocks of 16384 bytes... [ 4785.548522] [ 4785.550811] Fifo:Create2: Created 32 blocks of 16384 bytes, 524288 total [ 4785.557891] [ 4785.559570] Created 8 trace blocks of 4096 bytes, 32768 total. [ 4785.565734] Trace flush level set to 2620 events. [ 4785.570678] Trace:Create2: DBG_TRACE [ 4785.574432] DbgOut Server creation is complete! root@overo:~# insmod ExDriver.ko [ 4917.252532] ExDriver [Mar 4 2016 11:36:16] has arrived... [ 4917.258453] DbgOutCreate: ClntApi=11020003 'ExDriver'DbgOutCmdCreate: Unable to open '/proc/DbgOut11/KernelAPI.bin' [ 4917.269653] DbgOutCmdCreate: DbgOut module not present.

The KernelAPI file is not working, I will fix that later. It is promising that the ExDriver module loads.

Copy DbgOutRelay.exe and DbgViewer.exe to [Pogo]C:\Share\DbgOut\ to remove any VM or firewall obstacles. Launch DbgOutRelay on Pogo and Overo. Running DbgOut. Pogo is at 192.168.0.201.

DbgOutRelay: DbgOutRelay --server LISTENTCP:192.168.0.201 --datastore DataStore --notimestamp root@overo:~# ./DbgOutRelay --server DEVICE --datastore TCP:192.168.0.201 & [2] 1967 [1] Done(127) DbgOutRelay --server DEVICE --datastore TCP:192.168.0.201 root@overo:~# DbgOut Relay has arrived! DbgOutRelay 1.2.[ 5641.486938] DeviceOpen150 1.2.150 [root@ubuntu] ServerDev:Create: DbgOut has been created. Connected[TCP] to 192.168.0.201[192.168.0.201:7710]

Launch ExConsole...

Overo: root@overo:~# ./ExConsole DbgOut sample application. ExConsole 1.0.177 1.0.177 [root[ 5687.345825] DeviceOpen@ubuntu] Creating DbgOut interface... [ 5687.356140] FUNCA: 1 Fast [ 5687.358917] FUNCA: 2 Print [ 5687.361785] FUNCA: 3 Multiply [ 5687.374694] FUNCA: 4 LongTest DbgOutCreate: 38.909ms PrintFast: 15.350ms [ 5687.400054] Trace:CmdEnable: 1 TraceFast: 22.339ms [ 5687.439056] Trace block list: WrBlk= 6/8 [ 5687.443206] EventCt=2621 In=26210 Out=0 [ 5687.447540] EventCtSinceFlush=2621 FlushLvl=2620 [ 5687.452606] 0[1]: 1 4050 [ 5687.455596] 1[2]: 1 4050 [ 5687.458526] 2[3]: 1 4050 [ 5687.461486] 3[4]: 1 4050 [ 5687.464477] 4[5]: 1 4050 [ 5687.467437] 5[6]: 1 4050 [ 5687.470367] 6[7]: 0 1910 [ 5687.473358] 7[0]: 0 0 [ 5687.476196] FUNCA: 4 names, 184 bytes. TracePrint: 165.219ms Multiply Normally: 6.256ms Final multiply: 999999000000000 Multiply shift/add: 41.992ms Final multiply: [ 5687.645294] Trace block list: WrBlk= 3/8999999000000000 Destroying DbgOut interface... [ 5687.657318] EventCt=4008 In=40080 Out=26210 [ 5687.662048] EventCtSinceFlush=1387 FlushLvl=2620 [ 5687.667175] 0[8]: 1 4050 [ 5687.670104] 1[9]: 1 4050 [ 5687.673034] 2[10]: 1 4050 [ 5687.676147] 3[11]: 0 1720 [ 5687.679168] 4[5]: 0 0 [ 5687.681823] 5[6]: 0 0 [ 5687.684539] 6[7]: 0 0 [ 5687.687194] 7[0]: 0 0 Main: 375.573ms [ 5687.719696] DeviceClose0] root@overo:~#

Close DbgOutRelay and launch DbgViewer.
DbgViewer --datastore . --open DataStore --theme UIDefault.txt
Set the clock: set clock 1G 720M
Click over to the scope view, and bask in the glory:

First DbgOut data capture from Overo Linux 3.18  DbgOut on Overo 

This version uses the Linux system timer, not the hardware cycle counter. The trace clearly shows a tick resolution of only 30.5us, which I happen to know is driven by the ARM's onboard 32KHz counter. The system time is updated when the 32KHz counter increments (32,768 times per second or every 30.5us). This is not good enough, I need to use the cycle counter directly. Cycle Counting

Success!

12:05 (Friday)> DbgOut11 is now working on Overo Linux 3.18.



WebV7 (C)2018 nlited | Rendered by tikope in 171.386ms | 18.118.28.135