dev.nlited.com

| Contact |  Home | 
>>

Yocto From Scratch

<<<< prev
next >>>>

2015-12-22 04:08:40 chip Page 1475 📢 PUBLIC

Mon Dec 21 2015

I am following the tutorial on hackgnar.com to get started. Building "from scratch" using Yocto is a bit of a misnomer, as I will be creating almost nothing. Yocto is a build framework that allows me to mix and match projects written and published by others into a my own custom boot disk.


Mistake icon

I am documenting the entire process, including my mistakes. To skip over these steps, jump to to create the build machine, or Yocto Speed Run to jump straight into using Yocto.


First I need a build machine: Creating Vekolako

Then I need to create the builder account: Linux: Creating a New User

21:44> Now I have a usable chip profile, and I still have ubuntu in reserve in case the chip profile becomes unusable.

I create a yocto directory on the 80GB SSD that will hold my development environment.


vekolako: ~> sudo mkdir /mnt/yocto ~> sudo chown chip /mnt/yocto ~> ln -s /mnt/yocto yocto ~> cd yocto yocto>

Yocto Bootstrap

21:50> Back to the yocto tutorial...


vekolako: 05:49:29 yocto> sudo apt-get install -y gawk wget git-core diffstat unzip texinfo gcc-multilib build-essential chrpath libsdl1.2-dev xterm curl lots of 404 not found errors 05:51:41 yocto> sudo apt-get update 05:52:24 yocto> sudo apt-get install -y gawk wget git-core diffstat unzip texinfo gcc-multilib build-essential chrpath libsdl1.2-dev xterm curl much better 05:54:15 yocto> mkdir overo 05:55:21 yocto> cd overo 05:55:23 overo> curl -O http://commondatastorage.googleapis.com/git-repo-downloads/repo > repo 05:56:34 overo> chmod a+x repo 05:56:48 overo> sudo mv repo /usr/local/bin 05:57:07 overo> repo init -u git://github.com/gumstix/Gumstix-YoctoProject-Repo.git GitHub won't let me fetch without an email address *** Please tell me who you are. Run git config --global user.email "you@example.com" git config --global user.name "Your Name" to set your account's default identity. Omit --global to set the identity only in this repository. fatal: empty ident name (for ) not allowed 05:57:47 overo> git config --global user.email "chip.yocto@nlited.com" 05:59:19 overo> git config --global user.name "Chip Doran" 05:59:30 overo> repo init -u git://github.com/gumstix/Gumstix-YoctoProject-Repo.git repo has been initialized in /mnt/yocto/overo 06:00:07 overo> repo sync Super-fast, 12MB/s Fetching projects: 100% (7/7), done. Syncing work tree: 100% (7/7), done. 06:01:10 overo> export TEMPLATECONF=meta-gumstix-extras/conf 06:02:16 overo> source ./poky/oe-init-build-env Let's make this monster earn its paycheck... 06:02:28 build> export PARALLEL_MAKE="-j 32" 06:03:21 build> bitbake -c fetchall gumstix-console-image 06:13:15 build> bitbake gumstix-console-image NOTE: Started PRServer with DBfile: /mnt/yocto/overo/build/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 50864, PID: 19981 Loading cache: 100% |############################################| ETA: 00:00:00 Loaded 2836 entries from dependency cache. Parsing recipes: 100% |##########################################| Time: 00:00:01 Parsing of 2300 .bb files complete (2297 cached, 3 parsed). 2838 targets, 150 skipped, 9 masked, 0 errors. NOTE: Resolving any missing task queue dependencies Build Configuration: BB_VERSION = "1.26.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "Ubuntu-14.04" TARGET_SYS = "arm-poky-linux-gnueabi" MACHINE = "overo" DISTRO = "poky" DISTRO_VERSION = "1.8" TUNE_FEATURES = "arm armv7a vfp neon callconvention-hard cortexa8" TARGET_FPU = "vfp-neon" meta meta-yocto = "(nobranch):83aa565d93aacae484976562ef1ae8dbbb6b2bc0" meta-gnome meta-oe meta-xfce meta-systemd meta-networking meta-multimedia meta-python = "(nobranch):10d3c8f85280a0bf867a8e4f84bcda81c290d28e" meta-gumstix = "(nobranch):68e4b5771aef2d11fcf04bb8759fecd409de1ac3" meta-gumstix-extras = "(nobranch):0ee78d47ff3082000e9e087b1ec0d56df13457d9" meta-ros = "(nobranch):d465a23c0a96e651419d46b29ab7c0c3663da314" meta-ti = "(nobranch):03cb3ff15f3b8535ca409ce682f44ec1b81da0c3" meta-java = "(nobranch):66c97ae7461f4c1a13917131787bb76dc34e6b6f" NOTE: Preparing RunQueue NOTE: Executing SetScene Tasks NOTE: Executing RunQueue Tasks Currently 8 running tasks (591 of 3792): 0: gettext-native-0.19.4-r0 do_configure (pid 12869) 1: rpm-native-5.4.14-r0 do_configure (pid 8267) 2: perl-native-5.20.0-r0 do_configure (pid 23868) 3: linux-gumstix-3.18-r0 do_patch (pid 11711) 4: gcc-cross-initial-arm-4.9.2-r0 do_compile (pid 23125) 5: alsa-lib-native-1.0.28-r0 do_configure (pid 10089) 6: libxml2-native-2.9.2-r0 do_compile (pid 16940) 7: libpng-native-1.6.16-r0 do_compile (pid 24026) WARNING: /mnt/yocto/overo/build/tmp/work/all-poky-linux/volatile-binds/1.0-r0/volatile-binds-1.0 ('S') doesn't exist, please set 'S' to a proper value WARNING: /mnt/yocto/overo/build/tmp/work/overo-poky-linux-gnueabi/systemd-serialgetty/1.0-r5/systemd-serialgetty-1.0 ('S') doesn't exist, please set 'S' to a proper value WARNING: /mnt/yocto/overo/build/tmp/work/cortexa8hf-vfp-neon-poky-linux-gnueabi/nmon/13g-r2/nmon-13g ('S') doesn't exist, please set 'S' to a proper value WARNING: cpufrequtils-dev-008 was registered as shlib provider for libcpufreq.so.0, changing it to cpufrequtils-008 because it was built later WARNING: log_check: There is a warn message in the logfile WARNING: log_check: Matched keyword: [WARNING:] WARNING: log_check: WARNING: Staat_der_Nederlanden_Root_CA_-_G3.pem does not contain a certificate or CRL: skipping WARNING: log_check: There is a warn message in the logfile WARNING: log_check: Matched keyword: [WARNING:] WARNING: log_check: WARNING: USERTrust_RSA_Certification_Authority.pem does not contain a certificate or CRL: skipping WARNING: log_check: There is a warn message in the logfile WARNING: log_check: Matched keyword: [WARNING:] WARNING: log_check: WARNING: S-TRUST_Universal_Root_CA.pem does not contain a certificate or CRL: skipping WARNING: log_check: There is a warn message in the logfile WARNING: log_check: Matched keyword: [WARNING:] WARNING: log_check: WARNING: IdenTrust_Commercial_Root_CA_1.pem does not contain a certificate or CRL: skipping WARNING: log_check: There is a warn message in the logfile WARNING: log_check: Matched keyword: [WARNING:] WARNING: log_check: WARNING: Staat_der_Nederlanden_EV_Root_CA.pem does not contain a certificate or CRL: skipping WARNING: log_check: There is a warn message in the logfile WARNING: log_check: Matched keyword: [WARNING:] WARNING: log_check: WARNING: USERTrust_ECC_Certification_Authority.pem does not contain a certificate or CRL: skipping WARNING: log_check: There is a warn message in the logfile WARNING: log_check: Matched keyword: [WARNING:] WARNING: log_check: WARNING: GlobalSign_ECC_Root_CA_-_R5.pem does not contain a certificate or CRL: skipping WARNING: log_check: There is a warn message in the logfile WARNING: log_check: Matched keyword: [WARNING:] WARNING: log_check: WARNING: Entrust_Root_Certification_Authority_-_G2.pem does not contain a certificate or CRL: skipping WARNING: log_check: There is a warn message in the logfile WARNING: log_check: Matched keyword: [WARNING:] WARNING: log_check: WARNING: Entrust_Root_Certification_Authority_-_EC1.pem does not contain a certificate or CRL: skipping WARNING: log_check: There is a warn message in the logfile WARNING: log_check: Matched keyword: [WARNING:] WARNING: log_check: WARNING: GlobalSign_ECC_Root_CA_-_R4.pem does not contain a certificate or CRL: skipping WARNING: log_check: There is a warn message in the logfile WARNING: log_check: Matched keyword: [WARNING:] WARNING: log_check: WARNING: COMODO_RSA_Certification_Authority.pem does not contain a certificate or CRL: skipping WARNING: log_check: There is a warn message in the logfile WARNING: log_check: Matched keyword: [WARNING:] WARNING: log_check: WARNING: IdenTrust_Public_Sector_Root_CA_1.pem does not contain a certificate or CRL: skipping WARNING: log_check: There is a warn message in the logfile WARNING: log_check: Matched keyword: [WARNING:] WARNING: log_check: WARNING: CFCA_EV_ROOT.pem does not contain a certificate or CRL: skipping WARNING: log_check: There is a warn message in the logfile WARNING: log_check: Matched keyword: [WARNING:] WARNING: log_check: WARNING: The license listed Firmware-Marvell was not in the licenses collected for linux-firmware WARNING: log_check: There is a warn message in the logfile WARNING: log_check: Matched keyword: [WARNING:] WARNING: log_check: WARNING: The license listed Firmware-Marvell was not in the licenses collected for linux-firmware WARNING: log_check: There is a warn message in the logfile WARNING: log_check: Matched keyword: [WARNING:] WARNING: log_check: WARNING: The license listed Firmware-ti-connectivity was not in the licenses collected for linux-firmware WARNING: log_check: There is a warn message in the logfile WARNING: log_check: Matched keyword: [WARNING:] WARNING: log_check: WARNING: The license listed Firmware-ti-connectivity was not in the licenses collected for linux-firmware NOTE: Tasks Summary: Attempted 3792 tasks of which 512 didn't need to be rerun and all succeeded. Summary: There were 55 WARNING messages shown. 07:19:15 build>

The downloads in the fetchall command were done in less than a minute, most of the time was spent compiling the build tools. It looks like my decision to go with AWS is paying off so far.

22:15> The initial build will take at least an hour, even on a super-computer. Time to take a break and let vekolako churn through the build.

22:16> It appears bitbake is limited to 8 build tasks.

23:19> The initial build completed in just over one hour, or about 1:15 if including the prefetch. The build used about 30GB of disk space.


vekolako: 07:43:20 overo> df -h Filesystem Size Used Avail Use% Mounted on /dev/xvda1 7.8G 1.4G 6.0G 19% / none 4.0K 0 4.0K 0% /sys/fs/cgroup udev 7.4G 12K 7.4G 1% /dev tmpfs 1.5G 352K 1.5G 1% /run none 5.0M 0 5.0M 0% /run/lock none 7.4G 0 7.4G 0% /run/shm none 100M 0 100M 0% /run/user /dev/xvdb 79G 29G 47G 38% /mnt

Now I need to move the boot image files down to my local computer.


vekolako: 07:21:28 build> cd tmp/deploy/images/overo 07:22:50 overo> pwd /mnt/yocto/overo/build/tmp/deploy/images/overo
VS12: Z:\Share\overo>pscp -load vekolako chip@54.153.11.107:/home/chip/yocto/overo/build/tmp/deploy/images/overo/MLO . MLO | 56 kB | 56.2 kB/s | ETA: 00:00:00 | 100%

I need to copy MLO (bootstrap 57KB), u-boot.img (bootloader 459KB), zImage (linux kernel 4.6MB), and gumstix-console-image-overo.tar.bz2 (rootfs 62MB). UBI (101MB) and UBIFS (100MB) versions of rootfs were also built. Downloading takes about 10 minutes.

Now I need to copy my mkcard.sh script from the Nirvana project and modify it to use the slightly different file names.

I started with nothing but an AWS account at 20:42 and built a complete build machine and boot disk in about 3 hours. Very good, if it works!

However, it is late and I am tired. It is best to finish this job in the morning when I am less likely to make mistakes and less apt to become frustrated if the boot disk doesn't work on the first try.

I don't want to leave vekolako running, as I am paying for each cpu cycle.
07:50:17 overo> sudo shutdown now
Although I can no longer connect to vekolako, the EC2 page still shows it as running. I guess I need to explicitly stop the instance from the EC2 control panel? Restarting the instance from the EC2 page takes about 30 seconds to appear as "running".

Mistake icon

YIPES! First I noticed that the instance restarted with a completely different IP address, then I found I had lost everything on the SSD drive! The /mnt directory is completely empty after the restart! Apparently the SSD is considered "ephemeral storage". My chip user configuration persisted, so the 8GB /dev/sda volume is persistent. Tomorrow I will need to do some research on exactly what vekolako provides, possibly create a new elastic volume for it, experiment with stopping the instance and restarting it, then rerun the yocto build. I really only lost about an hour of compute time.

Blog post giving an in-depth explanation of what is meant by "ephemeral storage". For instance, as they are a temporary storage, you should not rely on these disks to keep long-term data or even other data that you would not like to lose when a failure happens (i.e stop/start instance, failure on the underlying hardware, terminating instance), for these purposes please bear in mind that EBS/S3 or any other persistent storage will be your best choice.


I need to create a dedicated Elastic Block Store volume of 80GB, allocate an Elastic IP address (on the 172.31.X.X subnet), and attach them to vekolako. Then create a test file on the 80GB volume, stop the instance, restart it, and verify that the IP address doesn't change and that the test file survives. Then I can rebuild the yocto environment.

Dec 22 2015

While I was waiting for the rebuilt vekolako to build again, I modified the mknirvanacard.sh script mkboot and cooked a boot disk from the image I created last night. Miraculously, it booted up!
root@overo:~# uname -a Linux overo 3.18.18-custom #1 SMP Tue Dec 22 06:43:36 UTC 2015 armv7l GNU/Linux root@overo:~#

SUCCESS!


WebV7 (C)2018 nlited | Rendered by tikope in 34.611ms | 3.138.119.248