Show Posts
1
Cosmo Communicator - Linux / On-device kernel compilation
« on: April 26, 2021, 10:21:12 pm »
This post is modified to capture improvements made for latest version (v4) of the Cosmo Debian/KDE installer: https://support.planetcom.co.uk/index.php/Linux_for_Cosmo
The whole process is now seamless, greatly simplified and is based on the gemian wiki entry for kernel compilation https://github.com/gemian/gemian/wiki/KernelCompilation.
Steps:
1. Install version 4 of the the Cosmo Debian/KDE image (first link above).
2. Follow instructions from the second link above but install the dependencies before pulling down the kernel sources and add git (needed for pulling down the sources) in the dependencies.
3. Happy (kernel) hacking.
Thanks to Adam Boardman for providing the links/missing pieces.
==============================Relevant to v3 of the debian installer only and left here for record and info===================
This post is an adaption to on-device compilation of the steps described in:
https://www.oesf.org/forum/index.php?topic=36321.0
It shows all the steps of arriving at the on-device kernel compile capability. I am not an expert at any of this and appreciate any useful suggestions (especially as far as the device tree compiler is concerned, if you know the actual sources used, please let me know).
Make sure you have enough space to compile the kernel when partitioning your cosmo. The minimum that works for me is 60 GB Android / 60 GB Linux partitioning scheme (90/30 crashed the kernel compile).
Set-up your environment as described in the article above:
Download the kernel source:
Edit the Makefile:
Find the line with KBUILD_CFLAGS (line 398) and edit from:
Import the kernel configuration:
This command results in a few warnings, but does not result in a bad kernel compile:
Configure the kernel:
Start compilation:
This will do the majority of the kernel compile, but eventually (~1 hour of happy compiling) results in an error that needs to be fixed:
Fix:
This fix involves a native compile of the device tree compiler and associated libraries, since the version in the kernel is compiled for X86-64, but the Cosmo needs AArch64:
Get the source code for the device tree compiler:
Download some needed dependencies:
Compile:
Also need a native compile of ufdt and associated libraries, since the version in the kernel is compiled for X86-64, but the Cosmo needs AArch64 (ie. similar problem as last error):
Get libufdt source code:
Compile the libufdt_sysdeps library:
Compile the libufdt library:
Compile the driver program:
Back to the main compile task:
This results in a clean compile (after ~5min):
The whole process is now seamless, greatly simplified and is based on the gemian wiki entry for kernel compilation https://github.com/gemian/gemian/wiki/KernelCompilation.
Steps:
1. Install version 4 of the the Cosmo Debian/KDE image (first link above).
2. Follow instructions from the second link above but install the dependencies before pulling down the kernel sources and add git (needed for pulling down the sources) in the dependencies.
3. Happy (kernel) hacking.
Thanks to Adam Boardman for providing the links/missing pieces.
==============================Relevant to v3 of the debian installer only and left here for record and info===================
This post is an adaption to on-device compilation of the steps described in:
https://www.oesf.org/forum/index.php?topic=36321.0
It shows all the steps of arriving at the on-device kernel compile capability. I am not an expert at any of this and appreciate any useful suggestions (especially as far as the device tree compiler is concerned, if you know the actual sources used, please let me know).
Make sure you have enough space to compile the kernel when partitioning your cosmo. The minimum that works for me is 60 GB Android / 60 GB Linux partitioning scheme (90/30 crashed the kernel compile).
Set-up your environment as described in the article above:
Code: [Select]
sudo apt install build-essential libncurses-dev bison flex libssl-dev libelf-dev bc git gcc-aarch64-linux-gnu vim
mkdir KERNEL_OUTDownload the kernel source:
Code: [Select]
git clone https://github.com/gemian/cosmo-linux-kernel-4.4.gitEdit the Makefile:
Code: [Select]
cd cosmo-linux-kernel-4.4
vim MakefileFind the line with KBUILD_CFLAGS (line 398) and edit from:
Code: [Select]
KBUILD_CFLAGS := -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs \toCode: [Select]
KBUILD_CFLAGS := -w -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs \and save the editImport the kernel configuration:
Code: [Select]
make O=../KERNEL_OUT ARCH=arm64 k71v1_64_bsp_defconfigThis command results in a few warnings, but does not result in a bad kernel compile:
Code: [Select]
cosmo@cosmopda:~/cosmo-linux-kernel-4.4$ make O=../KERNEL_OUT ARCH=arm64 k71v1_64_bsp_defconfig
make[1]: Entering directory '/home/cosmo/KERNEL_OUT'
HOSTCC scripts/basic/fixdep
GEN ./Makefile
HOSTCC scripts/kconfig/conf.o
SHIPPED scripts/kconfig/zconf.tab.c
SHIPPED scripts/kconfig/zconf.lex.c
SHIPPED scripts/kconfig/zconf.hash.c
HOSTCC scripts/kconfig/zconf.tab.o
HOSTLD scripts/kconfig/conf
drivers/input/touchscreen/mediatek/NT36xxx/Kconfig:4:warning: ignoring type redefinition of 'TOUCHSCREEN_MTK_NT36xxx' from 'boolean' to 'tristate'
drivers/input/fingerprint/chips_driver/Kconfig:1:warning: ignoring type redefinition of 'MTK_FINGERPRINT_CHIPSAILING_CS358' from 'tristate' to 'boolean'
drivers/input/fingerprint/aw6302_driver/Kconfig:1:warning: ignoring type redefinition of 'MTK_SIMSWAP_AW6302' from 'tristate' to 'boolean'
warning: (MTK_GROUP_ENABLE_VERITY) selects INTEGRITY_ASYMMETRIC_KEYS which has unmet direct dependencies (INTEGRITY && INTEGRITY_SIGNATURE)
warning: (MACH_MT6771 && MACH_MT6775 && PINCTRL_MT6739 && PINCTRL_MT6775) selects MTK_EINT_MULTI_TRIGGER_DESIGN which has unmet direct dependencies (PINCTRL && PINCTRL_MTK)
warning: (MTK_GROUP_ENABLE_VERITY) selects INTEGRITY_ASYMMETRIC_KEYS which has unmet direct dependencies (INTEGRITY && INTEGRITY_SIGNATURE)
warning: (MACH_MT6771 && MACH_MT6775 && PINCTRL_MT6739 && PINCTRL_MT6775) selects MTK_EINT_MULTI_TRIGGER_DESIGN which has unmet direct dependencies (PINCTRL && PINCTRL_MTK)
#
# configuration written to .config
#
make[1]: Leaving directory '/home/cosmo/cosmo/KERNEL_OUT'
Configure the kernel:
Code: [Select]
make O=../KERNEL_OUT ARCH=arm64 menuconfigHit save, ok, exit. exit;). Again, same warnings as above, but the kernel compile can continue.Start compilation:
Code: [Select]
make O=../KERNEL_OUT ARCH=arm64This will do the majority of the kernel compile, but eventually (~1 hour of happy compiling) results in an error that needs to be fixed:
Code: [Select]
CHK include/generated/compile.h
UPD include/generated/compile.h
CC init/version.o
LD init/built-in.o
KSYM .tmp_kallsyms1.o
KSYM .tmp_kallsyms2.o
LD vmlinux
SORTEX vmlinux
SYSMAP System.map
for i in /home/cosmo/cosmo-linux-kernel-4.4/arch/arm64/boot/dts/mediatek/k71v1_64_bsp.dts /home/cosmo/cosmo-linux-kernel-4.4/arch/arm64/boot/dts/mediatek/k71v1_64_bsp.dts /home/cosmo/cosmo-linux-kernel-4.4/arch/arm64/boot/dts/mediatek/k71v1_64_bsp.dts; do \
base_prj=`grep -m 1 '#include [<\"].*\/cust\.dtsi[>\"]' $i | sed 's/#include [<"]//g'\
| sed 's/\/cust\.dtsi[>"]//g' | sed 's/\/\*//g' | sed 's/\*\///g' | sed 's/ //g'`\
prj_path=./arch/arm64/boot/dts/$base_prj ;\
dws_path=/home/cosmo/cosmo-linux-kernel-4.4/drivers/misc/mediatek/dws/mt6771/$base_prj.dws ;\
if [ -f $dws_path ] ; then \
mkdir -p $prj_path ;\
/home/cosmo/cosmo-linux-kernel-4.4/tools/dct/DrvGen.py $dws_path $prj_path $prj_path cust_dtsi;\
fi \
done
[DCT_INFO]: DWS file path is /home/cosmo/cosmo-linux-kernel-4.4/drivers/misc/mediatek/dws/mt6771/k71v1_64_bsp.dws
[DCT_INFO]: Gen files path is /home/cosmo/KERNEL_OUT/arch/arm64/boot/dts/k71v1_64_bsp
[DCT_INFO]: Log files path is /home/cosmo/KERNEL_OUT/arch/arm64/boot/dts/k71v1_64_bsp
[DCT_INFO]: Parameter is cust_dtsi
[DCT_INFO]: chip id: MT6771
[DCT_INFO]: Chip ID : MT6771
[DCT_INFO]: Project Info: Sylvia
[DCT_INFO]: Start to generate cust_dtsi file...
[DCT_INFO]: Generate cust_dtsi file successfully!
[DCT_INFO]: DWS file path is /home/cosmo/cosmo-linux-kernel-4.4/drivers/misc/mediatek/dws/mt6771/k71v1_64_bsp.dws
[DCT_INFO]: Gen files path is /home/cosmo/KERNEL_OUT/arch/arm64/boot/dts/k71v1_64_bsp
[DCT_INFO]: Log files path is /home/cosmo/KERNEL_OUT/arch/arm64/boot/dts/k71v1_64_bsp
[DCT_INFO]: Parameter is cust_dtsi
[DCT_INFO]: chip id: MT6771
[DCT_INFO]: Chip ID : MT6771
[DCT_INFO]: Project Info: Sylvia
[DCT_INFO]: Start to generate cust_dtsi file...
[DCT_INFO]: Generate cust_dtsi file successfully!
[DCT_INFO]: DWS file path is /home/cosmo/cosmo-linux-kernel-4.4/drivers/misc/mediatek/dws/mt6771/k71v1_64_bsp.dws
[DCT_INFO]: Gen files path is /home/cosmo/KERNEL_OUT/arch/arm64/boot/dts/k71v1_64_bsp
[DCT_INFO]: Log files path is /home/cosmo/KERNEL_OUT/arch/arm64/boot/dts/k71v1_64_bsp
[DCT_INFO]: Parameter is cust_dtsi
[DCT_INFO]: chip id: MT6771
[DCT_INFO]: Chip ID : MT6771
[DCT_INFO]: Project Info: Sylvia
[DCT_INFO]: Start to generate cust_dtsi file...
[DCT_INFO]: Generate cust_dtsi file successfully!
for i in /home/cosmo/cosmo-linux-kernel-4.4/arch/arm64/boot/dts/mediatek/k71v1_64_bsp.dts /home/cosmo/cosmo-linux-kernel-4.4/arch/arm64/boot/dts/mediatek/k71v1_64_bsp.dts /home/cosmo/cosmo-linux-kernel-4.4/arch/arm64/boot/dts/mediatek/k71v1_64_bsp.dts; do \
base_prj=`grep -m 1 '#include [<\"].*\/cust\.dtsi[>\"]' $i | sed 's/#include [<"]//g'\
| sed 's/\/cust\.dtsi[>"]//g' | sed 's/\/\*//g' | sed 's/\*\///g' | sed 's/ //g'`\
prj_path=./arch/arm64/boot/dts/$base_prj ;\
dws_path=/home/cosmo/cosmo-linux-kernel-4.4/drivers/misc/mediatek/dws/mt6771/$base_prj.dws ;\
if [ -f $dws_path ] ; then \
mkdir -p $prj_path ;\
/home/cosmo/cosmo-linux-kernel-4.4/tools/dct/DrvGen.py $dws_path $prj_path $prj_path cust_dtsi;\
fi \
done
[DCT_INFO]: DWS file path is /home/cosmo/cosmo-linux-kernel-4.4/drivers/misc/mediatek/dws/mt6771/k71v1_64_bsp.dws
[DCT_INFO]: Gen files path is /home/cosmo/KERNEL_OUT/arch/arm64/boot/dts/k71v1_64_bsp
[DCT_INFO]: Log files path is /home/cosmo/KERNEL_OUT/arch/arm64/boot/dts/k71v1_64_bsp
[DCT_INFO]: Parameter is cust_dtsi
[DCT_INFO]: chip id: MT6771
[DCT_INFO]: Chip ID : MT6771
[DCT_INFO]: Project Info: Sylvia
[DCT_INFO]: Start to generate cust_dtsi file...
[DCT_INFO]: Generate cust_dtsi file successfully!
[DCT_INFO]: DWS file path is /home/cosmo/cosmo-linux-kernel-4.4/drivers/misc/mediatek/dws/mt6771/k71v1_64_bsp.dws
[DCT_INFO]: Gen files path is /home/cosmo/KERNEL_OUT/arch/arm64/boot/dts/k71v1_64_bsp
[DCT_INFO]: Log files path is /home/cosmo/KERNEL_OUT/arch/arm64/boot/dts/k71v1_64_bsp
[DCT_INFO]: Parameter is cust_dtsi
[DCT_INFO]: chip id: MT6771
[DCT_INFO]: Chip ID : MT6771
[DCT_INFO]: Project Info: Sylvia
[DCT_INFO]: Start to generate cust_dtsi file...
[DCT_INFO]: Generate cust_dtsi file successfully!
[DCT_INFO]: DWS file path is /home/cosmo/cosmo-linux-kernel-4.4/drivers/misc/mediatek/dws/mt6771/k71v1_64_bsp.dws
[DCT_INFO]: Gen files path is /home/cosmo/KERNEL_OUT/arch/arm64/boot/dts/k71v1_64_bsp
[DCT_INFO]: Log files path is /home/cosmo/KERNEL_OUT/arch/arm64/boot/dts/k71v1_64_bsp
[DCT_INFO]: Parameter is cust_dtsi
[DCT_INFO]: chip id: MT6771
[DCT_INFO]: Chip ID : MT6771
[DCT_INFO]: Project Info: Sylvia
[DCT_INFO]: Start to generate cust_dtsi file...
[DCT_INFO]: Generate cust_dtsi file successfully!
DTC arch/arm64/boot/dts/mediatek/mt6771.dtb
/bin/sh: 1: /home/cosmo/cosmo-linux-kernel-4.4/scripts/dtc/dtc_overlay: Exec format error
make[2]: *** [scripts/Makefile.lib:302: arch/arm64/boot/dts/mediatek/mt6771.dtb] Error 2
make[1]: *** [arch/arm64/Makefile:126: mediatek/mt6771.dtb] Error 2
make[1]: Leaving directory '/home/cosmo/KERNEL_OUT'
make: *** [Makefile:152: sub-make] Error 2
Fix:
This fix involves a native compile of the device tree compiler and associated libraries, since the version in the kernel is compiled for X86-64, but the Cosmo needs AArch64:
Code: [Select]
readelf -h scripts/dtc/dtc_overlayGet the source code for the device tree compiler:
Code: [Select]
cd
git clone https://git.kernel.org/pub/scm/utils/dtc/dtc.gitDownload some needed dependencies:
Code: [Select]
sudo apt install pkg-configCompile:
Code: [Select]
cd dtc
sudo make install PREFIX=/usr
make check
cp dtc /home/cosmo/cosmo-linux-kernel-4.4/scripts/dtc/dtc_overlay
cdAlso need a native compile of ufdt and associated libraries, since the version in the kernel is compiled for X86-64, but the Cosmo needs AArch64 (ie. similar problem as last error):
Code: [Select]
readelf -h /home/cosmo/cosmo-linux-kernel-4.4/scripts/dtc/ufdt_apply_overlayGet libufdt source code:
Code: [Select]
cd
mkdir libufdt
cd libufdthttps://android.googlesource.com/platform/system/libufdt/+/refs/heads/master(use your favorite web browser, select tgz and save master.tar.gz in libufdt directory)Code: [Select]
tar -xvf master.tar.gzCompile the libufdt_sysdeps library:
Code: [Select]
cd sysdeps
gcc -shared libufdt_sysdeps_posix.c -Iinclude -fPIC -o libufdt_sysdeps.so
sudo cp libufdt_sysdeps.so /usr/libCompile the libufdt library:
Code: [Select]
cd ..
gcc -c ufdt_convert.c ufdt_node.c ufdt_node_pool.c ufdt_overlay.c ufdt_prop_dict.c -Iinclude -Isysdeps/include -fPIC
gcc -shared ufdt_convert.o ufdt_node.o ufdt_node_pool.o ufdt_overlay.o ufdt_prop_dict.o -lfdt -o libufdt.so
sudo cp libufdt.so /usr/libCompile the driver program:
Code: [Select]
cd tests/src
gcc ufdt_overlay_test_app.c util.c -I../../include -I../../sysdeps/include -lufdt -lufdt_sysdeps -o ufdt_apply_overlay
cp ufdt_apply_overlay /home/cosmo/cosmo-linux-kernel-4.4/scripts/dtc
cdBack to the main compile task:
Code: [Select]
cd cosmo/cosmo-linux-kernel-4.4
make O=../KERNEL_OUT ARCH=arm64This results in a clean compile (after ~5min):
Code: [Select]
mtk-Kbuild CONFIG_MALI_PLATFORM_THIRDPARTY_NAME := "mt6771"
*GPU defined ENABLE_MTK_DEBUG
imgsensor drv by common ../common/v1_1/s5k2x7_mipi_raw/ ../common/v1_1/sp5509_mipi_raw/ ../common/v1_1/gc5035_mipi_raw/ ../common/v1_1/gc5025_mipi_raw/
imgsensor drv by platform
build trng...
Building modules, stage 2.
MODPOST 26 modules
CC crypto/md4.mod.o
LD [M] crypto/md4.ko
CC crypto/xor.mod.o
LD [M] crypto/xor.ko
CC drivers/misc/mediatek/connectivity/fmradio/fmradio_drv.mod.o
LD [M] drivers/misc/mediatek/connectivity/fmradio/fmradio_drv.ko
CC drivers/misc/mediatek/connectivity/gps/gps_drv.mod.o
LD [M] drivers/misc/mediatek/connectivity/gps/gps_drv.ko
CC drivers/net/macvlan.mod.o
LD [M] drivers/net/macvlan.ko
CC drivers/net/usb/r8152.mod.o
LD [M] drivers/net/usb/r8152.ko
CC drivers/net/usb/rtl8150.mod.o
LD [M] drivers/net/usb/rtl8150.ko
CC fs/btrfs/btrfs.mod.o
LD [M] fs/btrfs/btrfs.ko
CC fs/cifs/cifs.mod.o
LD [M] fs/cifs/cifs.ko
CC fs/ecryptfs/ecryptfs.mod.o
LD [M] fs/ecryptfs/ecryptfs.ko
CC fs/fscache/fscache.mod.o
LD [M] fs/fscache/fscache.ko
CC fs/lockd/lockd.mod.o
LD [M] fs/lockd/lockd.ko
CC fs/nfs/nfs.mod.o
LD [M] fs/nfs/nfs.ko
CC fs/nfs/nfsv2.mod.o
LD [M] fs/nfs/nfsv2.ko
CC fs/nfs/nfsv3.mod.o
LD [M] fs/nfs/nfsv3.ko
CC fs/nfs/nfsv4.mod.o
LD [M] fs/nfs/nfsv4.ko
CC fs/nfs_common/grace.mod.o
LD [M] fs/nfs_common/grace.ko
CC fs/udf/udf.mod.o
LD [M] fs/udf/udf.ko
CC lib/crc-itu-t.mod.o
LD [M] lib/crc-itu-t.ko
CC lib/raid6/raid6_pq.mod.o
LD [M] lib/raid6/raid6_pq.ko
CC net/dns_resolver/dns_resolver.mod.o
LD [M] net/dns_resolver/dns_resolver.ko
CC net/ipv4/tcp_htcp.mod.o
LD [M] net/ipv4/tcp_htcp.ko
CC net/ipv4/tcp_westwood.mod.o
LD [M] net/ipv4/tcp_westwood.ko
CC net/sunrpc/auth_gss/auth_rpcgss.mod.o
LD [M] net/sunrpc/auth_gss/auth_rpcgss.ko
CC net/sunrpc/auth_gss/rpcsec_gss_krb5.mod.o
LD [M] net/sunrpc/auth_gss/rpcsec_gss_krb5.ko
CC net/sunrpc/sunrpc.mod.o
LD [M] net/sunrpc/sunrpc.ko
make[1]: Leaving directory '/home/cosmo/KERNEL_OUT'