Messages

16

Cosmo Communicator - Linux / Re: Anbox on Cosmo Linux

« on: April 27, 2021, 05:09:53 pm »

However I've got stuck with the building of the suggested android-7 image file. On the Debian 10 build server it complains about lacking tools.jar (related to wanting an older version of java). On my desktop which happens to have many versions of java - external/libavc/decoder/arm/ih264d_function_selector.c:67: error: undefined reference to 'ih264d_init_function_ptr_av8' clang++: error: linker command failed with exit code 1 (use -v to see invocation)

I forgot to add workaround steps to my instructions.  I just edited my orginal post.  I simply commented them out for now... I was in a hurry to get Anbox working one way or another!  Besides, I didn't plan on watching any H264 videos under Anbox...

I could probably debug them but ideally we'd be using a newer version of android, and given that we already have a full copy of lineage* running in an LXC container to allow us to access the hardware, it seems a bit odd to have to go back to something so old.

* - granted this is pending release by Planet.

I initially tried booting Anbox with a rootfs image I built around the system partition (/dev/mmcblk0p36 on /system_root), but couldn't get /init to stop aborting.  I worked around a number of issues with /init trying to mount things that were already mounted by droid-hal, plus a reboot caused by reloading the SELinux policy, but no matter what I did /init still just kept dying with SIGABRT.  At that point I gave up on trying to use the native on-device Android and compiled AOSP from scratch.  Besides, the Anbox build instructions (https://github.com/anbox/anbox/blob/master/docs/build-android.md) say: "For Anbox we're using a minimal customized version of Android but otherwise base all our work of a recent release of the Android Open Source Project."  I figured if they customized it for Anbox, I'd be better off trying their version.  And it worked.  My goal was to run Android apps under Linux at native or near-native speed, which we achieve when meshed with gl4es.  Aside from the camera (really wishing I could use my Google Camera port https://www.oesf.org/forum/index.php?topic=35912.msg297145#msg297145), I didn't have a use for direct hardware access other than GPS which I worked around.  It's a hack, yes, but the hack also lets me run gpsd and Anbox GPS apps simultaneously.

17

Cosmo Communicator - Linux / Re: Anbox on Cosmo Linux

« on: April 24, 2021, 08:01:47 am »

By popular demand (one reader), I'm posting my steps to compile and flash the kernel for the Cosmo.


How to compile and flash the Linux and Android kernels for the Cosmo

The following document describes how I compiled both the Linux and the Android kernels for the Cosmo.  I also explain how to create a rooted Android image.  I used an x86_64 Gentoo development machine for all work and nothing was compiled directly on the Cosmo.  I also used live kernel configs as a starting point, not predefined configs from the kernel tree, in an attempt to keep the configuration as close to stock as possible before customizing.


Step 1: Prepare the kernel development tree
Create a directory where you will perform all of your kernel development work:

Code: [Select]

mkdir -p /path/to/cosmo/kernel
Retrieve the Cosmo Linux kernel source code:

Code: [Select]

cd /path/to/cosmo/kernel
git clone https://github.com/gemian/cosmo-linux-kernel-4.4.git
Retrieve the Cosmo Android kernel source code:

Code: [Select]

cd /path/to/cosmo/kernel
git clone https://github.com/dguidipc/cosmo-android-kernel.git
Retrieve and build mkbootimg for manipulating ramdisk boot images:

Code: [Select]

cd /path/to/cosmo/kernel
git clone https://github.com/osm0sis/mkbootimg.git
cd mkbootimg
make
cd ..
Retrieve the prebuilt gcc arm64 cross compiler tools:

Code: [Select]

cd /path/to/cosmo/kernel
git clone https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9 -b nougat-release --depth 1
Retrieve Planet Computers' Linux v3 firmware:

Code: [Select]

cd /path/to/cosmo
wget http://support.planetcom.co.uk/download/cosmo-customos-installer-v3.zip
unzip cosmo-customos-installer-v3.zip \
  cosmo-customos-installer/linux-cosmo-boot.img \
  cosmo-customos-installer/twrp.img \
  cosmo-customos-installer/root-boot.img
"linux-cosmo-boot.img" is the stock Linux ramdisk boot image which contains a ramdisk and the Linux kernel image.

"twrp.img" is the handy TWRP rescue image.

"root-boot.img" is the stock rooted Android boot image with ramdisk and kernel.

Unpack the stock Linux boot image:

Code: [Select]

cd /path/to/cosmo/kernel
mkdir linux_boot
mkbootimg/unpackbootimg -i ../cosmo-customos-installer/linux-cosmo-boot.img -o linux_bootIt prints this:

Code: [Select]

  BOARD_KERNEL_CMDLINE bootopt=64S3,32N2,64N2
  BOARD_KERNEL_BASE 40078000
  BOARD_NAME
  BOARD_PAGE_SIZE 2048
  BOARD_HASH_TYPE sha1
  BOARD_KERNEL_OFFSET 00008000
  BOARD_RAMDISK_OFFSET 14f88000
  BOARD_SECOND_OFFSET 00e88000
  BOARD_TAGS_OFFSET 13f88000And it creates linux-cosmo-boot.img-ramdisk.gz which we will use later.

Unpack the stock rooted Android boot image:

Code: [Select]

cd /path/to/cosmo/kernel
mkdir android_root_boot
mkbootimg/unpackbootimg -i ../cosmo-customos-installer/root-boot.img -o android_root_bootIt prints this:

Code: [Select]

  BOARD_KERNEL_CMDLINE bootopt=64S3,32N2,64N2 buildvariant=user veritykeyid=id:7e4333f9bba00adfe0ede979e28ed1920492b40f
  BOARD_KERNEL_BASE 40078000
  BOARD_NAME
  BOARD_PAGE_SIZE 2048
  BOARD_HASH_TYPE sha1
  BOARD_KERNEL_OFFSET 00008000
  BOARD_RAMDISK_OFFSET 14f88000
  BOARD_SECOND_OFFSET 00e88000
  BOARD_TAGS_OFFSET 13f88000
  BOARD_OS_VERSION 9.0.0
  BOARD_OS_PATCH_LEVEL 2021-01
  BOARD_DT_SIZE 1And it creates root-boot.img-ramdisk.gz which we will use later.

Retrieve live kernel configs.  Boot stock Linux or stock Android.  Transfer the file /proc/config.gz to your development machine, unzip, and rename:

Code: [Select]

cd /path/to/cosmo/kernel/cosmo-linux-kernel-4.4  # or cosmo-android-kernel
mv config.gz config-from-live-stock-kernel.gz
gunzip config-from-live-stock-kernel.gz
cp config-from-live-stock-kernel .config
Your kernel development directory tree should now resemble this:

Code: [Select]

/path/to/cosmo/
 `-- cosmo-customos-installer-v3.zip  # retrieved 2020-11-18
 `-- cosmo-customos-installer/  # from cosmo-customos-installer-v3.zip
     `-- linux-cosmo-boot.img  # stock Linux boot image
     `-- root-boot.img  # stock rooted Android boot image
     `-- twrp.img  # TWRP rescue image
 `-- kernel/
     `-- aarch64-linux-android-4.9/  # from https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9
     `-- cosmo-android-kernel/  # from https://github.com/dguidipc/cosmo-android-kernel.git
         `-- config-from-live-stock-kernel
         `-- .config
     `-- cosmo-linux-kernel-4.4/  # from https://github.com/gemian/cosmo-linux-kernel-4.4.git
         `-- config-from-live-stock-kernel
         `-- .config
     `-- mkbootimg/  # from https://github.com/osm0sis/mkbootimg.git
     `-- android_root_boot/  # from mkbootimg/unpackbootimg
         `-- root-boot.img-ramdisk.gz
     `-- linux_boot/  # from mkbootimg/unpackbootimg
         `-- linux-cosmo-boot.img-ramdisk.gz

Step 2: Prepare the kernel source for building
In my build environment, the kernel build fails "out of the box" with a truckload of #include errors, plus harmless warnings that are treated as errors.  Please note that other kernel build methods with other cross compilers and/or other host distributions may not experience the same build difficulties I did, and judging from other kernel compilation posts, most don't.  But in case you do, I'm including every issue I hit and how to work around it.

First modify some Makefiles to disable treating warnings as errors.

Code: [Select]

cd /path/to/cosmo/kernel/cosmo-linux-kernel-4.4/
Edit drivers/misc/mediatek/Makefile:

Code: [Select]

# SHUNTCAP: remove this  subdir-ccflags-y += -Werror
Edit: drivers/Makefile:

Code: [Select]

# SHUNTCAP: remove this  subdir-ccflags-y += -Werror
Edit Makefile:

Code: [Select]

#SHUNTCAP: remove this  KBUILD_CFLAGS   += $(call cc-option,-Werror=implicit-int)
Now for the messy part.  In my case, the build could not find a large number of header files.  Here is an example from my Gemini build, but the Cosmo build has the same issues (for me):

Code: [Select]

/devel/gemini/kernel/aarch64-linux-android-4.9/bin/aarch64-linux-android-gcc -Wp,-MD,drivers/misc/mediatek/leds/.mtk_leds_drv.o.d  -nostdinc -isystem /devel/gemini/kernel/aarch64-linux-android-4.9/bin/../lib/gcc/aarch64-linux-android/4.9/include -I./arch/arm64/include -Iarch/arm64/include/generated/uapi -Iarch/arm64/include/generated  -Iinclude -I./arch/arm64/include/uapi -Iarch/arm64/include/generated/uapi -I./include/uapi -Iinclude/generated/uapi -include ./include/linux/kconfig.h -I./drivers/misc/mediatek/include -I. -D__KERNEL__ -mlittle-endian -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -std=gnu89 -fno-PIE -mgeneral-regs-only -fno-pic -fno-asynchronous-unwind-tables -fno-delete-null-pointer-checks -Wno-maybe-uninitialized -O2 --param=allow-store-data-races=0 -DCC_HAVE_ASM_GOTO -Wframe-larger-than=2800 -fstack-protector-strong -Wno-unused-but-set-variable -fno-omit-frame-pointer -fno-optimize-sibling-calls -fno-var-tracking-assignments -g -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fno-merge-all-constants -fmerge-constants -fno-stack-check -fconserve-stack -Werror=implicit-int -Werror=strict-prototypes -Werror=date-time -I./drivers/misc/mediatek/hibcore -I./drivers/misc/mediatek/include -I./drivers/misc/mediatek/include/mt-plat/mt6771/include -I./drivers/misc/mediatek/include/mt-plat -I./drivers/mmc/host/mediatek/mt6771 -I./drivers/misc/mediatek/leds/mt6771 -I./drivers/misc/mediatek/video/include    -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(mtk_leds_drv)"  -D"KBUILD_MODNAME=KBUILD_STR(mtk_leds_drv)" -c -o drivers/misc/mediatek/leds/.tmp_mtk_leds_drv.o drivers/misc/mediatek/leds/mtk_leds_drv.c
drivers/misc/mediatek/leds/mtk_leds_drv.c:26:26: fatal error: mtk_leds_drv.h: No such file or directory
 #include <mtk_leds_drv.h>
This happens because the compiler is called from the toplevel directory on drivers/misc/mediatek/leds/mtk_leds_drv.c, while mtk_leds_drv.h is in drivers/misc/mediatek/leds, but drivers/misc/mediatek/ is not in the -I include path.  According to gcc documentation, <...h> will not search the current directory unless supplied with the -I directive.  And the build doesn't enter that directory anyway, so it has no way to find the header file even if we add "-I." to the toplevel Makefile.  In this case, drivers/misc/mediatek/leds/Makefile lacks an -I directive to include drivers/misc/mediatek/leds.  #include "mtk_leds_drv.h" would have found it, though.

Fix broken build include paths (again, this may not be necessary in other build environments, but it is in mine):

Code: [Select]

cd /path/to/cosmo/kernel/cosmo-linux-kernel-4.4/drivers/misc/mediatek/include
ln -s ../../../devfreq/*.h .
rm mtk_dvfsrc_reg.h  # Avoid ambiguity with misc/mediatek/base/power/spm_v4/*.h
ln -s ../../../input/fingerprint/aw6302_driver/*.h .
ln -s ../../../input/keyboard/mediatek/*.h .
ln -s ../../../misc/mediatek/base/power/cm_mgr_v1/*.h .
ln -s ../../../misc/mediatek/base/power/mt6771/*.h .
ln -s ../../../misc/mediatek/base/power/ppm_v3/src/mach/mt6771/*.h .
ln -s ../../../misc/mediatek/base/power/spm_v4/*.h .
ln -s ../../../misc/mediatek/base/power/swpm_v1/mt6771/*.h .
ln -s ../../../misc/mediatek/cameraisp/src/isp_50/inc .
ln -s ../../../misc/mediatek/ccu/src/mt6771/ccu_ext_interface .
ln -s ../../../misc/mediatek/cmdq/v3/*.h .
ln -s ../../../misc/mediatek/cmdq/v3/mt6771/*.h .
ln -s ../../../misc/mediatek/eccci/*.h .
ln -s ../../../misc/mediatek/eccci/hif .
ln -s ../../../misc/mediatek/eccci/hif/*.h .
ln -s ../../../misc/mediatek/emi/mt6771/*.h .
ln -s ../../../misc/mediatek/ext_disp/mt6771/*.h .
ln -s ../../../misc/mediatek/leds/*.h .
ln -s ../../../misc/mediatek/m4u/mt6771/*.h .
ln -s ../../../misc/mediatek/mu3phy/*.h .
ln -s ../../../misc/mediatek/sspm/*.h .
ln -s ../../../misc/mediatek/sspm/mt6771/*.h .
ln -s ../../../misc/mediatek/timer/timesync/*.h .
cd inc
ln -s ../../../../usb_c/tcpc/inc/* .
cd ..
ln -s ../../../misc/mediatek/video/common/layering_rule_base/v1.1/*.h .
ln -s ../../../misc/mediatek/video/mt6771/dispsys/*.h .
ln -s ../../../misc/mediatek/video/mt6771/videox/*.h .
ln -s ../../../misc/mediatek/vpu/mt6771/*.h .
ln -s ../../../mmc/host/mediatek/ComboA/*.h .
ln -s ../../../pinctrl/mediatek/*.h .
ln -s ../../../power/mediatek/battery/*.h .
ln -s ../../../watchdog/mediatek/wdt/common/wdt_v2/*.h .
Edit drivers/devfreq/helio-dvfsrc.c:

Code: [Select]

// SHUNTCAP: Ambiguous ./drivers/misc/mediatek/include/mtk_dvfsrc_reg.h symlink must be removed.
// SHUNTCAP: Causes Oops due to pulling wrong DVFSRC_LEVEL* definition (lacking DVFSRC_BASE + offset).
// SHUNTCAP: Shouldn't even need these symlinks, but the build system is broken with MTK using the
// SHUNTCAP: angled includes below when the -I directives don't contain the correct paths.  My
// SHUNTCAP: workaround is to create symlinks to the required header files and place them in
// SHUNTCAP: ./drivers/misc/mediatek/include/ for all MTK source to see despite angled includes.
//#include <helio-dvfsrc.h>  // SHUNTCAP
#include "drivers/devfreq/helio-dvfsrc.h"  // SHUNTCAP
//#include <helio-dvfsrc-opp.h>  // SHUNTCAP
#include "drivers/devfreq/helio-dvfsrc-opp.h" // SHUNTCAP
//#include <mtk_dvfsrc_reg.h>  // SHUNTCAP
#include "drivers/devfreq/mtk_dvfsrc_reg.h"  // SHUNTCAP
And only for the Android kernel, these required files weren't even in the tree:

Code: [Select]

ln -s ../../../../../cosmo-linux-kernel-4.4/include/uapi/linux .
cd /path/to/cosmo/kernel/cosmo-android-kernel/net/netfilter
ln -s ../../../cosmo-linux-kernel-4.4/net/netfilter/xt_HL.c .
ln -s ../../../cosmo-linux-kernel-4.4/net/netfilter/xt_TCPMSS.c .
cd -
Also only for the Android build, I had to change drivers/devfreq/helio-dvfsrc.c to the following (different include path than with the Linux build):

Code: [Select]

#include "helio-dvfsrc.h"  // SHUNTCAP
#include "helio-dvfsrc-opp.h" // SHUNTCAP
#include "mtk_dvfsrc_reg.h"  // SHUNTCAP

Step 3: Add exFAT support (optional)
It's frustrating to insert an exFAT-formatted SD card into the Cosmo only to find out there's no support for the filesystem!  So let's add it.  Credit goes to https://github.com/arter97/exfat-linux/.

Code: [Select]

cd /path/to/cosmo/kernel/
wget https://github.com/arter97/exfat-linux/archive/old.zip -O exfat-linux-old.zip # (do not git clone; it will pull one requiring Linux 4.9+)
unzip exfat-linux-old.zip
cp -a exfat-linux-old cosmo-linux-kernel-4.4/fs/exfat  # if you're compiling Linux kernel
cp -a exfat-linux-old cosmo-android-kernel/fs/exfat  # if you're compiling Android kernel
cd cosmo-linux-kernel-4.4 # or cosmo-android-kernel
Edit fs/Kconfig, add exfat:

Code: [Select]

source "fs/fat/Kconfig"
source "fs/exfat/Kconfig"
Edit fs/Makefile, add exfat:

Code: [Select]

obj-$(CONFIG_FAT_FS)            += fat/ 
obj-$(CONFIG_EXFAT_FS)          += exfat/

Step 4: Configure the kernel
Make any changes to the kernel configuration before building:

Code: [Select]

cd /path/to/cosmo/kernel/cosmo-linux-kernel-4.4  # or cosmo-android-kernel
CROSS_COMPILE=../aarch64-linux-android-4.9/bin/aarch64-linux-android- ARCH=arm64 make menuconfig
Don't forget to enable exFAT if you added the sources in step 3:

Code: [Select]

File systems -> DOS/FAT/NT Filesystems
  <*> exFAT fs support

Step 5: Compile the kernel
Now compile the kernel:

Code: [Select]

cd /path/to/cosmo/kernel/cosmo-linux-kernel-4.4  # or cosmo-android-kernel
CROSS_COMPILE=../aarch64-linux-android-4.9/bin/aarch64-linux-android- ARCH=arm64 make LOCALVERSION= -j8
You may wish to adjust the -j8 parameter based on your CPU.  -j8 will compile using eight threads.  For example, on a quad-core processor, this number comes from having two threads per core.

Note that you can also use clang to compile your kernel if you prefer.  The stock kernels were built with clang.  If you have installed arm64 clang on your development system and prefer it, use this command instead:

Code: [Select]

make ARCH=arm64 CC=clang CLANG_TRIPLE=aarch64-linux-gnu- CROSS_COMPILE=aarch64-linux-android- V=1 LOCALVERSION= -j8
V=1 is optional and will produce a more verbose output.

The stock Cosmo kernel was built with clang-4691093, which you can obtain here:         
https://android.googlesource.com/platform/prebuilts/clang/host/linux-x86/+archive/android-9.0.0_r1/clang-4691093.tar.gz


Step 6: Create a boot image
Now create the Linux kernel boot image:

Code: [Select]

cd /path/to/cosmo/kernel
mkbootimg/mkbootimg \
  --kernel cosmo-linux-kernel-4.4/arch/arm64/boot/Image.gz-dtb \
  --ramdisk linux_boot/linux-cosmo-boot.img-ramdisk.gz \
  --base 0x40078000 \
  --second_offset 0x00e88000 \
  --cmdline "bootopt=64S3,32N2,64N2" \
  --kernel_offset 0x00008000 \
  --ramdisk_offset 0x14f88000 \
  --tags_offset 0x13f88000 \
  --pagesize 2048 \
  --hash sha1 \
  -o linux_test.img
You will now have a new Linux boot image in
/path/to/cosmo/kernel/linux_test.img

Alternatively, create the Android boot image:

Code: [Select]

cd /path/to/cosmo/kernel
mkbootimg/mkbootimg \
  --kernel cosmo-android-kernel/arch/arm64/boot/Image.gz-dtb \
  --ramdisk android_root_boot/root-boot.img-ramdisk.gz \
  --base 0x40078000 \
  --second_offset 0x00e88000 \
  --cmdline "bootopt=64S3,32N2,64N2 buildvariant=user veritykeyid=id:7e4333f9bba00adfe0ede979e28ed1920492b40f" \
  --kernel_offset 0x00008000 \
  --ramdisk_offset 0x14f88000 \
  --tags_offset 0x13f88000 \
  --pagesize 2048 \
  --hash sha1 \
  --os_version 9.0.0 --os_patch_level 2021-01 \
  -o android_test.img
You will now have a new Android boot image in
/path/to/cosmo/kernel/android_test.img

Note that this Android boot image does not have root capability.  If you want rooted Android, continue to step 7.  Otherwise, skip to step 8.


Step 7: Root the Android boot image (optional)
To root your Android boot image, boot the Cosmo into Android.  Install Magisk Manager.  Transfer /path/to/cosmo/kernel/android_test.img to the Cosmo, placing it in /sdcard/.  You can use adb from your development machine:

Code: [Select]

adb -s <Cosmo_Serial_Num> push /path/to/cosmo/kernel/android_test.img /sdcard/
Open Magisk Manager, tap the Install button, select android_test.img, and let it root the image.  Now transfer the rooted image back to your development machine:

Code: [Select]

adb -s <Cosmo_Serial_Num> pull /sdcard/kernel/android_test.img rooted_android_test.img

Step 8: Flash your new kernel boot image
The Cosmo has four boot partitions which are accessible by /dev/block/platform/bootdevice/by-name/<partition_name>, where <partition_name> is one of the following:
boot - normal Android (not rooted)
ROOTED_ANDROID - rooted Android
DEBIAN_KDE - Linux
TWRP - TWRP rescue image

We will flash our newly created kernel boot image onto one of these partitions.

IMPORTANT: Before flashing, make sure you have a way to reflash a working Linux kernel if your test kernel is broken.  You can rely on either rooted Android or TWRP and use adb to push your test kernel onto the Cosmo, then flash from there.  In other words, if you don't want to use Planet's SD card method of reflashing your Linux or Android boot partitions, install TWRP or rooted Android NOW.
                                                                                                                                         
To flash from Cosmo Linux: first transfer linux_test.img to the Cosmo using the means of your choice.  My personal build script ftp's to the Cosmo at 10.15.19.82 (the Cosmo's RNDIS address as presented when plugged into the development machine's USB port).  Then either ssh into the Cosmo or directly type these commands on it:

Code: [Select]

dd if=/path/to/linux_test.img of=/dev/block/platform/bootdevice/by-name/DEBIAN_KDE
Or from Android or TWRP via adb on your development machine:

Code: [Select]

adb -s <Cosmo_Serial_Num> push ../linux_test.img /sdcard/
adb -s <Cosmo_Serial_Num> shell
su
dd if=/sdcard/root_test.img of=/dev/block/platform/bootdevice/by-name/DEBIAN_KDE
To flash an Android boot image, use the same dd command but flash to either "ROOTED_ANDROID" or "boot".  For example, for non-rooted Android:

Code: [Select]

dd if=/sdcard/android_test.img of=/dev/block/platform/bootdevice/by-name/bootOr for rooted Android:

Code: [Select]

dd if=/sdcard/rooted_android_test.img of=/dev/block/platform/bootdevice/by-name/ROOTED_ANDROID
I actually have a backup Linux boot image on "boot" and a working Android image on ROOTED_ANDROID.

You can also transfer and flash the stock boot images (see step 1 for names).

Now you can reboot and see if your new kernel boots or not.


Step 9: Find out what went wrong
What, your new kernel didn't boot the first time?  Neither did mine.  At this point, you can either flash a stock boot image back (see step 8 above), or boot to an alternative OS such as TWRP or rooted Android and check the last kernel log to see what went wrong during boot.

Inspect this file on the Cosmo: /proc/last_kmsg

In my case, the kernel source referenced an ambiguous driver header and the wrong one was compiled.  I noted this in /proc/last_kmsg:

Code: [Select]

[    0.991933] -(6)[1:swapper/0]Unable to handle kernel NULL pointer dereference at virtual address 000000e0
[    0.991945] -(6)[1:swapper/0]pgd = ffffff800a109000
[    0.991953] [000000e0] *pgd=00000000ffffe003, *pud=00000000ffffe003, *pmd=0000000000000000
[    0.991971] -(6)[1:swapper/0]Internal error: Oops: 96000045 [#1] PREEMPT SMP
[    0.991981] disable aee kernel api
[    0.991984] -(6)[1:swapper/0]Kernel Offset: 0x0 from 0xffffff8008000000
[    0.991996] Modules linked in:
[    1.992010] -(6)[1:swapper/0]Non-crashing CPUs did not react to IPI
[    1.992022] -(6)[1:swapper/0]CPU: 6 PID: 1 Comm: swapper/0 Tainted: G S      W       4.4.146+ #3
[    1.992036] -(6)[1:swapper/0]Hardware name: MT6771V/CT (DT)
[    1.992044] -(6)[1:swapper/0]task: ffffffc05ff58000 task.stack: ffffffc05ff6c000
[    1.992060] -(6)[1:swapper/0]PC is at dvfsrc_init.part.0+0x44/0x3ac  <-- NOTE THIS FUNCTION NAME: CRASH IS HERE
[    1.992069] -(6)[1:swapper/0]LR is at dvfsrc_init.part.0+0x38/0x3ac
... spew ...
[    2.005768] -(6)[1:swapper/0]<4>[111951692] mtk_wdt_probe+0x14c/0x388
[    2.006373] -(6)[1:swapper/0]****************dump wdt reg end*************
[    2.006943] -(6)[1:swapper/0]wdt_arch_reset: sw reset happen!
From there I saw that it was crashing in drivers/misc/mediatek/base/power/spm_v4/mtk_spm_vcorefs_mt6771.c:dvfsrc_init(). To fix it, I had to patch drivers/devfreq/helio-dvfsrc.c to include the correct header.  At least one other OESF user had the same problem: https://www.oesf.org/forum/index.php?topic=36147.msg295276#msg295276


This concludes my instructions on compiling and flashing the kernel.  I hope this helps some of you.

18

Cosmo Communicator - Linux / Re: Anbox on Cosmo Linux

« on: April 17, 2021, 04:25:02 am »

Step 5: Add OpenGL hardware acceleration
Anbox requires OpenGL support to run smoothly.  Although you could skip this step and rely on Mesa's software rendering, you'll be dissatisfied with the slow, choppy rendering while running Anbox.

We will use gl4es to give Cosmo accelerated OpenGL capabilities.  gl4es provides a translation layer between OpenGL for Linux and OpenGL ES for Android.  In other words, it uses Android's native GLES libraries to render OpenGL calls under Linux.

First install glxgears from the mesa-utils package:

Code: [Select]

apt install mesa-utils
Now run glxgears and note the frame rate.  On my Cosmo without gl4es 3D acceleration, I see roughly 156 FPS.  After gl4es, I see over 1100 FPS.  That should be enough motivation for the reader to complete this step!  I have seen rates as low as 900 but also as high as 1300, depending on how the Cosmo feels that day.

Fetch the latest gl4es code here:
https://github.com/ptitSeb/gl4es/archive/refs/heads/master.zip

Unzip it, then enter the directory.

Code: [Select]

unzip master.zip
cd gl4es-master
mkdir build
cd build
cmake ..
make -j8
cd ..
After that finishes building, install it manually:

Code: [Select]

mkdir -p /opt/gl4es/lib
cp lib/libGL.so.1 /opt/gl4es/lib
cp /system/lib64/{libEGL.so,libGLESv1_CM.so,libGLESv2.so,libGLESv3.so} /opt/gl4es/lib
The key to getting gl4es to work properly is copying the native GLES libraries from /system, which is automatically mounted by droid-hybris on boot.  These are from the Android system partition on /dev/mmcblk0p36 (/dev/block/platform/bootdevice/by-name/system).  Without these critical native libraries, gl4es will revert to using software rendering via libEGL.so as installed by libegl1 (and hence libegl-mesa0).

Now try running the following command:

Code: [Select]

LD_LIBRARY_PATH=/opt/gl4es/lib:$LD_LIBRARY_PATH glxinfo | grep GLSL
You should see this:

Code: [Select]

LIBGL: GLSL 300 es supported
LIBGL: GLSL 310 es supported and used
And try this command, too:

Code: [Select]

LD_LIBRARY_PATH=/opt/gl4es/lib:$LD_LIBRARY_PATH glxinfo | grep -i vendor
You should see this:

Code: [Select]

LIBGL: Hardware vendor is ARM
server glx vendor string: ptitSeb
client glx vendor string: ptitSeb
OpenGL vendor string: ptitSeb
If you see "Vendor: VMware, Inc." then your gl4es installation is incorrect.

Finally, try running glxgears and note the frame rate:

Code: [Select]

LD_LIBRARY_PATH=/opt/gl4es/lib:$LD_LIBRARY_PATH glxgears
You should see something like this:

Code: [Select]

5621 frames in 5.0 seconds = 1124.089 FPS
5465 frames in 5.0 seconds = 1092.959 FPS
5687 frames in 5.0 seconds = 1137.375 FPS
5705 frames in 5.0 seconds = 1140.879 FPS
You should NOT see something like this (slow software rendering):

Code: [Select]

714 frames in 5.0 seconds = 142.674 FPS
752 frames in 5.0 seconds = 150.320 FPS
754 frames in 5.0 seconds = 150.681 FPS
766 frames in 5.0 seconds = 153.027 FPS
If you saw high frame rates, you have successfully built and installed gl4es.  Try running some 3D OpenGL games now.  You'll be pleased with the speed increase.

You most likely won't want to keep setting LD_LIBRARY_PATH each time you run a command.  To install system-wide, add the following line to /etc/profile:

Code: [Select]

export LD_LIBRARY_PATH=/opt/gl4es/lib:$LD_LIBRARY_PATH

Step 6: Test Anbox
If you've made it this far, you're itching to try things out!  Anbox consists of two main parts: the LXC container which runs Android, and the session manager.  We have already configured systemd to start Anbox on boot, but we can also start the container and session manager manually for testing.

Start the Anbox container service (as root):

Code: [Select]

systemctl start anbox-container-manager
Start the Anbox session manager (as user):

Code: [Select]

systemctl --user start anbox-session-manager
Or, if you prefer to start it without systemctl to view any potential error output, do this:

Code: [Select]

LD_LIBRARY_PATH=/opt/gl4es/lib anbox session-manager
A properly running Anbox setup will show the following processes (try running ps ax | grep anbox | grep -v grep):

Code: [Select]

23899 ?        Ssl    0:00 /usr/bin/anbox container-manager --daemon --privileged --data-path=/var/lib/anbox
23918 ?        Sl     0:03 anbox session-manager
23949 ?        Ss     0:00 [lxc monitor] /var/lib/anbox/containers default
23989 pts/3    Sl     0:00 /system/bin/anboxd
24413 pts/3    Sl     0:00 org.anbox.appmgr
Now try starting the Anbox app manager:

Code: [Select]

anbox launch --package=org.anbox.appmgr --component=org.anbox.appmgr.AppViewActivity
You should see a window called "Anbox Application Manager" appear with various app icons: Calculator, Calendar, Settings, etc.  Try them out.  They should be fast and responsive if your gl4es installation is working.  Camera does not work (did you expect it to?)

Now reboot and try launching the application manager again to make sure everything starts up automatically.

Anbox logs are located in /var/lib/anbox/logs/ and are only available while the container is running because I've mounted this as a tmpfs to eliminate flash writes (see /lib/systemd/system/anbox-container-manager.service).  /var/lib/anbox/logs/console.log is the session manager log, while /var/lib/anbox/logs/container.log is the LXC container log.


Step 7: Add your own apps
Google Play Store does not work at present, but I am working on it and will update these instructions if I succeed.

Anbox runs adbd, so all you need to do to install an APK is use adb from a command line.  For example, F-Droid works well with Anbox.  Download the APK here:
https://f-droid.org/F-Droid.apk

To install it, simply do this:

Code: [Select]

adb install F-Droid.apk
You should see it appear in the application manager window.

Apps can also be started from the command line.  You just need to figure out the name of the intent that starts the app.  Here's how, using OsmAnd+ as an example.  First get a shell in the Anbox container (as root):

Code: [Select]

anbox-shell.sh
Now monitor the system log as follows:

Code: [Select]

logcat | grep START.*intent
In the application manager window, start your app (here, OsmAnd+).  Watch the last line of output from your shell:

Code: [Select]

04-17 02:58:24.196   120   881 I ActivityManager: START u0 {act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] flg=0x10000000 pkg=net.osmand.plus cmp=net.osmand.plus/.activities.MapActivity} from uid 10000 on display 0
Note the pkg= and cmp= values.  Removing any / character, insert them into the following command:

Code: [Select]

anbox launch --package=net.osmand.plus --component=net.osmand.plus.activities.MapActivity
Noe that --package matches pkg and --component matches cmp.  Just remember to remove the slash / character.

If you would like to add a shortcut or menu launcher to your favorite window manager, you can add the appropriate "anbox launch ..." command to your desktop configuration.

Not all apps work with Anbox, but a lot do.


Step 8: GPS (optional)
The Cosmo GPS receiver works under Linux.  And even better, it can be used with Anbox.  We will use libhybris to access the GPS sensor with the MediaTek closed-source Android driver, pipe the NMEA sentences into a fake serial port, and use D-Bus to send the sentences to the location service in Anbox.

First fetch the ubports libhybris source here:
https://github.com/ubports/libhybris/archive/refs/heads/master.zip

Unzip it and enter the directory:

Code: [Select]

unzip master.zip
cd libhybris-master
It doesn't compile without changing hybris/common/hooks.c:

Code: [Select]

static struct _hook hooks_common[] = {
...
    {"cfree", /*c*/free },  // SHUNTCAP
...
    {"writev", write/*v*/}, // SHUNTCAP
The GPS test program seg faults because it hasn't been updated to use the newer Android GPS callback structure, so we need to fix it.  This is why we cannot simply install libhybris-test from the Gemian repository.  Edit hybris/tests/test_gps.c and make the following changes:

Code: [Select]

static void empty_callback(void) {}  // SHUNTCAP: ADD THIS

GpsCallbacks callbacks = {
  sizeof(GpsCallbacks),
  location_callback,
  status_callback,
  sv_status_callback,
  nmea_callback,
  set_capabilities_callback,
  acquire_wakelock_callback,
  release_wakelock_callback,
  create_thread_callback,
  empty_callback,  // SHUNTCAP: ADD THIS (request_utc_time_cb)
  empty_callback,  // SHUNTCAP: ADD THIS (set_system_info_cb)
  empty_callback   // SHUNTCAP: ADD THIS (gnss_sv_status_cb)
};

...

  while(1/*sleeptime > 0*/)  // SHUNTCAP: RUN FOREVER
Now build it:

Code: [Select]

debian/rules build
This creates a file called ./hybris/tests/.libs/test_gps, among other things.  Install it:

Code: [Select]

cp ./hybris/tests/.libs/test_gps /usr/local/bin
Run test_gps to make sure you see NMEA sentences.

To feed GPS data into Anbox, run this command:

Code: [Select]

test_gps | grep nmea.*: | sed -u 's/nmea.*\$/\$/' | while read LINE; do dbus-send --session --dest=org.anbox --print-reply /org/anbox org.anbox.Gps.PushSentence string:"$LINE"; done &
Now under Anbox, see if you have a lock on your GPS coordinates.  Try SatStat.  It is available on F-Droid.

As a bonus, you can also use the GPS with Linux programs such as gpsd and QGIS by creating a fake serial port that acts like a serial GPS receiver.  Just run these commands as root:

Code: [Select]

test_gps | grep nmea.*: | sed -u 's/nmea.*\$/\$/' | socat - pty,link=/dev/ttyS2,rawer,echo=0,b4800 &
chmod 0644 /dev/ttyS2
Now configure gpsd or any other GPS software that directly reads a serial GPS receiver to use /dev/ttyS2.


This concludes my article on building Anbox for the Cosmo from scratch.  Comments are welcome.

19

Cosmo Communicator - Linux / Re: Anbox on Cosmo Linux

« on: April 17, 2021, 04:23:59 am »

Anbox for the Cosmo
This article describes how to build and install Anbox for Cosmo Linux.  Anbox is "Android in a Box."  It is a software package that runs Android apps under Linux using LXC (Linux Containers).  Your android apps can literally run in their own windows alongside your native Linux programs.  For many users, this could relieve them of the need to dual-boot the Cosmo.

I might provide prebuilt images of the kernel and Anbox image at a later date depending on user interest.  For now, I present my instructions for building Anbox from scratch.  I assume that the reader is comfortable building and flashing Linux kernels, compiling software from scratch, and building Android AOSP images.


Step 1: Configure the Cosmo Linux kernel
You will need to download v3 of the Cosmo Linux kernel here:
  https://github.com/gemian/cosmo-linux-kernel-4.4/archive/refs/heads/master.zip

Unzip and build for the Cosmo.  You can find instructions elsewhere on OESF, such as here:
  https://www.oesf.org/forum/index.php?topic=36321.0
I did not write them and I use a different procedure, but they may work for you.

Flash it and make sure it works.  If enough people have trouble building and flashing the Cosmo kernel, I can post the steps that I use in my development.  Alternatively, I might provide my own kernel image for users to flash at their own risk.  I did have to make a number of changes to the kernel source as it did not compile "out of the box."

Now reconfigure the kernel.

Enable these for iptables to work (called by /usr/share/anbox/anbox-bridge.sh):

Code: [Select]

Networking support -> Networking options -> Network packet filtering framework (Netfilter) -> Core Netfilter Configuration:
    <*> Netfilter nf_tables support
    <*>   Netfilter nf_tables mixed IPv4/IPv6 tables support
    <*>   Netfilter nf_tables netdev tables support
    <*>   Netfilter nf_tables IPv6 exthdr module
    <*>   Netfilter nf_tables meta module
    <*>   Netfilter nf_tables conntrack module
    <*>   Netfilter nf_tables rbtree set module
    <*>   Netfilter nf_tables hash set module
    <*>   Netfilter nf_tables counter module
    <*>   Netfilter nf_tables log module
    <*>   Netfilter nf_tables limit module
    <*>   Netfilter nf_tables masquerade support
    <*>   Netfilter nf_tables redirect support
    <*>   Netfilter nf_tables nat module
    <*>   Netfilter nf_tables queue module
    <*>   Netfilter nf_tables reject support
    <*>   Netfilter x_tables over nf_tables module
We only need some of these nf_tables options for Anbox, but there's no harm in giving our kernel full functionality.

Add this to kernel (CONFIG_NETFILTER_XT_TARGET_CHECKSUM=y):

Code: [Select]

Networking support -> Networking options -> Network packet filtering framework (Netfilter) -> Core Netfilter Configuration -> Netfilter Xtables support:
    <*>   CHECKSUM target support
Add this to kernel (CONFIG_VETH=y):

Code: [Select]

Device drivers -> Network device support -> Network core driver support:
    <*>     Virtual ethernet pair device
Add squashfs support:

Code: [Select]

File systems -> Miscellaneous filesystems:
    <*>   SquashFS 4.0 - Squashed file system support
    Decompressor parallelisation options (Use multiple decompressors for parallel I/O)
    [*]     Squashfs XATTR support
    [*]     Include support for ZLIB compressed file systems
    [*]     Include support for LZ4 compressed file systems
    [*]     Include support for LZO compressed file systems
    [*]     Include support for XZ compressed file systems
    [*]     Use 4K device block size?
And if you find that apt is unable to resolve hostnames despite your Cosmo having full Internet access, configure CONFIG_ANDROID_PARANOID_NETWORK=n here:

Code: [Select]

Networking support -> Networking options:
    [ ] Only allow certain groups to create sockets

Step 2: Fix the kernel
Anbox uses iptables.  But on the Cosmo, iptables causes a reboot as soon as it is run.  The cause is a null pointer access in the kernel.  Edit the following file and make the indicated fix (add the NULL check for ssk).

Enter the kernel source directory:

Code: [Select]

cd /path/to/cosmo-linux-kernel-4.4-master
Edit net/netlink/af_netlink.c:

Code: [Select]

int netlink_unicast(struct sock *ssk, struct sk_buff *skb,
                    u32 portid, int nonblock)
{
        struct sock *sk;
        int err;
        long timeo;

        skb = netlink_trim(skb, gfp_any());

        // SHUNTCAP: prevent null pointer access from crashing kernel
        // and rebooting Cosmo PDA when iptables command is run
        if (ssk == NULL) {
                printk("SHUNTCAP: warning: ssk is null in %s:%s line %d\n",
                        __FILE__, __FUNCTION__, __LINE__);
                return PTR_ERR(ssk);
        }  // End of SHUNTCAP added
        timeo = sock_sndtimeo(ssk, nonblock);
...
}
While we're inside the kernel source, we might as well fix that annoying Cosmo keyboard ghosting issue.  For those of us who type very fast, keyboard ghosting is an incredibly aggravating hardware bug.  Type "more" very quickly and you'll get "mor0e".  Type "()" very quickly (without releasing the shift key) and you'll get "(0)".  Insanely frustrating.

This ghosting issue started with the Gemini.  Users complained.  A community fix was implemented.  So when the Cosmo came along and Planet Computers had the opportunity to build the keyboard hardware properly or at least implement the community kernel patch, they promptly... did nothing.

I believe Adam Boardman is to credit for this fix on the Gemini, but I am not 100% certain.  EDIT: according to my old notes, it was Nathan Banks.


Edit drivers/misc/mediatek/aw9523/aw9523_key.c:

Code: [Select]

static void aw9523_key_eint_work(struct work_struct *work)
{
...
                        //printk("0x%02x, ", keyst_new[i]);
                        // //i=p1 keyst[i]=p0
                }
        }       
        // AW9523_LOG("\n");

        // SHUNTCAP: ADD THIS BLOCK
        /* This routine prevents ghosting.  As an example, if Control+L_Shift+N is pressed,
         * the keyboard detects both N & M at the same time due to the electronic circuit.
         * Rather than detect both keys, block both keys until either Control or L_Shift is
         * released, then detect the correct keypress. See youtu.be/L3ByBtM-w9I */
        if (memcmp(keyst_old, keyst_new, P1_NUM_MAX)) { // keyst changed
                val = P0_KROW_MASK; // The distinct columns used
                y = 0;
                update_now = true; // Now by default, do the update since keyst changed.
                for(i = 0; i < P1_NUM_MAX; i++) {
                        if (keyst_new[i]==P0_KROW_MASK) {
                                // Most of the time no key is pressed, skip it.
                                continue;
                        }
                        x = 0; // Count the number of keys pressed (ie. clear bits)
                        for (j = 0; j < P0_NUM_MAX; j++) {
                                if (! (keyst_new[i] & (1 << j))) {      // press
                                        x++; // Increase whenever this bit is clear.
                                        // Boolean expression is false if a previous row
                                        // also has this bit clear.  In this case, the
                                        // keyboard is ghosting, and update_now is false.
                                        update_now = update_now && (val & (1 << j));
                                }
                        }
                        // If more than one key in this row is pressed, remember which bits
                        // are clear so we can check for ghosting in other rows.
                        if (x>1) {
                                val &= keyst_new[i];
                        }
                }
        }  // SHUNTCAP: END OF ADDED BLOCK

        // update_now is set when the key state changes and there's no ghosting right now.  // SHUNTCAP: ADD
        if (update_now) {  // SHUNTCAP: ADD
                if (memcmp(keyst_old, keyst_new, P1_NUM_MAX)) { // keyst changed
...
                        memcpy(keyst_old, keyst_new, P1_NUM_MAX);
                }
        }  // SHUNTCAP: ADD
Build, flash, and test the new kernel.  I strongly suggest installing TWRP on one of your Cosmo boot partitions so you can reflash a working stock kernel via adb just in case you break your Linux boot.


Step 3: Build Anbox on the Cosmo
The anbox package in the Gemian repository does not work properly.  In fact, it doesn't even include a rootfs image.  We'll build that later, but first we must build Anbox from scratch.  We will do this directly on the Cosmo.

Install the necessary development packages:

Code: [Select]

apt install build-essential cmake cmake-data debhelper dbus google-mock libboost-dev \
 libboost-filesystem-dev libboost-log-dev libboost-iostreams-dev libboost-program-options-dev \
 libboost-system-dev libboost-test-dev libboost-thread-dev libcap-dev libexpat1-dev libsystemd-dev \
 libegl1-mesa-dev libgles2-mesa-dev libglm-dev libgtest-dev liblxc1 libproperties-cpp-dev \
 libprotobuf-dev libsdl2-dev libsdl2-image-dev lxc-dev pkg-config protobuf-compiler python3-minimal
Check out the Anbox source:

Code: [Select]

git clone https://github.com/anbox/anbox.git --recurse-submodules
Build Anbox:

Code: [Select]

cd anbox
# Comment out add_subdirectory(tests) in CMakeLists.txt as it breaks the build
# and we don't need it anyhow
sed -i 's/add_subdirectory(tests)/#add_subdirectory(tests)/' CMakeLists.txt
mkdir build
cd build
cmake ..
make -j8
Assuming a successful build, install it:

Code: [Select]

make install
You should now see an executable called anbox the /usr/local/bin directory.

Create the Anbox directory structure:

Code: [Select]

mkdir -p /var/lib/anbox/{cache,containers,data,devices,logs,ramdisk,rootfs,state}
Install the network bridge script to give Anbox networking capabilities:

Code: [Select]

mkdir -p /usr/share/anbox
cp ./scripts/anbox-bridge.sh /usr/share/anbox/
The /usr/share/anbox/anbox-bridge.sh requires some changes to work on the Cosmo.  Edit the file, then change all occurances of "iptables" to "iptables-legacy" and save.

Install a convenient root shell script to access the Anbox LXC container shell:

Code: [Select]

cp scripts/anbox-shell.sh /usr/local/bin
The Anbox container can be configured to start on boot, controlled by systemd.  Create a new systemd service file:

Code: [Select]

echo "[Unit]
Description=Anbox Container Manager
Documentation=man:anbox(1)
After=network.target
Wants=network.target
ConditionPathExists=/var/lib/anbox/android.img

[Service]
ExecStartPre=/usr/bin/mount -t tmpfs tmpfs /var/lib/anbox/logs
ExecStartPre=/usr/share/anbox/anbox-bridge.sh start
ExecStart=/usr/bin/anbox container-manager --daemon --privileged --data-path=/var/lib/anbox
ExecStopPost=/usr/share/anbox/anbox-bridge.sh stop
ExecStopPost=/usr/bin/umount /var/lib/anbox/logs

[Install]
WantedBy=multi-user.target" > /lib/systemd/system/anbox-container-manager.service
Create a new systemd user service to start and stop the Anbox session manager:

Code: [Select]

echo "[Unit]
Description=Anbox Session Manager
Documentation=man:anbox(1)

[Service]
ExecStart=/usr/bin/anbox session-manager

[Install]
WantedBy=default.target" > /usr/lib/systemd/user/anbox-session-manager.service
Now enable the session manager service.  It will automatically start on first login:

Code: [Select]

systemctl --global enable anbox-session-manager
Add support for hardware-accelerated OpenGL gl4es (described in step 5 below).  Edit /etc/systemd/user.conf and make this change:

Code: [Select]

DefaultEnvironment=LD_LIBRARY_PATH=/opt/gl4es/lib DISPLAY=:0
You might want a desktop entry for Anbox if you use a fancy window manager:

Code: [Select]

echo "[Desktop Entry]
Name=Anbox
GenericName=Application Manager
Comment=Android in a box
Version=1.0
Type=Application
Icon=anbox
Exec=anbox launch --package=org.anbox.appmgr --component=org.anbox.appmgr.AppViewActivity
Terminal=false
Keywords=android;container;
Categories=Utility;" > /usr/share/applications/anbox.desktop

Step 4: Build or download Android image from AOSP
Anbox requires an Android filesystem (rootfs) image to run.  On a device running Android natively, this is a ramdisk mounted on / which includes init and its associated scripts, plus a basic directory structure in which other data partitions are mounted (such as /system).  But with Anbox, we need to create a squashfs image from AOSP (Android Open Source Project) to simulate that filesystem.

If you do not wish to or cannot build AOSP from scratch, you can download my 238MB prebuilt image here:
https://www.dropbox.com/s/s51th81rv9ysuw0/android.img_AOSP_7.1.1_anbox_arm64_2021-04-08?dl=1

After downloading, rename and install the file:

Code: [Select]

mv /path/to/android.img_AOSP_7.1.1_anbox_arm64_2021-04-08 /var/lib/anbox/android.img
If you download the image, you can skip the following instructions and jump to step 5.

To build AOSP, we need an x86 machine with at least 16GB of RAM and 62GB of free hard drive space.  It's possible to build with less; if necessary, I can post instructions.  My instructions for building AOSP are based on this document:
  https://github.com/anbox/anbox/blob/master/docs/build-android.md
But follow my instructions, not those, or you'll end up wasting a lot of hard drive space.

You will need the repo tool which might be in your distribution already, but don't count on it being current.  I'd recommend getting the latest version as shown in my instructions below.

First check out the Anbox version of AOSP:

Code: [Select]

mkdir anbox-work
cd anbox-work
curl https://storage.googleapis.com/git-repo-downloads/repo > repo
export PATH=`pwd`:$PATH
repo init -u https://github.com/anbox/platform_manifests.git -b anbox --depth=1
repo sync -c -j4
The --depth=1 and the -c options will greatly reduce the size of the downloaded repository, only checking out the current branch (-c) and not retrieving all the unneeded history (--depth=1).

Over a fast fiberoptic WAN, the download took about an hour and a half.

Fix a few files with undefined functions that we (hopefully) won't need anyhow.  While I normally prefer to find the root cause of the compile failure, for now, let's just comment them out:
external/libavc/decoder/arm/ih264d_function_selector.c, line 67:

Code: [Select]

// SHUNTCAP: undefined function            ih264d_init_function_ptr_av8(ps_codec);
external/libavc/encoder/arm/ih264e_function_selector.c, line 111:

Code: [Select]

// SHUNTCAP: undefined function           ih264e_init_function_ptr_neon_av8(ps_codec);
external/libmpeg2/common/ideint.c, lines 162 and 180:

Code: [Select]

// SHUNTCAP: undefined function       ps_ctxt->s_params.e_arch = ideint_default_arch();
// SHUNTCAP: undefined function   ideint_init_function_ptr(ps_ctxt);
external/libmpeg2/decoder/arm/impeg2d_function_selector.c, line 80:

Code: [Select]

// SHUNTCAP: undefined function           impeg2d_init_function_ptr_av8(ps_codec);
Now build AOSP:

Code: [Select]

. build/envsetup.sh
lunch anbox_arm64-userdebug
make -j8
If successful, you'll see a file in ./out/target/product/arm64/system.img. Transfer that over to the Cosmo.  Mount it, then build a squashfs rootfs image from it (on the Cosmo):

Code: [Select]

apt install squashfs-tools
mkdir system_mnt
mount -o loop system.img system_mnt
mksquashfs system_mnt/* /var/lib/anbox/android.img
umount system_mnt
rmdir system_mnt

20

Cosmo Communicator - Linux / Re: Anbox on Cosmo Linux

« on: April 08, 2021, 12:00:43 am »

Okay.  It'll take me a little while to assemble and organize my documentation for public viewing, but as long as there's even a tiny bit of community interest, I'm happy to do so.  I will also be documenting how to get hardware-accelerated OpenGL working on the Cosmo (which Anbox requires), and how to get the GPS receiver working both with Cosmo's Linux (e.g., running gpsd) and with Anbox.  Yes, I have the Cosmo's on-board GPS working with Anbox under Linux.  I am running OsmAnd+ in another window as I type this on the Cosmo.

21

Cosmo Communicator - Linux / Anbox on Cosmo Linux

« on: April 05, 2021, 07:36:52 pm »

I have managed to get Anbox working under Linux on the Cosmo.  This allows one to run Android apps under Linux in a container.  This is not slow emulation.  See https://anbox.io/ for a better description.

The anbox package in the Gemian repository does not work.  I had to build an Android image and compile Anbox from scratch.  I also had to make some kernel changes and fixes to get everything working.

Is anyone interested in my work?  Does anyone have a use for it?  Or do people prefer to dual boot instead?

22

Astro Slide - Hardware / Re: Camera - is it IMX586?

« on: January 16, 2021, 07:15:31 am »

Planet announced the camera for Astro a while back to be 48MP from Sony. To me it sounds like the IMX586 model, so once again the true resolution is more like 12MP than 48 MP (https://www.nextpit.com/48-mp-or-not-look-at-sony-new-camera-sensor), but still a nice improvement over the Cosmo. Apparently, the final results can be quite different (and that is from phones that use an additional camera): https://www.notebookcheck.net/Sony-IMX586-Comparison-Review-Five-48-MP-smartphones-face-off-in-a-camera-duel.428434.0.html.

None of the phones in that review uses a MediaTek chipset.  They are as follows:
Asus ZenFone 6: Qualcomm                                                   
Honor View 20: HUAWEI                                                     
OnePlus 7: Qualcomm 
Xiaomi Mi 9: Qualcomm
ZTE Axon 10 Pro: Qualcomm

To see what a 48MP sensor looks like in a MediaTek phone, take a look at GSMArena's photo comparison tool:

https://www.gsmarena.com/piccmp.php3?idType=4&idPhone2=8310&idCamera1=300738&idCamera3=300610

In this link I have selected three phones:
Oppo Reno4 Pro 5G, 48MP (MediaTek chipset)
Motorola Moto Z Play, 16MP (standard sensor, not Quad Bayer or Tetracell)
OnePlus 7(48MP, Qualcomm chipset)

I chose the Oppo Reno4 Pro 5G because I figured it's probably the newest 48MP MediaTek-based phone in the GSM Arena database.

I chose the 2016-era Moto Z Play, with its "smaller" 16MP sensor, as a benchmark.  I also happen to own one because I backed the failed Moto Keyboard mod in 2017.

I randomly chose the OnePlus 7 from the list of phones in the article that sup provided.

In good lighting, the Moto Z Play wins easily in texture detail and resolving power.  Look at the street map: well-defined edges and small but legible text.  Look at the fabric patches: sharp, detailed texture.  It trails the OnePlus 7 in the paper currency detail, however, but that's the exception.

Even in low light (click the "Low light" image), the Moto Z at worst matches the image quality of the Oppo (and smokes the OnePlus).  At best, it looks much more natural (to me) than the Oppo with its interpolated image.  I would have expected the Oppo's 48MP sensor to perform much better than the Moto Z Play's in low light, but this is apparently not the case.  The Moto Z Play also easily beat the ZenFone 6 (false color artifacts and noise), the Honor View 20 (smeared away all detail), and the Xiaomi Mi 9 (smeared away all detail).  The ZTE Axon 10 Pro is not available in the comparison tool.

My point of this demonstration is two-fold.  First, according to the marketing, the 48MP sensors should be blowing away the 16MP in detail resolution and texture reproduction, especially in low light, but in almost all cases they fall behind, often way behind.  Second, the Oppo (MediaTek) images looks pretty bad in my opinion, with lots of jagged edges and halos due to interpolation and oversharpening.  And that's an improvement over the first Oppo 48MP model I picked, the Oppo F11 Pro.  Compare the route colors on the map with the Moto Z Play or the OnePlus 7.  They're all wrong, just plain wrong!  They look like the burnt colors from the Cosmo under the MediaTek stack.  So there's a smidgen of hope that the Astro's color reproduction will be improved over the Cosmo's using MediaTek-based image capture, but don't expect much of anything for sharpness and texture.  And above all, don't fall for the marketing hype.

23

Astro Slide - Hardware / Re: Camera - is it IMX586?

« on: January 16, 2021, 06:13:28 am »

I took the liberty of transcribing Planet's answer to a backer's question about the Astro's expected camera quality.  This is from Indiegogo's Astro Slide update #20, January 15, 2021:

"The second video addresses the questions that we were asked by you, our backers, this week."

https://youtu.be/SgQ8NTituUA 8:02-9:24

CHRIS BIGNELL - XL COMMUNICATIONS: "Entienne asks about the camera quality in terms of uh, you know, the, within the Cosmo, the MediaTek stack did not give um, did not support the full capabilities of the sensor, uh, that I know is something that has been improved over time, uh, is there likely to be, uh is that likely to be a problem with Astro Slide, or is that something that we think we've resolved?"

DR JANKO MRSIC-FLOGEL - CEO, PLANET: "So we think there'll be a huge improvement in the camera, uh, one because uh, the uh, the sensor is a Sony sensor, it's a high quality, you know, Tier 1 sensor, and uh, uh, the pre-- it's 48 megapixel and high quality with uh, um, standard oriented, so basically uh uh um uh, a square-type orientation, um, um, a square-type organization of the CMOS, whereas the uh, CMOS sensor, whereas the, uh, the previous sensor was a 24, uh, megapixel sensor from Samsung which has a hexagonal organization of CMOS, so basically it's, it's far be-- the n-- the new sensor not just that it has a larger resolution but it's uh, uh, the-- the organization of pre-- the CMOS sensor is actually better so you'll get a, a far superior performance."

CHRIS BIGNELL: "Good to hear, thank you, and I'd also like to thank Felagund and Colin W., two other backers that asked the same question around the camera."

God bless them for trying and I wish them the best, but wow... Planet has absolutely zero understanding of the Cosmo camera issue. The backer understood.  He obviously read our Cosmo camera forum posts.  Planet did not!  The issue lies entirely with the MediaTek software stack!  It's not a question of needing higher resolution.  It's not a question of "CMOS hexagonal organization" versus "CMOS square organization" (whatever that means).  The sensor itself, as we have proven, takes sharp, detailed, beautiful photos with the right software: Google Camera ported to the Cosmo (https://www.oesf.org/forum/index.php?topic=35912.msg297145#msg297145) which completely bypasses the MediaTek imaging stack for photos.

Regarding hexagonal or square pixel arrays, Planet's explanation truly left me scratching my head.  Samsung's Tetracell and Sony's Quad Bayer are both Bayer filter mosaics in a square pixel array, albeit with multiple adjacent pixels for each color.  But the colors are still staggered.  Neither is a hexagonal pixel array, which is very uncommon, and if the Cosmo's Samsung sensor were, it would actually be a computational improvement (but not an image improvement).  And neither has any bearing on MediaTek's inability to properly process raw image data.

It's also amazing how the Cosmo camera issue magically "has been improved over time."  Wonder how that happened.

I think GCam would produce fantastic photos with a 48MP Quad Bayer sensor.  But for the record, I am not backing the Astro.

24

Cosmo Communicator - General Discussion / Re: Camera

« on: January 04, 2021, 01:41:44 am »

The "camera sounds" setting that existed in v3 is no longer available in v4.
I will wish the "camera sounds" setting was still there.

Odd.  I see the sounds settings on mine (same v4 that I released), and if the ringer is enabled, I hear a shutter sound.

Does anyone else have the issue spa is seeing?

25

Cosmo Communicator - General Discussion / Re: Camera

« on: December 31, 2020, 08:50:37 pm »

Merry Christmas and Happy New Year!  GCam for the Cosmo v4 is here!

Download v4 here:
https://www.dropbox.com/s/oj7p3a0qbauylbl/gcam_for_cosmo_v4.apk?dl=1

New for v4, which has been ported fresh from scratch using stock Google Camera 7.2.018:
1. The option to disable HDR+ now works.  Note that this saves interpolated 24MP photos.  The stream from the sensor in this mode is YUV_420_888, not RAW, and RAW file saving is disabled in this case.  There is no difference in detail between Samsung's marketing-driven interpolated 24MP photos and 6MP photos taken in "HDR+ on" mode.  Don't blame me, the YUV stream is 24MP and GCam is just saving it as such.  I didn't see a convenient way in the code to resize it but I might look at this again in the future.
2. Video stabilization has been added.  This is electronic, not optical, so there is a crop (in addition to the crop already incurred by video mode).
3. Night Sight now works, though it is only marginally better than Enhanced HDR+ in low light due to excessive noise.
4. Viewfinder resolution has increased from 640x480 to 1920x1080 (but is neither faster nor slower than before).  More scene detail is visible in the preview now.
5. Slow Motion has been removed.  It will never work on the Cosmo, which can only output 30 FPS maximum.  Slow motion requires 60+ FPS.
6. Panorama has been fixed.  It uses MTK video libraries with smearing and washed-out colors, so don't expect too much from it.
7. Microvideo now works (Motion).  The files are saved as MVIMG_YYYYMMDD_HHMMSS.jpg, but they contain an MP4 video concatenated onto the end of the JPEG data.  Tap "ALL PHOTOS" in the photo viewer to switch to Google Photos where the microvideo will play.  There is a bug that garbles the timeline color on the Cosmo on microvideo files captured in "HDR+ on" mode (but not in "HDR+ off" mode for some reason).  However, the microvideo plays correctly, and on non-Cosmo devices, the timeline displays properly in Google Photos.  To extract the microvideo from the image in Google Photos, scroll down a bit, tap Export, Export as Video, Export.
8. Removed Time Lapse since after extensive debugging it does not appear likely to work on the Cosmo.
9. The tap-to-focus bug in Enhanced HDR+ mode is actually an overexposure issue with the Cosmo sensor.  Tapping to focus changed the exposure, but the sensor was tuned for the Pixel's sensor, not the Cosmo's, and the code invalidated the shot due to perceived overexposure.  This bug still exists as it's very hard to isolate, but I have greatly improved the success rate for enhanced HDR+ shots.  Previously many would fail before even tapping to focus.  This is now fixed with some tweaks to the rear sensor exposure.  This should alleviate the temptation to tap to focus after a shot failure, which enters the buggy state.  Keep in mind that enhanced HDR+ is best in dimmer lighting.  Aim it at the sun and you'll get a white picture.
10. Lens Blur has been added.  This takes a 2560x1920 photo and allows you to change the focus point and the amount of background blur.

I want to thank san1ty, the developer of F1MinimalMod, for creating his GCam mod.  Although v4 is an entirely new port and no longer based on F1MinimalMod, we wouldn't even have GCam for the Cosmo if I hadn't discovered that his port somewhat worked on the Cosmo.  His work was also instrumental in helping me get past three problems in my new v4 mod.

26

Cosmo Communicator - General Discussion / Re: Camera

« on: December 02, 2020, 08:18:38 pm »

Thank you Shuntcap. By the way, and I of courrse have zeo expectations, have you ordered Astro? I am afraid it would need something similar because I have vey low expectations from Planet Computers when it comes to photography.

No, I have not backed the Astro and I do not plan to.  I did back the Pro1 X, though.  As a programmer, I prefer a keyboard with a lot more keys than what Planet offers, even if I won't be able to touch type on it.  I also prefer an organization that is more transparent with its backers.  Plus, the camera in the  Pro1 X is a known entity, the IMX363, the same camera sensor used in the Pixel 3, 4, and 5.

You may be right about the Astro's camera.  Given how bad the Gemini's rear camera was, and how badly MTK butchered the Cosmo's rear camera, and given that Astro's front camera is likely still the same low quality 5MP camera that the Gemini and Cosmo have (based on Astro's specs), we backers and consumers have every reason to anticipate the worst for the Astro's camera.  It's simply Planet's track record.  If they can break that, great.

27

Cosmo Communicator - General Discussion / Re: Camera

« on: December 02, 2020, 12:22:16 am »

It's now been over a month since my last release.  I haven't been able to fix the HDR+ capture issue (capture may fail without a tap-to-focus), nor have I been able to fix the slow still image preview frame rate.  This will probably be my last release.  I'm getting tired of working on GCam (or just getting old and tired from working on GCam...)

Download Version 3 of GCam for the Cosmo here:
https://www.dropbox.com/s/t36cufk1se59uri/gcam_for_cosmo_v3.apk?dl=1

Version 3 just has two small fixes:
1. Tap-to-focus animator circle now appears where tapped instead of being rotated 90 degrees CCW.
2. Minor increase in captured image sizes (RAW is now 2816x2108 from 2800x2092, JPEG is now 2832x2124 from 2816x2108).  I don't know why the RAW is slightly smaller than the JPEG, but I am capturing at the sensor's reported active array size of 2832x2128.

Sorry to disappoint.

For those interested in the underlying details, I have spent countless hours debugging this thing, and I actually did speed up the preview considerably by changing the sensor output stream format from RAW to YUV_420_888, but that resulted in GCam being unable to capture images.  It really needs a RAW stream, which is just too much data for the Cosmo hardware to send at a high rate.  I ported an older version of GCam (6.1 for the Pixel 3) which had a high preview frame rate and verified that it was indeed using a YUV stream, but the images it saves are noticeably inferior to the current version and are closer to those produced by the stock camera app.  It also fails to save RAW images.  Regarding HDR+ capture failure, I traced that down to an odd state in the autofocus system that, as far as I can tell from the Camera2 API documentation, should not happen.  This odd state is entered when the user taps the screen to lock focus while in HDR+ mode, then quickly moves or pans the scene which triggers a reset of the autofocus mode (from AUTO to CONTINUOUS_PICTURE), but the focus state does not reset to passive scan/lock.  I don't see this behavior in OpenCamera, the stock camera app, or in the older version of GCam I ported, but none of those produces images as crisp and chromatically accurate as the current version.  The workaround is to either restart GCam or simply tap to focus again.  If the screen has never been tapped to focus, HDR+ appears to work without issue.

28

Cosmo Communicator - General Discussion / Re: Camera

« on: October 29, 2020, 07:07:24 pm »

It's just the visual feedback of that tap that gets drawn in a rotated fashion...

I will poke around to see if I can fix it, along with the other list of issues.

... and it's fixed!  I wish all the fixes were that fast.  Unless anyone is in a hurry, I'll just wait until I have some other fixes (working on HDR+ issue) before releasing.

29

Cosmo Communicator - General Discussion / Re: Camera

« on: October 29, 2020, 06:51:40 pm »

It's just the visual feedback of that tap that gets drawn in a rotated fashion...

Ah, now I understand.  Being the extreme minimalist that I am, I had disabled all animations in the developer options menu, so I never even saw any circular tap animation.  This might be the first time that I actually see a use for "eye candy" in a GUI.  Of course, it needs to be rendered properly to be useful.  I will poke around to see if I can fix it, along with the other list of issues.

30

Cosmo Communicator - General Discussion / Re: Camera

« on: October 28, 2020, 12:59:30 pm »

In my experience, touch focus behaves the same way in both v1 and v2 of GCam for the Cosmo. If I tap the screen at "12 o'clock", the screen shows the tap at "9 o'clock". If I tap at "3 o'clock, the screen shows the tap at "12 o'clock" and so on. The touch focus point ends up the correct distance from the center of the viewfinder, but rotated 90 degrees counter-clockwise. To me, it seems GCam thinks it's in portrait mode and somehow gets confused. The way Gcam draws the GUI seems to support this theory. With either of the grid options activated, and knowing I have to tap the desired distance from the center, but 90 degrees clockwise ahead of where I actually want to focus, I have been able to sort of work around this issue (but, if possible, please fix it). Tapping the viewfinder dead center works correctly, as that's the same spot, regardless of rotation.

Are you saying that even in the options grid, you have to manually "rotate" your finger position 90 degrees CCW to register the correct tap location?

As for the viewfinder, I don't find that the tap-to-focus is rotated.  In fact, I added a debug print and GCam does indeed believe it's in landscape mode.  I set up a test scene with objects at various distances and in different viewfinder quadrants, and each tap focuses correctly.  Furthermore, the padlock icon on the EV slider bar is oriented correctly for landscape layout.  If I force-rotate the Cosmo into portrait, the padlock then orients correctly for portrait layout.  Tap-to-focus also works correctly in portrait.

The EV slider actually works correctly according to the brightness icon, which is black on the right half and white on the left half.  So according to the icon, sliding left should brighten the image, while sliding right should darken it, which is exactly what it does.  I originally thought the slider wasn't even working, but that was because I was already in a dark environment and GCam had automatically adjusted the brightness to its maximum.

My Cosmo build number, if it matters, is v23 (Cosmo-9.0-Planet-07062020-V23).

Is anyone else experiencing rotated taps?