[GUIDE][Build|Mod|Update][kernel-ranchu][goldfish][5.4][5.10][GKI][ramdisk.img][modules][rootAVD][Android 11(R) 12(S)][AVD][Google Play Store API] - Android Software/Hacking General [Developers Only]

[GUIDE][Build|Mod|Update][kernel-ranchu][goldfish][5.4][5.10][GKI][ramdisk.img][modules][rootAVD][Android 11(R) 12(S)][AVD][Google Play Store API] - Android Software/Hacking General [Developers Only]

Hello Fellows,
with this Guide I would like to show you a much more easier and reliable way,
on how to build, mod and update your AVDs Kernel with its modules.
By using the official AOSP Build ENV.
The "classic" guide will remain at the end of this thread within a spoiler.
The Development Environment:
Apple Macbook Pro 2015 Dualboot
Linux Mint 19
Android Studio 4.1.3 (Mac OS/Darwin)
Android emulator version 30.5.4.0 (build_id 7243153)
Android SDK Platform-Tools (revision: 31.0.2)
AVDs:
Android 10 (Q) API 29 Google Apis Play Store x86_64 r08 Darwin/MacOS Production Build
Kernel 4.14.112+ -> 4.14.175
Android 11 (R) API 30 Google Apis Play Store x86_64 r10 Darwin/MacOS Production Build
Kernel 5.4.61 -> 5.4.113
Android 12 (S) API 30 Google Apis Play Store x86_64 r02 Darwin/MacOS Production Build
Kernel 5.10.15 -> 5.10.31
rootAVD - To install Magisk and the Kernels Modules into the ramdisk.img
Spoiler: dependencies and repo
Bash:
#############################################|#############################################
###### Build an AVD Kernel and its Ramdisk Modules ######
###### with the official AOSP Build ENV ######
#############################################|#############################################
#############################################|#############################################
###### Install build dependencies, libs and tools ######
#############################################|#############################################
sudo apt-get install -y build-essential libssl-dev kernel-package libncurses5-dev bzip2 \
lib32z1 bison flex libelf-dev qt5-default qttools5-dev-tools qttools5-dev meld geany \
gtk+-2.0 libgtk-3-dev libwebkit2gtk-4.0-dev autogen libgtk2.0-dev libglade2-dev
#############################################|#############################################
###### Get the AOSP repo bin ######
#############################################|#############################################
sudo wget https://storage.googleapis.com/git-repo-downloads/repo -O /usr/bin/repo
sudo chmod a+x /usr/bin/repo
Spoiler: Android 10 (Q) Kernel 4.14
Bash:
#############################################|#############################################
##### Android 10 (Q) Kernel 4.14 #####
##### BRANCH=q-goldfish-android-goldfish-4.14-dev #####
#############################################|#############################################
#############################################|#############################################
### Download Sources, Toolchain, Buildtools etc. (approx. 17GB) ###
#############################################|#############################################
BRANCH=q-goldfish-android-goldfish-4.14-dev
ROOTDIR=AVD-kernel-$BRANCH
mkdir $ROOTDIR && cd $ROOTDIR
repo init --depth=1 -u https://android.googlesource.com/kernel/manifest -b $BRANCH
repo sync --force-sync --no-clone-bundle --no-tags -j$(nproc)
#############################################|#############################################
### Make changes and enable features via the Menuconfig ###
### changes will be saved into the gki_defconfig ###
### i.e. USB 3.0, UHCI HCD for USB-Serial Adapters ###
### or USB Block Devices like /dev/block/sda1 ###
### Device Drivers -> USB support -> <*> xHCI HCD (USB 3.0) support ###
### -*- Generic xHCI driver for a platform device ###
### <*> UHCI HCD (most Intel and VIA) support ###
### <*> USB Mass Storage support ###
### <*> USB Attached SCSI ###
#############################################|#############################################
BUILD_CONFIG=goldfish/build.config.goldfish.x86_64 \
build/config.sh
#############################################|#############################################
### 1st run ###
### Building the Kernel (bzImage) ###
#############################################|#############################################
BUILD_CONFIG=goldfish/build.config.goldfish.x86_64 \
SKIP_CP_KERNEL_HDR=1 \
build/build.sh -j$(nproc)
Files copied to /home/newbit/workdir/AVD-kernel-q-goldfish-android-goldfish-4.14-dev/out/x86_64/dist
#############################################|#############################################
### (1+n)th Build run ###
### Building only changes, not everything ###
#############################################|#############################################
BUILD_CONFIG=goldfish/build.config.goldfish.x86_64 \
SKIP_CP_KERNEL_HDR=1 \
SKIP_MRPROPER=1 \
build/build.sh -j$(nproc)
#############################################|#############################################
### Copy bzImage as kernel-ranchu into the AVDs directory ###
#############################################|#############################################
mv ~/path-to-avd-system-images/android-29/kernel-ranchu ~/path-to-avd-system-images/android-29/kernel-ranchu.backup
cp out/x86_64/dist/bzImage ~/path-to-avd-system-images/android-29/kernel-ranchu
#############################################|#############################################
### Install Magisk with the rootAVD script, ###
### download the USB HOST Permissions Module Zip ###
### patch the fstab.ranchu to automount Block Devices like /dev/block/sda1 ###
#############################################|#############################################
./rootAVD.sh ~/Android/Sdk/system-images/android-29/google_apis_playstore/x86_64/ramdisk.img GetUSBHPmodZ PATCHFSTAB
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Spoiler: Android 11 (R) Kernel 5.4
Bash:
#############################################|#############################################
##### Android 11 (R) Kernel 5.4 #####
##### BRANCH=common-android11-5.4-lts #####
#############################################|#############################################
#############################################|#############################################
### Download Sources, Toolchain, Buildtools etc. (approx. 22GB) ###
#############################################|#############################################
BRANCH=common-android11-5.4-lts
ROOTDIR=AVD-kernel-$BRANCH
mkdir $ROOTDIR && cd $ROOTDIR
repo init --depth=1 -u https://android.googlesource.com/kernel/manifest -b $BRANCH
repo sync --force-sync --no-clone-bundle --no-tags -j$(nproc)
#############################################|#############################################
### Preparing and modding the build.config ###
### add these lines to the BUILD_CONFIG file ###
### common-modules/virtual-device/build.config.goldfish.x86_64 ###
#############################################|#############################################
BUILD_INITRAMFS=1
LZ4_RAMDISK=1
SKIP_CP_KERNEL_HDR=1
#############################################|#############################################
### Make changes and enable features via the Menuconfig ###
### changes will be saved into the gki_defconfig ###
### i.e. USB 3.0 or UHCI HCD for USB-Serial Adapters ###
### Device Drivers -> USB support -> <*> xHCI HCD (USB 3.0) support ###
### -*- Generic xHCI driver for a platform device ###
### <*> UHCI HCD (most Intel and VIA) support ###
#############################################|#############################################
BUILD_CONFIG=common-modules/virtual-device/build.config.goldfish.x86_64 \
FRAGMENT_CONFIG=common/arch/x86/configs/gki_defconfig \
build/config.sh
#############################################|#############################################
### 1st run ###
### Building the Modules and Kernel ###
#############################################|#############################################
BUILD_CONFIG=common-modules/virtual-device/build.config.goldfish.x86_64 \
build/build.sh -j$(nproc)
Files copied to ~/workdir/AVD-kernel-common-android11-5.4-lts/out/android11-5.4/dist
#############################################|#############################################
### Copy bzImage & initramfs.img into the rootAVD directory ###
### and run rootAVD ###
#############################################|#############################################
cp out/android11-5.4/dist/initramfs.img ~/rootAVD/
cp out/android11-5.4/dist/bzImage ~/rootAVD/
./rootAVD.sh ~/path-to-avd-system-images/android-30/ramdisk.img InstallKernelModules
[!] Installing new Kernel Modules
[*] Copy initramfs.img /data/data/com.android.shell/Magisk/tmp/initramfs
[-] Extracting Modules from initramfs.img
Detected format: [lz4_legacy]
Decompressing to [initramfs.cpio]
[*] Removing Stock Modules from ramdisk.img
[!] 5.4.61-android11-2-00064-g4271ad6e8ade-ab6991359
[!] Android (6443078 based on r383902)
[-] Installing new Modules into ramdisk.img
[!] 5.4.113-android11-2-g926c4200b8fc-dirty
[!] Android (7211189, based on r416183)
[*] Adjusting modules.load and modules.dep
[*] Repacking ramdisk ..
#############################################|#############################################
### (1+n)th Build run ###
### Building only changes, not everything ###
#############################################|#############################################
BUILD_CONFIG=common-modules/virtual-device/build.config.goldfish.x86_64 \
SKIP_MRPROPER=1 \
build/build.sh -j$(nproc)
#############################################|#############################################
### Copy bzImage as kernel-ranchu into the AVDs directory ###
### Once the modules are installed, just the kernel is needed ###
#############################################|#############################################
cp out/android11-5.4/dist/bzImage ~/path-to-avd-system-images/android-30/kernel-ranchu
Spoiler: Android 12 (S) Kernel 5.10
Bash:
#############################################|#############################################
##### Android 12 (S) Kernel 5.10 #####
##### BRANCH=common-android12-5.10-lts #####
#############################################|#############################################
#############################################|#############################################
### Download Sources, Toolchain, Buildtools etc. (approx. 22GB) ###
#############################################|#############################################
BRANCH=common-android12-5.10-lts
ROOTDIR=AVD-kernel-$BRANCH
mkdir $ROOTDIR && cd $ROOTDIR
repo init --depth=1 -u https://android.googlesource.com/kernel/manifest -b $BRANCH
repo sync --force-sync --no-clone-bundle --no-tags -j$(nproc)
#############################################|#############################################
### Preparing and modding the build.config ###
### add these lines to the BUILD_CONFIG file ###
### common-modules/virtual-device/build.config.virtual_device.x86_64 ###
#############################################|#############################################
SKIP_CP_KERNEL_HDR=1
FILES="
arch/x86/boot/bzImage
vmlinux
System.map
"
MAKE_GOALS="
bzImage
modules
"
#############################################|#############################################
### Open the Menuconfig, make changes and enable features ###
### changes will be saved into the gki_defconfig ###
### i.e. UHCI HCD for USB-Serial Adapters ###
### USB 3.0 is finally standard built in the ranchu-kernel since Android 12 (S) ###
### Device Drivers -> USB support -> <*> UHCI HCD (most Intel and VIA) support ###
#############################################|#############################################
BUILD_CONFIG=common-modules/virtual-device/build.config.virtual_device.x86_64 \
FRAGMENT_CONFIG=common/arch/x86/configs/gki_defconfig \
build/config.sh
#############################################|#############################################
### 1st Build run ###
### Building the modules and kernel ###
#############################################|#############################################
time BUILD_CONFIG=common-modules/virtual-device/build.config.virtual_device.x86_64 \
build/build.sh -j$(nproc)
Files copied to ~/workdir/AVD-kernel-common-android12-5.10-lts/out/android12-5.10/dist
real 33m16,742s
user 79m12,894s
sys 11m33,474s
#############################################|#############################################
### Copy bzImage & initramfs.img into the rootAVD directory ###
### and run rootAVD ###
### the AVD is running with the new Kernel and Modules ###
### complete the Magisk installation with EnvFixTask ###
#############################################|#############################################
cp out/android12-5.10/dist/initramfs.img ~/rootAVD/
cp out/android12-5.10/dist/bzImage ~/rootAVD/
./rootAVD.sh ~/path-to-avd-system-images/android-S/ramdisk.img InstallKernelModules
[!] Installing new Kernel Modules
[*] Copy initramfs.img /data/data/com.android.shell/Magisk/tmp/initramfs
[-] Extracting Modules from initramfs.img
Detected format: [lz4_legacy]
Decompressing to [initramfs.cpio]
[*] Removing Stock Modules from ramdisk.img
[!] 5.10.15-android12-0-00490-gfca78df78ef2-ab7137072
[!] Android (7037181, based on r407598)
[-] Installing new Modules into ramdisk.img
[!] 5.10.31-android12-1-gb0c3c31639b2-dirty
[!] Android (7211189, based on r416183)
[*] Adjusting modules.load and modules.dep
[*] Repacking ramdisk ..
./rootAVD.sh EnvFixTask
#############################################|#############################################
### (1+n)th Build run ###
### Building only changes, not everything ###
#############################################|#############################################
time BUILD_CONFIG=common-modules/virtual-device/build.config.virtual_device.x86_64 \
SKIP_MRPROPER=1 \
build/build.sh -j$(nproc)
#############################################|#############################################
### Copy bzImage as kernel-ranchu into the AVDs directory ###
### Once the modules are installed, just the kernel is needed ###
#############################################|#############################################
cp out/android12-5.10/dist/bzImage ~/path-to-avd-system-images/android-S/kernel-ranchu
Spoiler: "classic" Guide
with this Guide I would like to show some steps on how to build your own Kernel for an Android Studio Device with CLANG.
The Development Environment:
Apple Macbook Pro 2011 Dualboot
Linux Mint 20 Ulyana
Android Studio 4.1.1 (Software Manager)
KVM -> Cosmic (18.10) or later
Android emulator version 30.3.5.0 (build_id 7033400)
Android SDK Platform-Tools (revision: 30.0.5)
AVD: Android 11 (R) API 30 Google Apis Play Store x86_64 Linux Production Build (revision: 10)
Target: Kernel 5.4.61
### Install the following tools to work with and to build the kernel
Bash:
sudo apt-get install -y build-essential libssl-dev kernel-package libncurses5-dev bzip2 lib32z1 bison flex
sudo apt-get install -y libelf-dev libelf-devel or elfutils-libelf-dev
sudo apt-get install -y qt5-default qttools5-dev-tools qttools5-dev
sudo apt-get install -y geany git
### Gathering Informations about the Target Kernel
By starting the AVD over the command line, we can append some kernel and verbose options:
emulator -netdelay none -netspeed full -no-snapstorage -avd Pixel_4_API_30 -show-kernel -verbose
The terminal will show the kernel information which are important:
Code:
[ 0.000000] Linux version 5.4.61-android11-2-00064-g4271ad6e8ade-ab6991359 ([email protected]) (Android (6443078 based on r383902) clang version 11.0.1 (https://android.googlesource.com/toolchain/llvm-project b397f81060ce6d701042b782172ed13bee898b79), LLD 11.0.1 (/buildbot/tmp/tmp6_m7QH b397f81060ce6d701042b782172ed13bee898b79)) #1 SMP PREEMPT Mon Nov 23 17:45:44 UTC 2020
ValueMeaningLinux version 5.4.61Kernel VERSION.PATCHLEVEL.SUBLEVELandroid11-2retagged Kernel branch and KMI generation number -> android11-5.4-g4271ad6e8adeLocal Version String (very important to avoid: disagrees about version of symbol module_layout)-ab6991359Automatically append version information to the version stringbuild-userKBUILD_BUILD_USERbuild-hostKBUILD_BUILD_HOSTAndroid (6443078 based on r383902) clang version 11.0.1Android Clang/LLVM Prebuilt Version r383902
### Gathering more Informations about what is needed to build the kernel
To figure out what we need to build the kernel, one just needs to look into the build.config file inside the kernel source.
https://android.googlesource.com/kernel/common/+/refs/heads/android11-5.4
build.config.gki.x86_64
build.config.common
build.config.x86_64
build.config.gki
Inside the three build.config files we will find what we need to know.
Makefile:
BRANCH=android11-5.4
KMI_GENERATION=2
CC=clang
LD=ld.lld
NM=llvm-nm
OBJCOPY=llvm-objcopy
DEPMOD=depmod
CLANG_PREBUILT_BIN=prebuilts-master/clang/host/linux-x86/clang-r383902/bin
BUILDTOOLS_PREBUILT_BIN=build/build-tools/path/linux-x86
EXTRA_CMDS=''
STOP_SHIP_TRACEPRINTK=1
IN_KERNEL_MODULES=1
DO_NOT_STRIP_MODULES=1
Makefile:
ARCH=x86_64
CLANG_TRIPLE=x86_64-linux-gnu-
CROSS_COMPILE=x86_64-linux-androidkernel-
LINUX_GCC_CROSS_COMPILE_PREBUILTS_BIN=prebuilts/gcc/linux-x86/x86/x86_64-linux-android-4.9/bin
FILES="arch/x86/boot/bzImage
vmlinux
System.map"
Makefile:
DEFCONFIG=gki_defconfig
POST_DEFCONFIG_CMDS="check_defconfig"
Apparently we don't need the build.config.gki
Without further ado, the gathered informations from above will eventually lead up to:
### Download the kernel source, CLANG toolchain, Buildtools and GCC
Bash:
mkdir avdkernel5.4compile && cd avdkernel5.4compile
git clone -b android11-5.4 --single-branch https://android.googlesource.com/kernel/common
mkdir clang-r383902 && cd clang-r383902
wget https://android.googlesource.com/platform/prebuilts/clang/host/linux-x86/+archive/android-11.0.0_r28/clang-r383902.tar.gz
tar -xzf clang-r383902.tar.gz && cd ..
git clone https://android.googlesource.com/kernel/prebuilts/build-tools
git clone \
-b android-11.0.0_r28 \
--single-branch https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/x86/x86_64-linux-android-4.9
### Pull the Kernel .config from the AVD
Bash:
adb pull /proc/config.gz
gunzip -k config.gz
### Exports: (must be done in kernel-source directory)
Bash:
cd common
export ARCH=x86_64
export CLANG_TRIPLE=x86_64-linux-gnu-
export CROSS_COMPILE=x86_64-linux-androidkernel-
export LINUX_GCC_CROSS_COMPILE_PREBUILTS_BIN=$(pwd)/../x86_64-linux-android-4.9/bin
export CLANG_PREBUILT_BIN=$(pwd)/../clang-r383902/bin
export BUILDTOOLS_PREBUILT_BIN=$(pwd)/../build-tools/linux-x86/bin
DEVEXPS="CC=clang LD=ld.lld NM=llvm-nm OBJCOPY=llvm-objcopy DEPMOD=depmod EXTRA_CMDS='' STOP_SHIP_TRACEPRINTK=1 DO_NOT_STRIP_MODULES=1"
export KBUILD_BUILD_USER=build-user
export KBUILD_BUILD_HOST=build-host
export PATH=$LINUX_GCC_CROSS_COMPILE_PREBUILTS_BIN:$CLANG_PREBUILT_BIN:$BUILDTOOLS_PREBUILT_BIN:$PATH
make $DEVEXPS mrproper
cp ../config .config
### add -g4271ad6e8ade to LOCALVERSION -> otherwise modules won't load => module xy disagrees about version of symbol module_layout
Bash:
make $DEVEXPS xconfig
If you come back here later, this is the right place to mod your kernel.
I like the GUI Interface because you have a better overview and a search function.
### build it
Bash:
time make $DEVEXPS -j$(nproc)
### Boot the AVD ...
#### ... from the command line
Bash:
emulator -netdelay none -netspeed full -no-snapstorage -avd Pixel_4_API_30 \
-no-snapshot-load \
-show-kernel \
-verbose \
-ranchu \
-kernel ~/avdkernel5.4compile/common/arch/x86/boot/bzImage
#### ... by replacing the stock kernel-ranchu
Code:
mv ~/Android/Sdk/system-images/android-30/google_apis_playstore/x86_64/kernel-ranchu \
~/Android/Sdk/system-images/android-30/google_apis_playstore/x86_64/kernel-ranchu-backup
cp ~/avdkernel5.4compile/common/arch/x86/boot/bzImage \
~/Android/Sdk/system-images/android-30/google_apis_playstore/x86_64/kernel-ranchu
### Compare the Kernel Verbose Messages
Code:
[ 0.000000] Linux version 5.4.61-g4271ad6e8ade-00014-g158eae717346 ([email protected]) (Android (6443078 based on r383902) clang version 11.0.1 (https://android.googlesource.com/toolchain/llvm-project b397f81060ce6d701042b782172ed13bee898b79), LLD 11.0.1 (/buildbot/tmp/tmp6_m7QH b397f81060ce6d701042b782172ed13bee898b79)) #1 SMP PREEMPT Mon Jan 11 12:58:18 CET 2021
Thanks for reading
Cheers NewBit
Thanks and Credits to @nathanchance, for his wonderful
[REFERENCE] How to compile an Android kernel
brought me a lot of ideas and inspirations

Hello newbit,
I am currently following your guide but when I try to run
Bash:
make $DEVEXPS xconfig
and it resulted in the following error
Bash:
clang: error: no such file or directory: 'LD=ld.lld'
clang: error: no such file or directory: 'NM=llvm-nm'
clang: error: no such file or directory: 'OBJCOPY=llvm-objcopy'
clang: error: no such file or directory: 'DEPMOD=depmod'
clang: error: no such file or directory: 'EXTRA_CMDS='
clang: error: no such file or directory: 'STOP_SHIP_TRACEPRINTK=1'
clang: error: no such file or directory: 'DO_NOT_STRIP_MODULES=1'
clang: error: no such file or directory: 'LD=ld.lld'
clang: error: no such file or directory: 'NM=llvm-nm'
clang: error: no such file or directory: 'OBJCOPY=llvm-objcopy'
clang: error: no such file or directory: 'DEPMOD=depmod'
clang: error: no such file or directory: 'EXTRA_CMDS='
clang: error: no such file or directory: 'STOP_SHIP_TRACEPRINTK=1'
clang: error: no such file or directory: 'DO_NOT_STRIP_MODULES=1'
clang: error: no such file or directory: 'LD=ld.lld'
clang: error: no such file or directory: 'NM=llvm-nm'
clang: error: no such file or directory: 'OBJCOPY=llvm-objcopy'
clang: error: no such file or directory: 'DEPMOD=depmod'
clang: error: no such file or directory: 'EXTRA_CMDS='
clang: error: no such file or directory: 'STOP_SHIP_TRACEPRINTK=1'
clang: error: no such file or directory: 'DO_NOT_STRIP_MODULES=1'
clang: error: no such file or directory: 'LD=ld.lld'
clang: error: no such file or directory: 'NM=llvm-nm'
clang: error: no such file or directory: 'OBJCOPY=llvm-objcopy'
clang: error: no such file or directory: 'DEPMOD=depmod'
clang: error: no such file or directory: 'EXTRA_CMDS='
clang: error: no such file or directory: 'STOP_SHIP_TRACEPRINTK=1'
clang: error: no such file or directory: 'DO_NOT_STRIP_MODULES=1'
clang: error: no such file or directory: 'LD=ld.lld'
clang: error: no such file or directory: 'NM=llvm-nm'
clang: error: no such file or directory: 'OBJCOPY=llvm-objcopy'
clang: error: no such file or directory: 'DEPMOD=depmod'
clang: error: no such file or directory: 'EXTRA_CMDS='
clang: error: no such file or directory: 'STOP_SHIP_TRACEPRINTK=1'
clang: error: no such file or directory: 'DO_NOT_STRIP_MODULES=1'
clang: error: no such file or directory: 'LD=ld.lld'
clang: error: no such file or directory: 'NM=llvm-nm'
clang: error: no such file or directory: 'OBJCOPY=llvm-objcopy'
clang: error: no such file or directory: 'DEPMOD=depmod'
clang: error: no such file or directory: 'EXTRA_CMDS='
clang: error: no such file or directory: 'STOP_SHIP_TRACEPRINTK=1'
clang: error: no such file or directory: 'DO_NOT_STRIP_MODULES=1'
clang: error: no such file or directory: 'LD=ld.lld'
clang: error: no such file or directory: 'NM=llvm-nm'
clang: error: no such file or directory: 'OBJCOPY=llvm-objcopy'
clang: error: no such file or directory: 'DEPMOD=depmod'
clang: error: no such file or directory: 'EXTRA_CMDS='
clang: error: no such file or directory: 'STOP_SHIP_TRACEPRINTK=1'
clang: error: no such file or directory: 'DO_NOT_STRIP_MODULES=1'
clang: error: no such file or directory: 'LD=ld.lld'
clang: error: no such file or directory: 'NM=llvm-nm'
clang: error: no such file or directory: 'OBJCOPY=llvm-objcopy'
clang: error: no such file or directory: 'DEPMOD=depmod'
clang: error: no such file or directory: 'EXTRA_CMDS='
clang: error: no such file or directory: 'STOP_SHIP_TRACEPRINTK=1'
clang: error: no such file or directory: 'DO_NOT_STRIP_MODULES=1'
clang: error: no such file or directory: 'LD=ld.lld'
clang: error: no such file or directory: 'NM=llvm-nm'
clang: error: no such file or directory: 'OBJCOPY=llvm-objcopy'
clang: error: no such file or directory: 'DEPMOD=depmod'
clang: error: no such file or directory: 'EXTRA_CMDS='
clang: error: no such file or directory: 'STOP_SHIP_TRACEPRINTK=1'
clang: error: no such file or directory: 'DO_NOT_STRIP_MODULES=1'
init/Kconfig:34:warning: 'GCC_VERSION': number is invalid
init/Kconfig:45:warning: 'CLANG_VERSION': number is invalid
did you encounter this?

reeferman said:
Hello newbit,
I am currently following your guide but when I try to run
Bash:
make $DEVEXPS xconfig
and it resulted in the following error
Bash:
clang: error: no such file or directory: 'LD=ld.lld'
clang: error: no such file or directory: 'NM=llvm-nm'
clang: error: no such file or directory: 'OBJCOPY=llvm-objcopy'
clang: error: no such file or directory: 'DEPMOD=depmod'
clang: error: no such file or directory: 'EXTRA_CMDS='
clang: error: no such file or directory: 'STOP_SHIP_TRACEPRINTK=1'
clang: error: no such file or directory: 'DO_NOT_STRIP_MODULES=1'
...
init/Kconfig:34:warning: 'GCC_VERSION': number is invalid
init/Kconfig:45:warning: 'CLANG_VERSION': number is invalid
did you encounter this?
Click to expand...
Click to collapse
Hmm, looks weird. No I didn't. Also it is a bit taken out of context. It seems that your $DEVEXPS content gets repeated many times.
Do you have more background infos for me? Like your development environment and If you did change little things? Maybe a typo in the $PATH?

Pretty weird, but I was using kali to do it before and when I switched Linux Mint eveerything worked!

Hi newbit!
Thanks for some great tutorials, unfortunately for me I can't get it to work properly. Hopefully, you're able to help me
I'm trying to build and install a custom kernel in an AOSP built emulator. I've been trying out the "official" guide at https://source.android.com/setup/build/building-kernels#customize-build, your rootAVD guide for Android 11 and the "classic" guide for Android 11. All of the guides give me the same result:
Code:
[ 0.610588] Run /init as init process
[ 0.611985] init: init first stage started!
[ 0.612930] init: Loading module /lib/modules/dummy-cpufreq.ko with args ""
[ 0.613579] dummy_cpufreq: disagrees about version of symbol module_layout
[ 0.614170] init: Failed to insmod '/lib/modules/dummy-cpufreq.ko' with args ''
[ 0.614793] init: LoadWithAliases was unable to load dummy_cpufreq
[ 0.615366] init: [libfs_mgr]ReadFstabFromDt(): failed to read fstab from dt
[ 0.616164] init: Using Android DT directory /proc/device-tree/firmware/android/
[ 0.629196] init: bool android::init::BlockDevInitializer::InitDevices(std::set<std::string>): partition(s) not found in /sys, waiting for their uevent(s): metadata, super, vbmeta
After that, the emulator hangs for a short while and the it reboots to try again..
In your "classic" guide, you're warning about this:
### add -g4271ad6e8ade to LOCALVERSION -> otherwise modules won't load => module xy disagrees about version of symbol module_layout
Click to expand...
Click to collapse
When I checked the version of my running emulator i got this:
Code:
Linux version 5.4.50-01145-g056684c0d252-ab6656030 ([email protected]) (Android (6443078 based on r383902) clang version 11.0.1 (https://android.googlesource.com/toolchain/llvm-project b397f81060ce6d701042b782172ed13bee898b79), LLD 11.0.1 (/buildbot/tmp/tmp6_m7QH b397f81060ce6d701042b782172ed13bee898b79)) #1 SMP PREEMPT Mon Jul 6 18:09:10 UTC 2020
So I used xconfig to add the "-g056684c0d252" string and also made sure to use the r383902 clang toolchain to compile, but no luck.. After compilation and running with the new kernel, I get this version print:
Code:
[ 0.000000] Linux version 5.4.162-g056684c0d252-android11-2-00102-gfe0ed45e42fe-dirty ([email protected]) (Android (6443078 based on r383902) clang version 11.0.1 (https://android.googlesource.com/toolchain/llvm-project b397f81060ce6d701042b782172ed13bee898b79), LLD 11.0.1 (/buildbot/tmp/tmp6_m7QH b397f81060ce6d701042b782172ed13bee898b79)) #1 SMP PREEMPT Fri Nov 26 10:38:38 UTC 2021
Can you see anything obvious from that?
When I tried with rootAVD, I'm getting this output when running the script:
Code:
./rootAVD.sh /aosp/out/target/product/myproduct/ramdisk.img InstallKernelModules
[!] and we are NOT in an emulator shell
[*] Set Directorys
[-] Test if ADB SHELL is working
[-] In any AVD via ADB, you can execute code without root in /data/data/com.android.shell
[-] Magisk installer Zip exists already
[*] Cleaning up the ADB working space
[*] Creating the ADB working space
[*] Push Magisk.zip into /data/data/com.android.shell/Magisk
[-] ./Magisk.zip: 1 file pushed, 0 skipped. 254.4 MB/s (6874374 bytes in 0.026s)
[*] create Backup File of ramdisk.img
[*] Push ramdisk.img into /data/data/com.android.shell/Magisk
[-] /ramdisk.img: 1 file pushed, 0 skipped. 1006.6 MB/s (903723 bytes in 0.001s)
[*] Push initramfs.img into /data/data/com.android.shell/Magisk
[-] ./initramfs.img: 1 file pushed, 0 skipped. 299.9 MB/s (2256062 bytes in 0.007s)
[*] Push rootAVD.sh into /data/data/com.android.shell/Magisk
[-] rootAVD.sh: 1 file pushed, 0 skipped. 243.4 MB/s (7826127 bytes in 0.031s)
[-] run the actually Boot/Ramdisk/Kernel Image Patch Script
[*] from Magisk by topjohnwu and modded by NewBit XDA
[!] We are in an emulator shell
[-] Api Level Arch Detect
[-] Device Platform: x64
[-] ARCH32 x86
[-] Device SDK API: 30
[-] First API Level: 28
[-] Switch to the location of the script file
[*] Extracting busybox and Magisk.zip ...
[-] Checking AVDs Internet connection...
[!] AVD is offline
[*] Re-Run rootAVD in Magisk Busybox STANDALONE (D)ASH
[!] We are in an emulator shell
[-] We are now in Magisk Busybox STANDALONE (D)ASH
[*] rootAVD with Magisk '23.0' Installer
[-] Get Flags
[*] System-as-root, keep dm/avb-verity
[-] Encrypted data, keep forceencrypt
[*] RECOVERYMODE=false
[-] KEEPVERITY=true
[*] KEEPFORCEENCRYPT=true
[*] No 64-Bit Binarys found, please consider Magisk Alpha
[-] copy all x86 files from /data/data/com.android.shell/Magisk/lib/x86 to /data/data/com.android.shell/Magisk
[*] Detecting ramdisk.img compression
[!] Ramdisk.img uses gzip compression
[-] taken from shakalaca's MagiskOnEmulator/process.sh
[*] executing ramdisk splitting / extraction / repacking
[-] API level greater then 30
[*] Check if we need to repack ramdisk before patching ..
[*] After decompressing ramdisk.img, magiskboot will work
Detected format: [gzip]
[-] Test patch status and do restore
[-] Checking ramdisk status
[-] STATUS=0
[-] Stock boot image detected
[-] Patching ramdisk
[*] adding overlay.d/sbin folders to ramdisk
Loading cpio: [ramdisk.cpio]
Create directory [overlay.d] (0750)
Create directory [overlay.d/sbin] (0750)
Dump cpio: [ramdisk.cpio]
[!] PATCHFSTAB=false
[!] Skipping fstab.ranchu patch with /dev/block/sda
[?] If you want fstab.ranchu patched, Call rootAVD with PATCHFSTAB
[!] AddRCscripts=false
[!] Skip adding *.rc scripts into ramdisk.img/sbin/*.rc
[?] If you want *.rc scripts added into ramdisk.img/sbin/*.rc, Call rootAVD with AddRCscripts
[!] patching the ramdisk with Magisk Init
Loading cpio: [ramdisk.cpio]
Add entry [init] (0750)
Add entry [overlay.d/sbin/magisk32.xz] (0644)
Add entry [overlay.d/sbin/magisk64.xz] (0644)
Patch with flag KEEPVERITY=[true] KEEPFORCEENCRYPT=[true]
Loading cpio: [ramdisk.cpio.orig]
Backup mismatch entry: [init] -> [.backup/init]
Record new entry: [overlay.d] -> [.backup/.rmlist]
Record new entry: [overlay.d/sbin] -> [.backup/.rmlist]
Record new entry: [overlay.d/sbin/magisk32.xz] -> [.backup/.rmlist]
Record new entry: [overlay.d/sbin/magisk64.xz] -> [.backup/.rmlist]
Create directory [.backup] (0000)
Add entry [.backup/.magisk] (0000)
Dump cpio: [ramdisk.cpio]
[*] repacking back to ramdisk.img format
[!] Rename Magisk.zip to Magisk.apk
[*] Pull ramdiskpatched4AVD.img into ramdisk.img
[-] /data/data/com.android.shell/Magisk/ramdiskpatched4AVD.img: 1 file pulled, 0 skipped. 294.3 MB/s (1270228 bytes in 0.004s)
[*] Pull Magisk.apk into
[-] /data/data/com.android.shell/Magisk/Magisk.apk: 1 file pulled, 0 skipped. 358.8 MB/s (6342346 bytes in 0.017s)
[*] create Backup File of kernel-ranchu
[*] Copy ./bzImage (Kernel) into kernel-ranchu
[-] Clean up the ADB working space
[-] Install all APKs placed in the Apps folder
[*] Trying to install Apps/Magisk.apk
[*] Performing Streamed Install
[*] Success
[-] Shut-Down & Reboot (Cold Boot Now) the AVD and see if it worked
[-] Root and Su with Magisk for Android Studio AVDs
[-] Modded by NewBit XDA - Jan. 2021
[!] Huge Credits and big Thanks to topjohnwu, shakalaca and vvb2060
[-] Trying to shut down the AVD
[!] If the AVD doesn't shut down, try it manually!
Which is not the same as the one you're showing in the rootAVD guide. Is it anything I'm doing incorrect here or has the prints just changed?
Would be really awesome if you're able to help in any way let me know if you need more info!

I was a little quick on asking questions Got it working! It was the Linux version that was off, the one running in the emulator was 5.4.50 while I was compiling 5.4.162.
When I checked out the commit that had 5.4.50, it all worked like a charm doing it with your "classic" guide!
Thanks again for the awesome guides!

rkull said:
I was a little quick on asking questions Got it working! It was the Linux version that was off, the one running in the emulator was 5.4.50 while I was compiling 5.4.162.
When I checked out the commit that had 5.4.50, it all worked like a charm doing it with your "classic" guide!
Thanks again for the awesome guides!
Click to expand...
Click to collapse
I am glade it worked for you. Sometimes you need some luck. I had quite some issues with the module versions, even If I added the local version right. So the best solutions so far is, to build the modules as well.
I see that your AVD is offline, make sure the Magisk Version is compatible with your AVD API Level.
Occasionally, even with the module versions right, the AVD boot looped, and I had to wipe the AVD and then
it worked. I don't have an explanation for this, just my observations.

Yeah I'm building/installing a custom kernel for an AOSP build, so I'm not using an AVD. Just running the AOSP emulator build:
Code:
source build/envsetup.sh
lunch
emulator
Not packing the build into an AVD but maybe we'll do that in the future
When you say
So the best solutions so far is, to build the modules as well.
Click to expand...
Click to collapse
How's that done exactly? Do I need to do something more that just running
Code:
make $DEVEXPS -j$(nproc)
Or am I still stuck with version 5.4.50 of the kernel?
Not that I see the need for me to update the version, but it would be fun to know how it could be done

rkull said:
Yeah I'm building/installing a custom kernel for an AOSP build, so I'm not using an AVD. Just running the AOSP emulator build:
Click to expand...
Click to collapse
rkull said:
When you say
How's that done exactly? Do I need to do something more that just running
Code:
make $DEVEXPS -j$(nproc)
Or am I still stuck with version 5.4.50 of the kernel?
Not that I see the need for me to update the version, but it would be fun to know how it could be done
Click to expand...
Click to collapse
With the AOSP Build ENV I had to set
Code:
BUILD_INITRAMFS=1
to get the modules build and packed. Don't know how this is done the classical way. Checkout your output directory if you can find
any modules or initramfs.img
rootAVD provides you an option to download prebuild kernels and its modules from AOSP.
You can try this to get a higher Kernel version.

Thank you very much！good job！

Related

Simple HOWTO building some linux apps and libraries from scratch on android

This howto is only draft! Sometimes i went upgrade this, or make page on my site and post link on it. Also we have a project for automated building, like buildroot and opkg repository for android.
All needed configs in attached file. If you have error on unpack - just rename my.zip in my.tar.bz2 and try to unpack. Please correct me, if needed.
----------------------------------------------------------------------------------------------------------------------------------
All work will be in home directory (/home/xvilka)
create dir build
mkdir build
1. Download the kernel (for android on your version for device) and unpack
make headers_install ARCH=arm INSTALL_HDR_PATH=~/build/kern_h/
For some optimisations you need know, which proccessor (SoC) in your android-system(gcc.gnu.org/onlinedocs/gcc-4.4.3/gcc/ARM-Options.html#ARM-Options)
Im have Motorola Milestone - SoC = TI OMAP3430 - ARM Cortex a8 (arch - armv7a)
(en.wikipedia.org/wiki/ARM_architecture)
So, if you have OMAP3430-processor - use the config files with -omap3430 suffix
If you have Quallcomm MSM7200A (HTC Hero) - use the config files with -msm7200a suffix
First, we need to define _XXFLAGS, which contain some optimisation options for your compiler:
export _XXCFLAGS=" -mcpu=cortex-a8 -march=armv7-a -mtune=cortex-a8 -mfpu=vfpv3 -O3 -Os" (if you have Motorola Milestone/Droid or other device with TI OMAP3430)
export _XXCFLAGS=" -march=armv6j -mtune=arm1136jf-s -O3 -Os" (if you have HTC Hero or other device with Quallcomm MSM 7200A)
............................................................................................
If you don't know which chip you have in phone use _XXCFLAGS=" -O3 -Os"
2. Download crosstool-ng (ymorin.is-a-geek.org/dokuwiki/projects/crosstool),
unpack, build (you must have make,install,bash,cut,sed,grep,gcc,awk,bison,flex,automake,libtool,stat,
wget,cvs,patch,tar,gzip,bzip2,lzma,readlink,ncurses)
.configure
make
make install
3. Create directory toolchain-android, and copy config to this
XVilka-crosstool-<suffix>.config in .config
XVilka-uClibc-<suffix>.config in uClibc-0.9.30.2.config (or later version, if available)
and then run:
ct-ng menuconfig
you may change some options, if needed!
(Also use options from XXCFLAGS:
Target options -> Target optimizations ->
Architecture-level equal -march, use armv7-a
Emit assembly for CPU equal -mcpu, use cortex-a8
Tune for cpu equal -mtune, use cortex-a8
Use specific FPU equal -mfpu, use vfpv3
Floating point use hardware, because it built-in TI OMAP3430
Target CFLAGS - use -O3 -Os
) - for the Motorola Milestone/Droid
(Also use options from _XXCFLAGS:
Target options -> Target optimizations ->
Architecture-level equal -march, use armv6j
Emit assembly for CPU equal -mcpu, use nothing
Tune for cpu equal -mtune, use arm1136jf-s
Floating point use software
Target CFLAGS - use -O3 -Os
) - for the HTC Hero
Save in .config
In file uClibs-0.9.30.2.config
edit this strings:
{
UCLIBC_HAS_FLOATS=y
UCLIBC_HAS_FPU=y
# UCLIBC_HAS_SOFT_FLOAT=y
UCLIBC_EXTRA_CFLAGS=" -march=armv7-a -mcpu=cortex-a8 -mtune=cortex-a8 -mfpu=vfpv3 -O3 -Os"
} - for the Motorola Milestone/Droid
{
UCLIBC_HAS_FLOATS=y
#UCLIBC_HAS_FPU=y
UCLIBC_HAS_SOFT_FLOAT=y
UCLIBC_EXTRA_CFLAGS=" -march=armv6j -mtune=arm1136jf-s -O3 -Os"
} - for the HTC Hero
and then run
ct-ng build
4. Ok, done. We have installed cross - toolchain in directory build/x-tools.
And all programs in it with names arm-android-linux-uclibsgnueabi-*
For easy using add this directory in PATH:
PATH=~/build/x-tools/arm-android-linux-uclibcgnueabi/bin:$PATH
so we can run cross-gcc by the command:
arm-android-linux-uclibcgnueabi-gcc
Root directory of target system is:
~/build/x-tools/arm-android-linux-uclibcgnueabi/arm-android-linux-uclibcgnueabi/sys-root
But for the safe - make new root dir and copy files too.
Copy files in ~/build/cross/sys-root
chmod +w sys-root
chmod +w sys-root/usr
5. Building busybox (only static build, but if we have static busybox in any problems with system problems with libs
we can easy use busybox as repair-mode tool)
copy file XVilka-busybox-<suffix>.config in .config
make menuconfig
make ARCH=arm CC="arm-android-linux-uclibcgnueabi-gcc" CROSS_COMPILE="arm-android-linux-uclibcgnueabi-" \
CFLAGS=" -static $_XXFLAGS"
make install CONFIG_PREFIX=~/build/cross/sys-root
Copy ~/build/cross/sys-root/bin/busybox in /system/bin on device
chown root /system/bin/busybox
chmod 755 /system/bin/busybox
ln -s /system/bin/busybox /system/bin/ash
Or in other directory and you can install all programs included in busybox by the command:
busybox --install
7. Building simple but powerful text editor jupp (mirbsd.org/jupp.htm)
chmod +x configure
./configure --prefix=/system/local --disable-curses --disable-termcap --disable-termidx --host=arm-android-linux-uclibcgnueabi ARCH=arm \
CC="arm-android-linux-uclibcgnueabi-gcc" CROSS_COMPILE="arm-android-linux-uclibcgnueabi-" sysconfdir=/system/usr/local/etc sysconfjoesubdir=/joe \
CFLAGS=" -static $_XXFLAGS"
make
arm-android-linux-uclibcgnueabi-strip joe
On device:
copy joe in /system/bin
copy joerc in /system/usr/local/etc/joe
copy jupprc in /system/usr/local/etc/joe
copy jmacsrc in /system/usr/local/etc/joe
copy sintax files in /system/usr/local/etc/joe/syntax
chmod 755 /system/bin/joe
ln -s /system/bin/joe /system/bin/jupp
ln -s /system/bin/joe /system/bin/jmacs
chmod 644 /system/usr/local/etc/joe/joerc
chmod 644 /system/usr/local/etc/joe/jupprc
chmod 644 /system/usr/local/etc/joe/jmacsrc
before running joe we must define TERM
export TERM=ansi
8. How to use dinamic linking and shared libraries
(if we installed programs in local directory in sys-root,
so copy lib from local)
cd ~/build/cross/sys-root
tar -jcf lib.tar.bz2 lib/
And on device:
cd /system/usr/local
busybox tar -xcf /sdcard/lib.tar.bz2
For our programs, wich uses this non-standart library path we may create this script and run your programs by it:
#!/bin/sh
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
exec /usr/bin/local/some_program $*
9. Building libFLAC in shared library, which we can use easy and copy to /system/lib,
which have nothing in NEEDED - just kernel and you, also no any libc (bionic, or uClibc) not used.
Download latest version of libFLAC from the official site, unpack and cd to dir:
./autogen.sh --host=arm --target=arm-android-linux-uclibcgnueabi CC="arm-android-linux-uclibcgnueabi-gcc" \
CXX="arm-android-linux-uclibcgnueabi-g++" STRIP="arm-android-linux-uclibcgnueabi-strip" \
AR="arm-android-linux-uclibcgnueabi-ar" AS="arm-android-linux-uclibcgnueabi-as" LD="arm-android-linux-uclibcgnueabi-ld" \
RANLIB="arm-android-linux-uclibcgnueabi-ranlib" OBJDUMP="arm-android-linux-uclibcgnueabi-objdump" \
NM="arm-android-linux-uclibcgnueabi-nm" CROSS_COMPILE="arm-android-linux-uclibcgnueabi-" \
CFLAGS=" -nodefaultlibs -nostdlib -fPIC $_XXFLAGS" \
CXXFLAGS=" -nodefaultlibs -nostdlib -fPIC $_XXFLAGS" \
LDFLAGS=" -fPIC -static -s -L$HOME/build/cross/sys-root/usr/lib -lm -lc" --prefix="$HOME/build/cross/sys-root/usr" \
--disable-ogg --disable-oggtest --disable-rpath --disable-xmms-plugin --disable-cpplibs --disable-altivec \
--disable-3dnow --disable-thorough-tests --disable-doxygen-docs
cd src/libFLAC
make
cd .libs
arm-android-linux-uclibcgnueabi-ld -shared -s -o libFLAC.so -whole-archive libFLAC.a
And now we have shared library libFLAC.so, lybrary without any dependency, and which we can just copy in /system/lib
on device and use it from any program.
For programming on libFLAC you must see header files of libFLAC
10. Building xvid in shared library, without any dependecy, same as libFLAC.
cd src/build/generic
./configure --host=arm --target=arm-android-linux-uclibcgnueabi CC="arm-android-linux-uclibcgnueabi-gcc" \
CXX="arm-android-linux-uclibcgnueabi-g++" STRIP="arm-android-linux-uclibcgnueabi-strip" \
AR="arm-android-linux-uclibcgnueabi-ar" AS="arm-android-linux-uclibcgnueabi-as" \
LD="arm-android-linux-uclibcgnueabi-ld" RANLIB="arm-android-linux-uclibcgnueabi-ranlib" \
OBJDUMP="arm-android-linux-uclibcgnueabi-objdump" NM="arm-android-linux-uclibcgnueabi-nm" \
CROSS_COMPILE="arm-android-linux-uclibcgnueabi-" \
CFLAGS=" -nodefaultlibs -nostdlib -fPIC $_XXFLAGS" \
CXXFLAGS=" -nodefaultlibs -nostdlib -fPIC $_XXFLAGS" \
LDFLAGS=" -fPIC -static -s -L$HOME/build/cross/sys-root/usr/lib -lm -lc" --prefix="$HOME/build/cross/sys-root/usr"
cd =build
rm-android-linux-uclibcgnueabi-ld -shared -s -o libxvidcore.so -whole-archive libxvidcore.a
So we have libxvidcore.so, which using very-very simple and copy-only-use too.
11. Building nmap
svn co --username guest --password "" svn://svn.insecure.org/nmap/
cd nmap
./configure --host=arm-android-linux-uclibcgnueabi CC="arm-android-linux-uclibcgnueabi-gcc" \
CROSS_COMPILE="arm-android-linux-uclibcgnueabi-" CFLAGS=" -static $_XXFLAGS" \
CXXFLAGS=" -static $_XXFLAGS" \
--prefix="$HOME/build/cross/sys-root/system/usr/local" --with-lua=included --with-pcap=included --with-pcre=included --with-dnet=included
make
make install
12. Building dropbear
./configure --host=arm-android-linux-uclibcgnueabi CC="arm-android-linux-uclibcgnueabi-gcc" \
CROSS_COMPILE="arm-android-linux-uclibcgnueabi-" \
CFLAGS=" -static $_XXFLAGS" \
--prefix="$HOME/build/cross/sys-root/system/usr/local" --disable-zlib
make
make install
copy dbclient, dropbearconvert and dropbearkey in /system/bin
copy dropbear in /system/sbin
chown root on all copied files
chmod 755 on all copied files

Hi, I've been folowing your howto in order to build busybox but when I download the my.zip file I think it's corrupted or something because it's empty and I can't extract the config files. Could you check it please? Thanks in advance

the my.zip file is compressed strangely. I was able to open it using 7-zip then opened the "my" file in there and it showed a bunch of file within it. Can the op look into his archive and see about recompressing it in a more compliant format. It isn't a zip file.
EDIT: Actually forget it. When you download it replace the .zip with .tar.bz2 or you can just append it to the filename probably if the .zip part isn't showing for you. It's a bzip2 compressed tar archive.

shinji257 said:
the my.zip file is compressed strangely. I was able to open it using 7-zip then opened the "my" file in there and it showed a bunch of file within it. Can the op look into his archive and see about recompressing it in a more compliant format. It isn't a zip file.
EDIT: Actually forget it. When you download it replace the .zip with .tar.bz2 or you can just append it to the filename probably if the .zip part isn't showing for you. It's a bzip2 compressed tar archive.
Click to expand...
Click to collapse
worked like a charm. Thanks!!

I would like to thank you for this guide. With it (and a small alteration to my own busybox .config) I was able to build 1.18.5 and 1.19.0.git with network support working finally. The files are a bit bigger for my taste but I'll go back and see if I can get a dynamic build next.
The flash* items (flashcp, flash_lock, flash_unlock, flash_eraseall) will not build unless you get the mtd-utils package from the android repo.
EDIT: It will dynamic build but I came across a roadblock. The uClibc included on my phone already is 0.9.31 and the toolchain built 0.9.30.2 so I'm changing it up a bit... Should be fine though.
Dynamic link reduces from 1.7MB to ~1.0MB

Here is russian version of manual http://wiki.androidfan.ru/Сборка_c_нуля
Also there some updates too: https://www.droid-developers.org/wiki/Toolchain

Easy way to build jupp… if you’ve got Debian, that is.
XVilka said:
7. Building simple but powerful text editor jupp (mirbsd.org/jupp.htm)
Click to expand...
Click to collapse
Thank you very much for liking jupp and publishing him here!
If you’re running Debian on an ARM system, there’s an even easier way to get jupp.
• set up the system for building packages (or login to a Debian porterbox like harris.debian.org and switch into the “sid” chroot): install the build-essential package
• install debhelper and autotools-dev
• install dietlibc-dev (or libklibc-dev, but I tested with dietlibc-dev right now as I cannot install libklibc-dev on the porterbox by myself)
• go to packages.debian.org/source/sid/jupp and figure out the download link to the .dsc file
• download that with “dget”:
Code:
dget -d http://ftp.de.debian.org/debian/pool/main/j/jupp/jupp_3.1.23-1.dsc
(of course, adjust the link; this is the version up to date at the time of this writing)
• extract the package (adjust the name as usual):
Code:
dpkg-source -x jupp_3.1.23-1.dsc
• switch into the source directory:
Code:
cd jupp-3.1.23/
• run a compilation:
Code:
env DEB_BUILD_OPTIONS=diet fakeroot debian/rules binary
(or DEB_BUILD_OPTIONS=klibc)
Now you have in the directories ./debian/jupp/ and ./debian/joe-jupp/ the full build result. Test it by running
Code:
./debian/jupp/usr/bin/jupp
(you can press ^Kq, that is Ctrl-K followed by q, to exit it immediately; remember ^J (Ctrl-J) opens the help).
Now install the following files into their final locations:
• debian/jupp/etc/jupp/* to /etc/jupp/ (including subdirectories)
• debian/jupp/usr/bin/jupp to /usr/bin/jupp
• optionally, debian/joe-jupp/etc/jupp/ to /etc/jupp/ and the symlinks from debian/joe-jupp/usr/bin/ to /usr/bin/
Strictly speaking, for running “jupp”, no other file than the executable is needed, and that can be in any path. It includes a copy of jupprc in the executable, although without syntax highlighting.
If you have to change the paths, look at the file debian/rules (--prefix, --sysconfdir, --mandir although you don’t need the mandir) and debian/jupp.install and debian/joe-jupp.install (install paths) before compiling.
JOE finds the *rc files by looking into a path that is set at compile time (if --sysconfdir=/etc it looks into /etc/joe/ but debian/rules also sets sysconfjoesubdir=/jupp at make time, so it uses /etc/jupp/ instead), a file that’s the basename of the editor command plus “rc”, so /usr/bin/jfoo will let it load jfoorc. The klingon charmap and the syntax files are expected in subdirectories of where the configs (rc files) are.
Hope that helps!
If you want, I’m in the IRC channel #MirBSD and #!/bin/mksh on Freenode IRC network, and can help (but will assume you’re either running Debian/ARM or have your cross compiler set up correctly).

[UTIL] signapk.sh: onboard .apk signing. 100% java-free

Since I tend to modify roms before flashing them to my phone, and sometimes want to do so away from a computer, I ended up writing this. It's an ash script that uses the openssl binary found on most custom roms, and infozip. The sdk test/platform/media/etc keys are included. Usage:
Usage: signapk (options) [command] (files)
commands:
sign FILE sign a .zip or .apk
sign FILE1 FILE2 create a signed copy of FILE1 as FILE2
cert FILE(s) print cert info on FILE's signer
certinfo FILE print detailed cert info on FILE's signer
cmp FILE compare signer of FILE with default/selected cert
cmp FILE1 FILE2 compare signer of FILE1 to signer of FILE2
options:
-k, --key FILE key to sign with
-c, --cert FILE cert to sign with
if -c or -k are not files then they are considered
aliases to builtins (ie -k testkey or -c platform)
-f, --force sign even if cert differs from original
-t, --tmp DIR use DIR for tempdir instead of '/cache'
-d, --debug output debugging
-V, --version print 'signapk v0.3.1'
exit codes:
1: read error (file 1) 2: read error (file 2)
3: write error 4: ssl error
5: zip write error 9: key error
8: sign: cert mismatch 10: cmp: cert mismatch
128: script error 255: user error
Installation:
unpack somewhere. If needed, edit the variables at the top of the script. By default it uses /cache/ for 4 small temporary files.
Faq:
* Why did you include a busybox and openssl? * Some roms don't include the openssl utility. Busybox's unzip refuses to deal with zipaligned apks on alot of roms, because it wasn't compiled with ENABLE_DESKTOP. Busybox's ash is missing things that the script needs, on some roms. The script will use whatever the rom provides, if these binaries are not present. Good luck.
* How do I change the cert? * see signapk --help for cert/key selection flags
* Can I use this in my MarketAppThingy? * Sure, and please let me know. If it's a paid MarketAppThingy, consider sending me a free copy.
Changelog:
release 3: cmp function, key selection, documented exit codes that make sense, and paranoia. Includes trimmed down busybox and openssl binaries. If these are present, script will reload itself to run in it's busybox ash interpreter to avoid broken busybox compiles, and can run without using anything provided by the rom. In theory this means no compatibility issues, but is limited to ARM devices.
release 2: Apparently it's valid to have ANYNAME.SF/ANYNAME.RSA or ANYNAME.DSA. signapk cert and certinfo now handles this. The sign function will blindly write to CERT.SF and CERT.RSA as this is the behavior of the standard SignApk.java. This may be fixed later.
If you don't wanna signup to download attachment, snag this from http://code.google.com/p/signapk/downloads/list

very, very useful. I'm testing this now Signing stuff was always way too complicated.

Cool. Can't wait for someone to make a market app that creates a simple ui to edit update.zips before flashing. Hint hint

I put the files in /system/signapk (new folder)
When i run the script I get the error below. The test file is an unsigned apk exported from eclipse.
Code:
# ./signapk sign /sdcard/unsigned.apk
./signapk sign /sdcard/unsigned.apk
Checksumming /mnt/sdcard/unsigned.apk:
res/layout/main.xml AndroidManifest.xml
resources.arsc res/drawable-hdpi/icon.png res/drawable-ldpi/icon.png res/drawable-mdpi/icon.png classes.dex
[COLOR="Red"]unable to write 'random state'
./signapk: line 132: ./signapk.zip: not found[/COLOR]
#
Any idea what needs changing to fix this?
update: When I used the full path it worked.
Code:
# /system/signapk/signapk sign /sdcard/unsigned.apk
/system/signapk/signapk sign /sdcard/unsigned.apk
Checksumming /mnt/sdcard/unsigned.apk:
res/layout/main.xml AndroidManifest.xml
resources.arsc res/drawable-hdpi/icon.png res/drawable-ldpi/icon.png res/drawable-mdpi/icon.png classes.dex
unable to write 'random state'
adding: META-INF/MANIFEST.MF (deflated 49%)
adding: META-INF/CERT.SF (deflated 48%)
adding: META-INF/CERT.RSA (deflated 33%)
# /system/signapk/signapk cert /sdcard/unsigned.apk
/system/signapk/signapk cert /sdcard/unsigned.apk
/sdcard/unsigned.apk 936EACBE07F201DF SDK Test Key

Was playing around with app inventor and came up with this:
edit: AppInventor does not generate java code and it cannot be worked on further in anything else (i.e eclipse)
I was really hoping I would atleast get the activity xml

Heya, the latest is at http://code.google.com/p/signapk/downloads/list and should fix the zip error. signapk-bb, signapk-zip, and signapk-openssl are static, trimmed down versions of utilities required by the script, and should be extracted to the same directory as the script. If they are not present it will try to use stuff in $PATH

Just throwing it (the obvious) out there that this could enable a new generation of morphs for /data apps
Copy apk,uninstall,morph, install

Hmmmm....

britoso said:
Just throwing it (the obvious) out there that this could enable a new generation of morphs for /data apps
Copy apk,uninstall,morph, install
Click to expand...
Click to collapse
How can i get this to work with a morph and to edit the updater-script?

Someone explain the commands with example...
Another noob question is .. it is for android or pc...
Edit:- Made my own mod here https://forum.xda-developers.com/t/...-apk-zip-within-android.3835975/post-81324327
below code works with latest(1.32.0) busybox
Bash:
#!/system/bin/sh
# depends unzip printf zip openssl
# put zip openssl busybox in PATH
# Usage:-
# Sign.sh key.pk8 key.x509.pem in.zip out.zip
ordie() {
if [ $? -ne 0 ]; then
IFS=" "
$BB printf "$1\n"
exit $2
fi
}
PKEY="$1"
CERT="$2"
ORIG="$3"
TARGET="$4"
BB=busybox
cat "$ORIG" > "$TARGET"; ordie "Cannot write to $TARGET" 3
$BB printf "Checksumming $ORIG:\n"
TMP=$HOME
TMPDIR="$TMP/sign-$$"
TMPPKEY="$TMPDIR/tmp.pkey"
mkdir -p "$TMPDIR/META-INF"; ordie "" 3
$BB printf "Manifest-Version: 1.0\r\nCreated-By: 1.0 (Android SignApk)\r\n\r\n" > "$TMPDIR/META-INF/MANIFEST.MF"
ZIPls=$($BB unzip -qql "$ORIG"); ordie "$ORIG: unzip error" 1
OLDIFS=$IFS
IFS=${IFS:2:2}
for i in $ZIPls; do
IFS=$OLDIFS
set -- $i
if [ "$#" -ge 3 ] && [ "$1" != "0" ]; then
file="${i:30}"
case "$file" in META-INF/MANIFEST.MF|META-INF/CERT.SF|META-INF/CERT.RSA) continue;; esac
$BB printf "$file "
hash=$($BB unzip -p "$ORIG" "$file" | openssl sha1 -binary | openssl base64); ordie "$ORIG: unzip error" 1
ret="Name: $file\r\nSHA1-Digest: $hash\r\n\r\n"
$BB printf "$ret" >> "$TMPDIR/META-INF/MANIFEST.MF"
hash=$($BB printf "$ret" | openssl sha1 -binary | openssl base64)
$BB printf "Name: $file\r\nSHA1-Digest: $hash\r\n\r\n" >> "$TMPDIR/META-INF/CERT.SF.temp"
fi
done
$BB printf "\n"
mfhash=$(cat "$TMPDIR/META-INF/MANIFEST.MF" | openssl sha1 -binary | openssl base64)
$BB printf "Signature-Version: 1.0\r\nCreated-By: 1.0 (Android SignApk)\r\nSHA1-Digest-Manifest: $mfhash\r\n\r\n" > "$TMPDIR/META-INF/CERT.SF"
cat "$TMPDIR/META-INF/CERT.SF.temp" >> "$TMPDIR/META-INF/CERT.SF"
openssl pkcs8 -inform DER -nocrypt -in "$PKEY" > "$TMPPKEY"; ordie "" 4
cat "$TMPDIR/META-INF/CERT.SF" | openssl smime -sign -inkey "$TMPPKEY" -signer "$CERT" -binary -outform DER -noattr > "$TMPDIR/META-INF/CERT.RSA"; ordie "" 4
cd "$TMPDIR"
zip "$TARGET" META-INF/MANIFEST.MF META-INF/CERT.SF META-INF/CERT.RSA; ordie "" 5
cd - > /dev/null
rm -r "$TMPDIR"

Guide To Build Sultanxda CM13

since i'm a flashaholic and a great fan of sultan's cm13, i'm trying my hands on android rom development and i thought the first step would be to learn how to compile a rom from source and the best choice was cm13 by sultan.
i'm using google's cloud platform for vm instance and using ubuntu 14.04.
i've used @akhilnarang script (link to his github:github.com/akhilnarang/scripts ) to setup the build environment and followed these commands to build the rom.
Code:
~$ repo init -u git://github.com/CyanogenMod/android.git -b stable/cm-13.0-ZNH2K
~$ mkdir .repo/local_manifests
~$ curl https://raw.githubusercontent.com/sultanxda/android/master/bacon/cm-13.0-stable/local_manifest.xml > .repo/local_manifests/local_manifest.xml
~$ repo sync -c -j10
~$ ./patcher/patcher.sh
~$ make clobber
~$ . build/envsetup.sh
~$ lunch cm_bacon-user
~$ time mka bacon -j8
but after 1h or so the compiler spilled these errors
Code:
Package OTA: /home/arjunmanoj1995/out/target/product/bacon/cm_bacon-ota-f59d4b6456.zip
Traceback (most recent call last):
File "./build/tools/releasetools/ota_from_target_files", line 1782, in <module>
main(sys.argv[1:])
File "./build/tools/releasetools/ota_from_target_files", line 1674, in main
], extra_option_handler=option_handler)
File "/home/arjunmanoj1995/build/tools/releasetools/common.py", line 826, in ParseOptions
if extra_option_handler is None or not extra_option_handler(o, a):
File "./build/tools/releasetools/ota_from_target_files", line 1644, in option_handler
from backports import lzma
ImportError: No module named backports
make: *** [/home/arjunmanoj1995/out/target/product/bacon/cm_bacon-ota-f59d4b6456.zip] Error 1
make: Leaving directory `/home/arjunmanoj1995'
#### make failed to build some targets (01:07:57 (hh:mm:ss)) ####
can someone plz help me resolve this error.

baconxda said:
since i'm a flashaholic and a great fan of sultan's cm13, i'm trying my hands on android rom development and i thought the first step would be to learn how to compile a rom from source and the best choice was cm13 by sultan.
i'm using google's cloud platform for vm instance and using ubuntu 14.04.
i've used @akhilnarang script (link to his github:github.com/akhilnarang/scripts ) to setup the build environment and followed these commands to build the rom.
Code:
~$ repo init -u git://github.com/CyanogenMod/android.git -b stable/cm-13.0-ZNH2K
~$ mkdir .repo/local_manifests
~$ curl https://raw.githubusercontent.com/sultanxda/android/master/bacon/cm-13.0-stable/local_manifest.xml > .repo/local_manifests/local_manifest.xml
~$ repo sync -c -j10
~$ ./patcher/patcher.sh
~$ make clobber
~$ . build/envsetup.sh
~$ lunch cm_bacon-user
~$ time mka bacon -j8
but after 1h or so the compiler spilled these errors
Code:
Package OTA: /home/arjunmanoj1995/out/target/product/bacon/cm_bacon-ota-f59d4b6456.zip
Traceback (most recent call last):
File "./build/tools/releasetools/ota_from_target_files", line 1782, in <module>
main(sys.argv[1:])
File "./build/tools/releasetools/ota_from_target_files", line 1674, in main
], extra_option_handler=option_handler)
File "/home/arjunmanoj1995/build/tools/releasetools/common.py", line 826, in ParseOptions
if extra_option_handler is None or not extra_option_handler(o, a):
File "./build/tools/releasetools/ota_from_target_files", line 1644, in option_handler
from backports import lzma
ImportError: No module named backports
make: *** [/home/arjunmanoj1995/out/target/product/bacon/cm_bacon-ota-f59d4b6456.zip] Error 1
make: Leaving directory `/home/arjunmanoj1995'
#### make failed to build some targets (01:07:57 (hh:mm:ss)) ####
can someone plz help me resolve this error.
Click to expand...
Click to collapse
It seems like your missing the backports module to build certain parts of the rom. As i looked at the prepare script you used I noticed an commented piece of code in there refering exectly to your error regarding lzma backports:
Code:
#echo Cloning LZMA repo
#git clone https://github.com/peterjc/backports.lzma /tmp/backports.lzma
#cd /tmp/backports.lzma
#sudo python2 setup.py install
#python2 test/test_lzma.py
#rm -rf /tmp/backports.lzma
#echo LZMA compression for ROMs enabled
#echo "WITH_LZMA_OTA=true" >> ~/.bashrc
Running this, or running the prepare script again with this uncommented might acctualy fix your import problem

gs-crash-24-7 said:
It seems like your missing the backports module to build certain parts of the rom. As i looked at the prepare script you used I noticed an commented piece of code in there refering exectly to your error regarding lzma backports:
Code:
#echo Cloning LZMA repo
#git clone https://github.com/peterjc/backports.lzma /tmp/backports.lzma
#cd /tmp/backports.lzma
#sudo python2 setup.py install
#python2 test/test_lzma.py
#rm -rf /tmp/backports.lzma
#echo LZMA compression for ROMs enabled
#echo "WITH_LZMA_OTA=true" >> ~/.bashrc
Running this, or running the prepare script again with this uncommented might acctualy fix your import problem
Click to expand...
Click to collapse
thanks mate, i'll give this a try and report back
edit: thanks .... it worked...:good:

I can't install the backports thing... I can't use the script to setup the build environment. How did you do it @baconxda? I uncommented those lines and it says that it doesn't have the lzma.h thing.

gs-crash-24-7 said:
It seems like your missing the backports module to build certain parts of the rom. As i looked at the prepare script you used I noticed an commented piece of code in there refering exectly to your error regarding lzma backports:
Code:
#echo Cloning LZMA repo
#git clone https://github.com/peterjc/backports.lzma /tmp/backports.lzma
#cd /tmp/backports.lzma
#sudo python2 setup.py install
#python2 test/test_lzma.py
#rm -rf /tmp/backports.lzma
#echo LZMA compression for ROMs enabled
#echo "WITH_LZMA_OTA=true" >> ~/.bashrc
Running this, or running the prepare script again with this uncommented might acctualy fix your import problem
Click to expand...
Click to collapse
Cesaragus said:
I can't install the backports thing... I can't use the script to setup the build environment. How did you do it @baconxda? I uncommented those lines and it says that it doesn't have the lzma.h thing.
Click to expand...
Click to collapse
just run the script then run those commands quoted above, thats it.

@baconxda Any idea if this ROM could be built for any other device?

Has.007 said:
@baconxda Any idea if this ROM could be built for any other device?
Click to expand...
Click to collapse
only for devices supported by @Sultanxda.

Error while building RedWolf recovery: cp does not take correct arguments

I am currently compiling a Red Wolf custom recovery for Xiaomi Mi A2 lite (daisy). Sources are synced, device tree etc. all set up correctly. Now I am facing a rather banal-looking error during build. cp refuses to copy a directory, because -r is not specified in the build file.
I obviously tried to find the build file (rw_recovery/out/build_omni_daisy.ninja) and add the -r argument, but it seems to be regenerated at the beginning of every build process, as the process still fails and the previously changed line appears unchanged when opening the file after the attempted build.
These are the build steps:
Code:
. build/envsetup.sh
lunch omni_daisy-eng #config for the device
mka bootimage #device uses boot.img as recovery
This command leaves me with the following error:
Code:
[ 99% 6883/6884] Prebuilt (rw_recovery/out/target/product/daisy/kernel)
FAILED: rw_recovery/out/target/product/daisy/kernel
/bin/bash -c "(rm -f /home/luca/rw_recovery/out/target/product/daisy/kernel) && (cp rw_recovery/out/target/product/daisy/obj/KERNEL_OBJ/arch/arm64/boot/ rw_recovery/out/target/product/daisy/kernel )"
cp: -r not specified; omitting directory 'rw_recovery/out/target/product/daisy/obj/KERNEL_OBJ/arch/arm64/boot/'
ninja: build stopped: subcommand failed.
17:13:06 ninja failed with: exit status 1
I would love to hear any suggestions about how to force ninja to execute the command with -r. Alternatively, suggestions about where to find the file from which the above mentioned build file is recreated after executing mka command are much welcomed as well.

Several .o files not found during build process while compiling a kernel for my SAMSUNG GALAXY J7 (SM-J700F)

I am trying to build a kernel for SM-J700F from the source at https://opensource.samsung.com/.
I got a toolchain from android-ndk that is with the help of following command:
Bash:
$NDK/build/tools/make_standalone_toolchain.py \
--arch arm64 --api 21 --install-dir /tmp/my-android-toolchain
and has put the toolchains where the Makefile searches for it.
Now at kernel source root directory I did the following:
Bash:
make ARCH=arm64 j7elte_00_defconfig
OUTPUT:
Bash:
[[email protected] kernel]$ make ARCH=arm64 j7elte_00_defconfig
HOSTCC scripts/basic/fixdep
HOSTCC scripts/kconfig/conf.o
HOSTCC scripts/kconfig/zconf.tab.o
In file included from scripts/kconfig/zconf.tab.c:2503:
scripts/kconfig/menu.c: In function ‘get_symbol_str’:
scripts/kconfig/menu.c:567:18: warning: ‘jump’ may be used uninitialized in this function [-Wmaybe-uninitialized]
567 | jump->offset = r->len - 1;
| ~~~~~~~~~~~~~^~~~~~~~~~~~
scripts/kconfig/menu.c:528:19: note: ‘jump’ was declared here
528 | struct jump_key *jump;
| ^~~~
HOSTLD scripts/kconfig/conf
#
# configuration written to .config
#
After that I did the following:
Bash:
make ARCH=arm64 -j64 -no-integreted-as
OUTPUT
Bash:
...
set -e; echo ' LINK vmlinux'; /bin/sh scripts/link-vmlinux.sh ../PLATFORM/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin/aarch64-linux-android-ld -EL -p --no-undefined -X --build-id; echo 'cmd_vmlinux := /bin/sh scripts/link-vmlinux.sh ../PLATFORM/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin/aarch64-linux-android-ld -EL -p --no-undefined -X --build-id' > ./.vmlinux.cmd
LINK vmlinux
LD vmlinux.o
aarch64-linux-android-ld: error: cannot open arch/arm64/kernel/head.o: No such file or directory
aarch64-linux-android-ld: error: cannot open init/built-in.o: No such file or directory
aarch64-linux-android-ld: error: cannot open usr/built-in.o: No such file or directory
aarch64-linux-android-ld: error: cannot open arch/arm64/kernel/built-in.o: No such file or directory
aarch64-linux-android-ld: error: cannot open arch/arm64/mm/built-in.o: No such file or directory
aarch64-linux-android-ld: error: cannot open arch/arm64/crypto/built-in.o: No such file or directory
aarch64-linux-android-ld: error: cannot open arch/arm64/mach-exynos/built-in.o: No such file or directory
aarch64-linux-android-ld: error: cannot open arch/arm64/plat-samsung/built-in.o: No such file or directory
aarch64-linux-android-ld: error: cannot open kernel/built-in.o: No such file or directory
aarch64-linux-android-ld: error: cannot open mm/built-in.o: No such file or directory
aarch64-linux-android-ld: error: cannot open fs/built-in.o: No such file or directory
aarch64-linux-android-ld: error: cannot open ipc/built-in.o: No such file or directory
aarch64-linux-android-ld: error: cannot open security/built-in.o: No such file or directory
aarch64-linux-android-ld: error: cannot open crypto/built-in.o: No such file or directory
aarch64-linux-android-ld: error: cannot open block/built-in.o: No such file or directory
aarch64-linux-android-ld: error: cannot open arch/arm64/lib/lib.a: No such file or directory
aarch64-linux-android-ld: error: cannot open lib/lib.a: No such file or directory
aarch64-linux-android-ld: error: cannot open arch/arm64/lib/built-in.o: No such file or directory
aarch64-linux-android-ld: error: cannot open lib/built-in.o: No such file or directory
aarch64-linux-android-ld: error: cannot open drivers/built-in.o: No such file or directory
aarch64-linux-android-ld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)
make: *** [Makefile:805: vmlinux] Error 1
I can see some of the source file like head.S but head.o is not there. I assume missing .o file should be build before it is required. I tried to figure out whats wrong but had no luck.
Please shed some light.
@Eliminator79
@Albe96
@LahKeda
@The flash
@MZO
@krasCGQ
@flar2
@jazba

Database Info

welcome