Related
This is a basic kernel build guide, to help those that keep coming into IRC asking for help.
If you follow this, you should have a working kernel that will work with ext4 or RFS.
NOTE::I wrote this at 1am, when Im kinda falling asleep, so there might be errors. Please let me know, and Ill fix it up
What you need:
A Linux system, with about 2gb free (This is written for 32bit systems, 64bit will need a compatible toolchain)
Development libraries
The kernel source
An ARM toolchain
An initramfs from an existing kernel
Downloads
EB13 source - search for D700 under mobile phone
Code Sourcery 2009q3 ARM toolchain
Daemon's Ext4/RFS Initramfs
Steps:
Set up the build environment
The first thing you need to do is set up the build environment.
For Ubuntu or other Debian-based system, you need the build-essential package and libncurses.
You can install this by running this command:
Code:
sudo apt-get install build-essential libncurses5-dev
If your not running Ubuntu/Debian, the packages are gcc, g++, libc-dev, and make.
Next, your going to need a directory for all your kernel build. You can't have spaces in any folder name. Im going to use /home/tortel/build/ for the rest of this guide.
Now you need to extract the kernel source to your directory. In the Samsung source file, its SPH-D700_kernel.tar
Next, extract the ARM toolchain into your build directory.
Last, extract your initramfs into the directory.
After this, your build directory should contain 4 things:
Code:
arm-2009q3/
eb13init/
Kernel/
build.sh
For this guide, Im not going to use the build.sh. If you want to, you'll need to configure that yourself.
Configure your kernel
Now the fun begins.
You need to edit the Kernel/Makefile file, and change line 184 to match where your toolchain is. My line 184:
Code:
CROSS_COMPILE ?= /home/tortel/build/arm2009q3/bin/arm-none-linux-gnueabi-
Now you need to configure the kernel.
The kernel configuration file is called .config, which is hidden by default (Ctrl+H shows hidden files in GNOME). You need to set lines 81-91, so it uses your initramfs. My lines:
Code:
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE="/home/tortel/kernel/eb13init/"
CONFIG_INITRAMFS_ROOT_UID=0
CONFIG_INITRAMFS_ROOT_GID=0
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
# CONFIG_INITRAMFS_COMPRESSION_NONE is not set
CONFIG_INITRAMFS_COMPRESSION_GZIP=y
# CONFIG_INITRAMFS_COMPRESSION_BZIP2 is not set
# CONFIG_INITRAMFS_COMPRESSION_LZMA is not set
This will build your initramfs into the kernel image.
Now, your set to build.
Build the kernel
This is the easy part. Open a terminal, cd to build/Kernel and run make.
The commands:
Code:
cd build/Kernel/
make
If you have more than one core, you should add -j {1+ # of CPU cores} to make so it finishes faster. It only takes about 5min on my 2ghz C2D laptop.
Make it flashable!
Congrats, its built!
The file you want is Kernel/arch/arm/boot/zImage in your kernel source directory. After make finishes, run:
Code:
cp arch/arm/boot/zImage ..
This copies your kernel image file into your main build directory. Now, you need to put it into a flashable zip. To make it easy, here. Just throw the zImage into the root of that zip, put it on your SD card, and flash it.
Might want later
Advanced config stuff to come
Might want this too
Thank you for this, I'm glad too see this and others who have contributed great guides like this.
sent from "The Other Woman"
I was one of those people who kept bothering lol. I learned how to compile a kernal in two day though, now im on my way making a OC/UV Voodoo color kernal (coming soon) :]
Thanks alot
globalninja said:
I was one of those people who kept bothering lol. I learned how to compile a kernal in two day though, now im on my way making a OC/UV Voodoo color kernal (coming soon) :]
Thanks alot
Click to expand...
Click to collapse
what should I look into for making the changes necessary for an overclocking kernel?
Thanks for this helpful thread, and I had a question previously, that someone pm'ed me for. OP, please remove this reply if you feel inclined to. I had a question about the initramfs because my computer was having problems loading the page for the link.
Hey guys, kindof a noob to this.
How can I get an initramfs for DI18? Ideally, I could just use the one in the stock ROM unmodified, but I have no idea how to obtain it.
I figured it out! Used the scripts here to extract it from zImage:
http://forum.xda-developers.com/showthread.php?t=901152
This thread aims to be a comprehensive guide to building and packaging kernels for US Variant Samsung Galaxy SIIIs
In my opinion, a kernel is a great way to get into building things for your device and its pretty easy to do too.
Intro
What is a kernel?
http://en.wikipedia.org/wiki/Kernel_(computing)
This guide is for US SGSIII's (d2att,d2cri,d2mtr,d2spr,d2tmo,d2usc,d2vzw,others?)
It may be possible to adapt this to other devices, but I am not responsible for anything that happens should you try to do this.
This guide assumes you have a general knowledge of the Linux operating system. If you've never used it, you might consider playing around
with it for awhile before attempting this guide.
Click to expand...
Click to collapse
Prerequisites
On all devices you must be rooted, on Verizon SGS3 (d2vzw) you must also have the unlocked (VRALE6) bootloader installed.
This is not the thread for figuring out how to do this. You can use the forum's search function to figure out how to do this on your device.
You'll need a computer or a virtual machine running ubuntu. You may be able to figure out how to get this working on other distributions,
but since ubuntu is generally the most accepted distribution to use for building android things, I'll stick to using that here.
At the time of this writing, I'm using ubuntu 12.10, 64-bit.
You'll need to install some packages on your ubuntu machine:
Code:
sudo apt-get install build-essential git zip unzip
On 64-bit you'll also need some multilib and 32-bit compatibility packages:
Code:
sudo apt-get install gcc-multilib g++-multilib lib32z1-dev
Click to expand...
Click to collapse
Setting up the Build Environment
Next, you'll need a toolchain which is used to actually build the kernel. You may download one of these:
GCC 4.4.3: Download || Mirror
GCC 4.6: Download || Mirror
GCC 4.7: Download || Mirror
If you aren't sure, go for 4.4.3 or 4.6.
4.7 requires some code changes to work. The original kernel developer may or may not have made these changes.
Here is what I needed to do in order for 4.7 to build, boot and have wifi work:
https://github.com/invisiblek/linux-msm-d2/commit/f8d7199d37cfbfa1bcb6b4bcae3fc15ae71fbdea
https://github.com/invisiblek/linux-msm-d2/commit/ea58076501e5874db7b934c215c4dae81ddfd0a6
The toolchains are also available in the android NDK.
*** There are many toolchains out there, some of you may know of the Linaro toolchain which is aimed to optimize your binary even further ***
*** If you choose to use a different toolchain, that is fine. Keep in mind that you may run into issues depending on the toolchain you use ***
You can check what your currently running kernel was built with by issuing these commands:
Code:
adb root
adb shell cat /proc/version
It should return something like:
Linux version 3.4.0-cyanogenmod-gc4f332c-00230-g93fb4aa-dirty ([email protected]) (gcc version 4.7 (GCC) ) #134 SMP PREEMPT Thu Feb 28 00:22:41 CST 2013
Click to expand...
Click to collapse
This shows my particular kernel here was built with GCC 4.7
You can use wget to download one of the links from above, in this instance we'll download version 4.4.3 from the first link:
Code:
wget http://invisiblek.org/arm-eabi-4.4.3.tar.bz2
Extract this to somewhere you will remember, probably your home directory.
Code:
mkdir arm-eabi-4.4.3
tar -xf arm-eabi-4.4.3.tar.bz2 -C arm-eabi-4.4.3/
Click to expand...
Click to collapse
Obtaining Source
Find someone's source to use as a base. This can be a source archive from Samsung, a kernel tree from CyanogenMod, or any other developer around that makes kernels for your device.
TIMEOUT
This is a good spot to stop and take note that the Linux kernel is licensed under the GNU General Public License (GPL): http://www.gnu.org/licenses/gpl-2.0.html
What does this mean you ask? It means that if you plan to share your kernel with the community (if it's good, please do so!) then you MUST share your
source code as well. I am not liable for what you choose to do once you start building kernels, but know this: if you share your kernel and do not
provide source code for it, you will get warnings from XDA for a determined amount of time, after that you may have your threads closed, deleted and
possibly your user account terminated. This is extremely important!
Also, you may run into more problems than just XDA. There are organizations out there that do take action if you consistently refuse to comply with the GPL.
I recommend you read this: http://www.gnu.org/licenses/gpl-2.0.html so that you are familiar with what legalities you are getting yourself into.
The main thing to remember is to share your source code if you decide to share your built kernel.
Click to expand...
Click to collapse
In this instance, we will use CyanogenMod's kernel source for the US Galaxy S3's. You may browse the source code here:
https://github.com/CyanogenMod/android_kernel_samsung_d2
You'll notice that the branch there is cm-10.1
This is the default branch of this repository on github. This means that if you intend to build this branch, you'll need to use it on CM version 10.1. Most
likely it will not function on another version.
To obtain the source code:
Code:
git clone https://github.com/CyanogenMod/android_kernel_samsung_d2
This will take a little while, be patient.
When done, you'll have a directory called android_kernel_samsung_d2, cd into this directory.
Code:
cd android_kernel_samsung_d2
Next, you'll need to set up a couple environment variables. These tell the system two things:
1. What CPU architecture to build for, in this case arm
2. Where to find the toolchain we downloaded earlier, so that the system can cross compile for arm
Code:
export ARCH=arm
export CROSS_COMPILE=~/arm-eabi-4.4.3/bin/arm-eabi-
You'll need to set these variables on each new session. You can modify your Makefile in the root of your kernel tree in order to have these set permanently.
Click to expand...
Click to collapse
Building
At this point you can make any changes to the source code that you want. If this is your first time, I recommend not making any changes and make sure you have a
sane build environment before adding any complications.
When you build a kernel, you need to choose a defconfig. This is a specialized configuration file, specifically tailored for your device.
CyanogenMod names their defconfigs for their devices like so: cyanogen_<device>_defconfig and they are located in arch/arm/configs/
Code:
ls arch/arm/configs/cyanogen*
In this example, we will build for d2vzw.
Set up your tree to build for the d2vzw:
Code:
make cyanogen_d2vzw_defconfig
(do this in your kernel's root directory, in this example it was android_kernel_samsung_d2/ )
Now you are ready to build:
First, determine how many cpu's your computer has. You'll use this number to determine how many jobs the compiler command will use. The more jobs you can use, the more
cpu threads the compile will take advantage of, thus you'll get faster builds. If you don't know, just assume you'll use the number 2. We'll use 2 as an example here.
Code:
make -j2
Where 2 is the number of CPU cores your build system has.
And now we wait...until it's done compiling...
You'll know it successfully compiled when you have this line when it stops:
Kernel: arch/arm/boot/zImage is ready
Click to expand...
Click to collapse
PROTIP:
If it stops somewhere other than "zImage is ready" then you had build errors. Try running the 'make' command with no options after it. This will run the compile on a single thread
and will cause it to stop compiling as soon as it hits an error. When you run it on multiple threads, it definitely goes much faster, but if an error occurs, the console doesn't stop
until it finishes all of its threads. Causing you to have to scroll up and search around for an error
Click to expand...
Click to collapse
Now, assuming the build completed successfully, you have two things you are concerned with: A zImage (the kernel binary itself) and your kernel modules, which get built based
on what was configured in your defconfig.
You'll find your zImage at: arch/arm/boot/zImage
Code:
ls arch/arm/boot/zImage
The modules are scattered all over the place, depending on where the source existed that they were compiled from. We can easily search for them using this command:
Code:
find . -name "*.ko"
If both of the previous commands completed, you are now ready to package your kernel up for testing.
Move up a directory before continuing.
Code:
cd ..
Click to expand...
Click to collapse
Packaging
You may know of an awesome developer by the name of koush.
Well, once upon a time, koush created a rather simple zip, called AnyKernel, that would flash a kernel on a device, regardless of what ramdisk the kernel has on it.
I've taken his zip and modified it for d2 devices and to work with the newer recoveries out there.
This has a script in it that will dump your current boot.img (kernel+ramdisk), unpack it, replace the kernel, repack it and flash it.
It'll also copy any modules to the proper directory (/system/lib/modules) and set permissions appropriately.
You can get a zip here: Download || Mirror
(You can get it here as well: https://github.com/invisiblek/AnyKernel )
(Everyone is invited to use this zip, it'll probably make your life easier to not have to worry about the ramdisk. Enjoy!)
IMPORTANT
This AnyKernel package is for US variations of the Galaxy S3.
NOT the international (I9300) or any other device.
There are checks in the updater-script that will ensure you are running a d2 device before it does anything.
If you were to remove these checks, and not modify the partition that it flashes to later, you could end up with a brick.
If you intend to adapt this package for another device (please, do this! its a very handy script!), make sure you know it well, or ask someone to help you determine your device's
partition scheme before using it.
The risk here is due to the fact that the script doesn't know your device's partition scheme. It is configured specifically for the d2 devices. Flashing it on something else, who's boot
partition is somewhere else, might cause a bad flash to the bootloader partition (bad bad news if this happens).
Just be careful if you want to use this on another device. You won't run into problems if you use this on a d2 device.
EDIT: I made modifications that should make this less likely, but please, if you intend to use this on a different device (which is completely fine!) make sure you configure
the scripts to flash to the proper partitions.
Click to expand...
Click to collapse
Download and extract one of the above, we'll again use the first link for this example:
Code:
wget http://invisiblek.org/AnyKernel_samsung-d2.zip
unzip AnyKernel_samsung-d2.zip -d AnyKernel/
Now we'll copy our newly compiled zImage (still referring to the same kernel directory we used above, your repo might be called something different)
Code:
cp android_kernel_samsung_d2/arch/arm/boot/zImage AnyKernel/kernel/
cp `find android_kernel_samsung_d2 -name "*.ko"` AnyKernel/modules/
Finally we are ready to zip this up and test out flashing it.
Code:
cd AnyKernel
zip ../MyAwesomeKernel.zip -r *
cd ..
You'll now have a file named MyAwesomeKernel.zip which you should be able to flash via custom recovery (TWRP or CWM)
Click to expand...
Click to collapse
Extra Credit/Protips
Learn to use git. It's very powerful and great way to store your code.
Learn to use adb. It's an invaluable tool for any android developer.
Touchwiz and AOSP-based kernels are different. This means you cannot take CyanogenMod's source, build a kernel and expect it to work on a Touchwiz-based ROM.
Build a ROM next: http://wiki.cyanogenmod.org/w/Build_for_d2vzw
Crackflash your own stuff!
ALWAYS NANDROID!
Click to expand...
Click to collapse
Source code for all of my projects can be found here: http://github.com/invisiblek
FAQ
Q: How do I update my source tree to the latest that is available from where I downloaded it?
A: This can be handy if, for instance, you are building a CyanogenMod kernel and they added some patches, after you downloaded the source, that you want to include in your next build. You'll want to cd to your kernel tree and issue a git pull:
Code:
cd android_kernel_samsung_d2
git pull
You may then continue with the building instructions.
This may, however, have other problems if you've made changes to files. You might run into conflicts. I won't cover fixing any of this here, its not in the scope of this thread.
Q: I'm using X as a kernel base, but Y has a patch that I really like. How do I get it in my kernel easily?
A: I'll let you check Google for this answer, but I'll give you a hint use: git cherry-pick
Nice tutorial bro! Always good to learn something new everyday
Really is a good thread，thanks
This guide would have made things too easy for me.
Too easy, indeed. haha
Great job, invisiblek! AnyKernel is the beez neez.
Ok so this is a noob question but I gotta ask anyway lol. Ok so I cloned the kernel source, I made my edits, now how do I push all this to my github?
I already have a github account, I already made a new repo for the kernel. Here's a link to my github if you need it...
https://github.com/ghicks12/d2vzw_kernel.git
spc_hicks09 said:
Ok so this is a noob question but I gotta ask anyway lol. Ok so I cloned the kernel source, I made my edits, now how do I push all this to my github?
I already have a github account, I already made a new repo for the kernel. Here's a link to my github if you need it...
https://github.com/ghicks12/d2vzw_kernel.git
Click to expand...
Click to collapse
git remote add origin git_location_you_created_on_github.git
git push -u origin somebranch
The -u is for first time run only, you can just git push afterwards.
Sent from my SCH-I535
GideonX said:
git remote add origin git_location_you_created_on_github.git
git push -u origin somebranch
The -u is for first time run only, you can just git push afterwards.
Sent from my SCH-I535
Click to expand...
Click to collapse
Thanks! When I run
Code:
git remote add origin https://github.com/ghicks12/d2vzw_kernel.git
I get this back:
Code:
fatal: remote origin already exists.
I'm editing a CM based kernel, not sure if that matters or not?
That just means you added the remote already. Just issue the push command then.
Sent from my SCH-I535
Why is this happening? I don't know what i did wrong
[email protected]:~/cm$ make VARIANT_DEFCONFIG=cyanogen_d2att_defconfig
scripts/kconfig/conf --silentoldconfig Kconfig
drivers/media/video/msm/Kconfig:123:warning: choice value used outside its choice group
drivers/media/video/msm/Kconfig:128:warning: choice value used outside its choice group
***
*** Configuration file ".config" not found!
***
*** Please run some configurator (e.g. "make oldconfig" or
*** "make menuconfig" or "make xconfig").
***
make[2]: *** [silentoldconfig] Error 1
make[1]: *** [silentoldconfig] Error 2
make: *** No rule to make target `include/config/auto.conf', needed by `include/config/kernel.release'. Stop.
[email protected]:~/cm$
Hey. I'm having some problems with some GIT terminology and procedures. I'm a .NET developer and I use TFS and SVN on a daily basis. Forgive me if this is complete off basis from what you'd do with GIT.
What I want to do is merge one branch into another branch. In other words I want to take the latest kernel source from my favorite dev and merge in the latest from cyanogen's 4.3 d2 branch. Is this a rebase thing? It doesn't seem like cherrypicking to me.
I have successfully compiled kernel and made modules.I inserted zImage and modules inside any kernel updater,flashed via TWRP.When reboot stuck in odin and it says could not do normal boot.
Hi Folks
I wasn't sure where this should belong but as it is a bit of an Hack this forum is probably the most appropriate
Introduction
This short tutorial will show you how to patch the Android Build System to allow you to cross-compile Android AOSP host tools ( adb, fastboot etc ) for OSX using a linux based machine. This is something Google said was impossible or at the very least unsupported.
Assumptions
You have a linux based machine and working copy of the AOSP source tree.
You can/have successfully compile(d) a full Android Release from this tree.
A basic idea of how the Android Build System works is beneficial.
Getting Started
I've set-up a git repository which contains a binary copy of the OSX SDK 10.6 and the apple-darwin10-gcc cross compiler. So first things first. open a terminal and set the root of the AOSP sources tree to the current directory.
STAGE 1: Copy the OSX SDK
Step 1.
Clone the repo with the SDK and toolchain
Code:
git clone https://github.com/trevd/android_platform_build2.git build2
Step 2.
Create /Developer directory at your filesystem root, this is a known location for the SDKs
Code:
sudo mkdir /Developer
sudo chown $USER.$USER /Developer
Step 3.
Copy and unpack the SDK package
Code:
cp build2/osxsdks10.6.tar.gz /Developer
cd /Developer
tar -zxvf osxsdks10.6.tar.gz
rm osxsdks10.6.tar.gz
cd - # back to aosp root
STAGE 2 : Swapping the Toolchain
This is where the fun begins :laugh:
The Android Build system has the majority of the infrastructure in place already to build for OSX, the only problem is that you need OSX to build for OSX. However we can remedy that with a couple of dirty hacks :laugh:.
The prebuilts/gcc/darwin-x86 directory contains a toolchain compatible with osx ( mach-o binaries ). We are going to swap this for a linux compatible ( elf ) executables.
Step 4:
Copy and unpack the elf compatible darwin cross toolchain
Code:
cp build2/i686-apple-darwin-4.2.1.tar.gz prebuilts/gcc/linux-x86/host
cd prebuilts/gcc/linux-x86/host
tar -zxvf i686-apple-darwin-4.2.1.tar.gz
cd - # back to aosp root
Step 5:
Remove the mach-o binaries and symlink the elf binaries in it's place
Code:
cd prebuilts/gcc
rm -rf darwin-x86
ln -s linux-x86 darwin-x86
cd - # back to aosp root
Step 6:
We also need to replace the mach-o version of the ccache executable which live in the prebuilt/misc directory
Code:
cd prebuilts/misc
rm -rf darwin-x86
ln -s linux-x86 darwin-x86
cd - # back to aosp root
STAGE 3: Patching the build system .mk files
We need to patch a couple of files in the build directory namely the build/core/combo/HOST_darwin-x86.mk the main crux of this is swapping the ar tool for libtool so static libraries can be created without error.
Code:
patch -p1 < build2/build.patch
If the patch has been applied successfully you should see the following
Code:
patching file system/core/adb/Android.mk
patching file build/core/combo/HOST_darwin-x86.mk
patching file build/core/combo/select.mk
patching file build/core/envsetup.mk
patching file build/envsetup.sh
You are now ready to cross compile!! :good: ..... well not quite, but nearly.... here's why!
The Android Build System will attempt to build both the Target and Host files for most modules so I'd advise using a lunch option which already has a full target built for it or alternatively you can build the generic sdk using the following commands at the AOSP source tree root.
Code:
. build/envsetup.sh
lunch sdk-eng
make sdk
This will stop target dependency errors occurring when you build individual modules.
NOW we're ready to cross compile.
STAGE 4: Building Modules
At present module build is very much a piecemeal process. To build adb for example we need to build the dependencies first. This is not too onerous as most host modules have very few dependencies.
Building adb
adb has dependencies on the following libraries
Code:
external/zlib
external/openssl
system/core/liblog
system/core/libcutils
system/core/libzipfile
I've found the easiest way to compile the dependencies is to navigate to each directory in turn an use to "mm" build system command to compile the individual module. the commands I run to compile adb are as follows.
From AOSP Source Root
Code:
cd external/zlib
USE_DARWIN=true mm -j8
cd ../openssl
USE_DARWIN=true mm -j8
croot # go back to the AOSP root
cd system/core/liblog
USE_DARWIN=true mm -j8
cd ../libcutils
USE_DARWIN=true mm -j8
cd ../libzipfile/
USE_DARWIN=true mm -j8
cd ../adb
USE_DARWIN=true mm -j8
All being well you should now have and adb binary located at out/host/darwin-x86/bin/adb. running the file command on this binary should produce the following output
Code:
adb: Mach-O executable i386
Conclusion
Although this method is a little rough and ready, it should produce the desired results if you need to cross compile for OSX. The eventual goal would be to compile a full OSX Android SDK on linux in a similar manner to the way the windows-sdk is currently compiled. This requires more investigation as compiling the windows sdk on linux employs a little bit of trickery on the part of the build system.
Final Notes and FAQs:
Why can't I just type make <module> from the root?
Doing this triggers building of additional modules such as LLVM and clang which are to deployed out/host/darwin-x86/bin the build system then attempts to use binary later on. These are obviously built for the Mach-o architecture and as such are incompatible with the linux. This results in a build error which can and would be resolved by the above mentioned trickery ( see conclusion )
I use OSX binaries (along with Windows and my native Linux) in one of my projects. Thanks, I have always relied on finding compiled binaries elsewhere. Lack of an OSX aapt held up an update at one point.
One of those things that you don't really use until you need it, but I will try to remember to give it a shot. I don't have any doubt that it works.
mateorod said:
I use OSX binaries (along with Windows and my native Linux) in one of my projects. Thanks, I have always relied on finding compiled binaries elsewhere. Lack of an OSX aapt held up an update at one point.
One of those things that you don't really use until you need it, but I will try to remember to give it a shot. I don't have any doubt that it works.
Click to expand...
Click to collapse
Thanks. Yes this really is an edge case. Hopefully It will help some folks out.
Regarding aapt in particular.... It's perfectly possible to build aapt, however, we need to do some slight of hand with the clang and clang++ executables as libpng on which aapt depends uses these 2 binaries as part of it's build process.
Here's the build list and the clang trick if you want to try it some time.
Code:
build/libs/host
external/expat
external/zlib
system/core/liblog
system/core/libcutils
mkdir out/host/darwin-x86/bin
cp out/host/linux-x86/bin/clang out/host/darwin-x86/bin
cp out/host/linux-x86/bin/clang++ out/host/darwin-x86/bin
external/libpng
frameworks/base/libs/androidfw
frameworks/native/libs/utils
frameworks/base/tools/aapt
I started off with a clean out/host/darwin-x86 directory so I didn't miss any dependencies.
like I mentioned the clang "swap out" is something the make win_sdk option does automatically so with it a little more research I should be able to get the mac build to do the same but you'll have to "fill yer boots" with the ghetto method for now
For reference here's a link to the sdk building instructions http://tools.android.com/build which describes how to cross compile the windows sdk on linux ( in case anyone was wondering what the hell i'm on about)
My use case has come up
I will be cross-compiling for OSX today...specifically with aapt in mind. I will report back, but I fully expect it to work as described.
mateorod said:
My use case has come up
I will be cross-compiling for OSX today...specifically with aapt in mind. I will report back, but I fully expect it to work as described.
Click to expand...
Click to collapse
Cheers Man!
Hopefully no bitrot has crept in since april and now. I know I've changed my OS version since to Lubuntu 13.04, not like the OS version really matters any.
mateorod said:
but I fully expect it to work as described.
Click to expand...
Click to collapse
Then you Sir, are either Drunk or a Fool! LOL Keep expectations Quantum and only decided when the result is observed a'la Schrodinger Cat
okay...So I was trying to compile SlimRom (so as to get an OSX aapt binary with the SlimRom changes) and things did not necessarily go as planned. There were enough changes to the SlimRom android_build that your build/build.patch does not apply cleanly. I spent some time and tried to modify the patch so that it would work for both SlimRom, AOSP and probably others, but each android_build repo has some differences in surrounding the HOST_AR line, so commenting that just was not portable between flavors. Not cool.
Anyway, turns out that this method does not quite work out of the box for non-AOSP versions (not that you claimed that it did). I got some unfamiliar errors related to (I believe) some OSX toolchains. But in both times I tried this, I actually had to pretty immediately swap out of that flavor and so I was unable to do much debugging. (I keep all the flavors I build {CM, AOKP, SlimRom, PAC, PA, OpenPDroid, etc, etc, etc} all layered in one android/system/jellybean directory. It saves a ton of space, but only allows me to do one thing at a time.)
So the only feedback I have is nothing...I even formatted my hard drive in-between and forgot to put up a paste, so the errors are currently lost to history.
Things that I noticed, for better or worse
You recommend putting the SDKs in the root dir. I believe the documentation is recommending the Developer be placed in home (as per the SDK/ADT bundle docs).
You might want a
Code:
mv android_platform_build2 build2
line. I normally wouldn't bother, but it looks like you are trying to post a line-by-line guide.
I would put the recommendation that a full build be available to the out folder (or a built generic sdk) right at the top, since it is a preliminary step. I had to revert my handwritten changes, then build, then reapply the changes and rebuild since I thought it needed a clean out dir.
Did you have any trouble with git reverting the toolchain swap? On two separate machines, I had to go so far as to delete .repo/projects/prebuilts/gcc/* and prebuilts/gcc/darwin-x86/arm/arm-eabi-4.6. It kept complaining of that the project in the .repo folder was a bad match. No amount of git trickery (which I am not terrible at) let me back out more easily.
I am willing to try again...but I have some other small things to attend to first. It is an admirable hack you have here sir. I will return to it soon and report back once more.
mateorod said:
okay...So I was trying to compile SlimRom (so as to get an OSX aapt binary with the SlimRom changes) and things did not necessarily go as planned. There were enough changes to the SlimRom android_build that your build/build.patch does not apply cleanly. I spent some time and tried to modify the patch so that it would work for both SlimRom, AOSP and probably others, but each android_build repo has some differences in surrounding the HOST_AR line, so commenting that just was not portable between flavors. Not cool.
Anyway, turns out that this method does not quite work out of the box for non-AOSP versions (not that you claimed that it did). I got some unfamiliar errors related to (I believe) some OSX toolchains. But in both times I tried this, I actually had to pretty immediately swap out of that flavor and so I was unable to do much debugging. (I keep all the flavors I build {CM, AOKP, SlimRom, PAC, PA, OpenPDroid, etc, etc, etc} all layered in one android/system/jellybean directory. It saves a ton of space, but only allows me to do one thing at a time.)
So the only feedback I have is nothing...I even formatted my hard drive in-between and forgot to put up a paste, so the errors are currently lost to history.
Things that I noticed, for better or worse
You recommend putting the SDKs in the root dir. I believe the documentation is recommending the Developer be placed in home (as per the SDK/ADT bundle docs).
You might want a
Code:
mv android_platform_build2 build2
line. I normally wouldn't bother, but it looks like you are trying to post a line-by-line guide.
I would put the recommendation that a full build be available to the out folder (or a built generic sdk) right at the top, since it is a preliminary step. I had to revert my handwritten changes, then build, then reapply the changes and rebuild since I thought it needed a clean out dir.
Did you have any trouble with git reverting the toolchain swap? On two separate machines, I had to go so far as to delete .repo/projects/prebuilts/gcc/* and prebuilts/gcc/darwin-x86/arm/arm-eabi-4.6. It kept complaining of that the project in the .repo folder was a bad match. No amount of git trickery (which I am not terrible at) let me back out more easily.
I am willing to try again...but I have some other small things to attend to first. It is an admirable hack you have here sir. I will return to it soon and report back once more.
Click to expand...
Click to collapse
Hi
Thanks for this, It sounds like you've suffered an exercise in frustration there. I wasn't aware that "SlimRom" had a different aapt ( just out of general ignorance and not having paid any attention )
SDK - My Tree last time I used this was /Developer directory in the root - I think It comes from what the toolchain is expecting, I just gave it what it wants
mv android_platform_build2 build2 - Yep I did mean that it's the git clone line which wants changing
Code:
git clone https://github.com/trevd/android_platform_build2.git build2
SDK Recommendation - I shall move that to the top, even though it is already in there, It should probably be highlighted better and possible it's own "Stage"
Reverting the toolchain - Ahh , It appears I work slightly different from most in this respect. I have a general mistrust of SCM's ( I lost too much code on too many different SCM's, Probably through my own inability to use them correctly but ) what I do to revert to change is
Code:
cd prebuilts/gcc/darwin-x86/host/
rm -rf i686-apple-darwin-4.2.1
repo sync i686-apple-darwin-4.2.1
You can do this "trick" on any project in the source tree it's only on rare occasions where I screwed up badly that I have to delete anything in .repo/projects but I also have my distro in their own individual directories with there own full git trees, which is a massive waste of space and has a ton of redundancy due to the AOSP repositories being mirror by every single one but switching between them is a lot easier
If SlimRom's changes are localized to aapt, I'd be more inclined to drop it into the AOSP build and try that... If you have a link to slimrom's frameworks/base repository I'll grab it and try it myself.
On a final note there's a "full version" of the HOST_darwin make file in the build2/core/combo directory the changes to envsetup.mk and select.mk are minimal and can easily be applied manually. You don't need to patch the adb makefile if your not building it.
Again Thanks for the feedback
Hi all
I thought I'd try and write an easy and simple guide on how I extract blobs from vendor updates, cause I couldn't find a guide online myself.
A few things first:
- I will be writing this guide on Windows. You will need to find the Linux/MacOS tools yourself cause this is what I use.
- I use Windows Subsystem for Linux in this guide. It's not needed, but will make your life a lot easier. Follow this guide if you haven't got it installed already.
- This will assume you know how to build a ROM, know what device/vendor trees are, have some slight knowledge of the command line and know how to use basic Git.
First, we need to grab an OTA. I own a MI 8 Lite, so I usually use xiaomi.eu to update my blobs as it's easy to capture the zips. Depends on your device.
Now you've captured your OTA, you need to get the blobs extracted out. I use UR-Tool on Windows 10 but it's completely your choice. I will be proceeding this tutorial with UR-Tool.
Extract your zip into the UR-Tool build environment. Run the tool and choose your zip, extract it, like here: https://i.imgur.com/Y50Y3IZ.png
Okay, so now we have two directories, system and vendor (I'm fairly sure if you're non-treble you'll have one folder which means you can skip this next step.)
Go into system, and remove the symlink named vendor. You can then copy in the vendor folder into system.
In my device tree, I also had /sbin/charge needed in my proprietary-files.txt. I needed to extract my boot.img and put this folder into the system folder. Just check your device tree's needed files.
Copy the entire folder to somewhere easy to find and preferably a path which doesn't contain any spaces. We will be using it later.
Okay, so now we've done all this, we are going to move to the vendor and Git side of things. I will be using WSL from here!
So, now we have our fully ready to go blobs... but what now?
Well we need to prepare our vendor of course!
Like I said above I'll be using WSL from here as I needed "patchelf" for mlipay blobs. You will need something that can execute .sh files and using WSL is probably the easier option. If you're on Linux you don't need to do anything really.
We need 3 things to clone in. I've taken the liberty of stripping down Lineage's vendor to just the critical stuff for extracting blobs. You can find my repository here.
Lets make a new directory. We will clone in our trees like this:
Code:
git clone (device tree link) device/(vendor)/(codename)
git clone (vendor tree link) vendor/(vendor)/(codename)
git clone https://github.com/nysadev/extract_blobs_vendor vendor/lineage
This will create a directory that looks like a ROM, but severely stripped down. The reason we do the folders like this is because the scripts in device tree will point to a structure like this. This will help us avoid script location errors later. Make sure in your device tree that extract-files.sh points to vendor/lineage (should be in the first few lines).
Your root directory now should look like this: https://i.imgur.com/YiqMaDJ.png
Now go to your device tree. This is where all the good stuff happens. You need to know the path here of your extracted blobs from earlier.
Run these two commands:
Code:
chmod +x setup-makefiles.sh
./extract-files.sh (path/to/your/extracted/system)
Hopefully if all goes well you should see this with no "file missing in source errors": https://i.imgur.com/jErpyCv.png
Just to be sure, at the end of when it extracts, run "./extract-files.sh" again, just to prepare your vendor.
Now you're done! You'll hopefully see a few good modified libs, changes to Android.mk and your device-vendor.mk.
You're probably going to want to use git here to upload your changes. Just run:
Code:
git add .
git commit -m "(codename): update to (ota-version)"
git push -f
or something of the sort.
Well done! You're now done! Feel free to reply with any errors you get.
My Telegram is @nysadev; you're maybe better off catching me there.
I hope my guide helped you. Drop a like or something, idek.
It will be very useful
You rock! Love ya
how do u get proprietary-files.txt if u r building from scratch?
wheres UR-tool?
I have a rooted device, with enought power to run my server, it would be nice if possible.
AFAIK the docker engine is not currently supported on any version of Android.
BTW: Android apps are already isolated. Each app you install creates a new user and that user has only permissions within the folder the app was installed and permissions you explicitly give it.
Yes, running docker is absolutely possible on Android, doing the same on my device. The only thing you need is a custom kernel with all required docker features activated - so be prepared to compile a kernel. In addition, Termux offers a docker installation package which can be simply fetched and run (dockerd) afterwards.
Here you go with a step by step guide (not written by me): https://gist.github.com/FreddieOliveira/efe850df7ff3951cb62d74bd770dce27
These are the features which were sufficient for me:
CONFIG_NAMESPACES=y
CONFIG_NET_NS=y
CONFIG_PID_NS=y
CONFIG_IPC_NS=y
CONFIG_UTS_NS=y
CONFIG_CGROUPS=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_SCHED=y
CONFIG_CPUSETS=y
CONFIG_MEMCG=y
CONFIG_KEYS=y
CONFIG_VETH=y
CONFIG_BRIDGE=y
CONFIG_BRIDGE_NETFILTER=y
CONFIG_IP_NF_FILTER=y
CONFIG_IP_NF_TARGET_MASQUERADE=y
CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=y
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=y
CONFIG_NETFILTER_XT_MATCH_IPVS=y
CONFIG_NETFILTER_XT_MARK=y
CONFIG_IP_NF_NAT=y
CONFIG_NF_NAT=y
CONFIG_POSIX_MQUEUE=y
CONFIG_DEVPTS_MULTIPLE_INSTANCES=y
CONFIG_NF_NAT_IPV4=y
CONFIG_NF_NAT_NEEDED=y
CONFIG_OVERLAY_FS=y
Furthermore you need to apply a small and easy patch to "net/netfilter/xt_qtaguid.c", otherwise the kernel will panic now and then. But its really easy to apply, take a look at the guide. Modification of Makefile "kernel/Makefile" is not necessary.
mtdshare said:
Yes, running docker is absolutely possible on Android, doing the same on my device. The only thing you need is a custom kernel with all required docker features activated - so be prepared to compile a kernel. In addition, Termux offers a docker installation package which can be simply fetched and run (dockerd) afterwards.
Here you go with a step by step guide (not written by me): https://gist.github.com/FreddieOliveira/efe850df7ff3951cb62d74bd770dce27
These are the features which were sufficient for me:
CONFIG_NAMESPACES=y
CONFIG_NET_NS=y
CONFIG_PID_NS=y
CONFIG_IPC_NS=y
CONFIG_UTS_NS=y
CONFIG_CGROUPS=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_SCHED=y
CONFIG_CPUSETS=y
CONFIG_MEMCG=y
CONFIG_KEYS=y
CONFIG_VETH=y
CONFIG_BRIDGE=y
CONFIG_BRIDGE_NETFILTER=y
CONFIG_IP_NF_FILTER=y
CONFIG_IP_NF_TARGET_MASQUERADE=y
CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=y
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=y
CONFIG_NETFILTER_XT_MATCH_IPVS=y
CONFIG_NETFILTER_XT_MARK=y
CONFIG_IP_NF_NAT=y
CONFIG_NF_NAT=y
CONFIG_POSIX_MQUEUE=y
CONFIG_DEVPTS_MULTIPLE_INSTANCES=y
CONFIG_NF_NAT_IPV4=y
CONFIG_NF_NAT_NEEDED=y
CONFIG_OVERLAY_FS=y
Furthermore you need to apply a small and easy patch to "net/netfilter/xt_qtaguid.c", otherwise the kernel will panic now and then. But its really easy to apply, take a look at the guide. Modification of Makefile "kernel/Makefile" is not necessary.
Click to expand...
Click to collapse
I have a couple of questions if you don't mind me asking..
1- Would this basically let me use linux images from linuxserver.io to run servers on my phone ?
2- Do I really need to compile a kernel or can I just flash any kernel that is available for my device ?
savvyyy said:
I have a couple of questions if you don't mind me asking..
1- Would this basically let me use linux images from linuxserver.io to run servers on my phone ?
2- Do I really need to compile a kernel or can I just flash any kernel that is available for my device ?
Click to expand...
Click to collapse
1. Only if these are available in the right processor architecture. Most of the CPUs used in mobile devices are arm/arm64.
2. You will need to compile one with the mentioned options and patches yourself, since most of the available kernels don't include this functionality.
mtdshare said:
1. Only if these are available in the right processor architecture. Most of the CPUs used in mobile devices are arm/arm64.
2. You will need to compile one with the mentioned options and patches yourself, since most of the available kernels don't include this functionality.
Click to expand...
Click to collapse
Thanks for taking the time.
1- checks out, there seems to be both arm and arm64 compatible dockers.
2- What if the docker is to be used inside a chroot linux distro on android ? Would the android kernel still have to be patched ?
Thanks again.
savvyyy said:
Thanks for taking the time.
1- checks out, there seems to be both arm and arm64 compatible dockers.
2- What if the docker is to be used inside a chroot linux distro on android ? Would the android kernel still have to be patched ?
Thanks again.
Click to expand...
Click to collapse
2 - Following logic, I guess it can't because chroot is using android kernel to run the linux distro, therefore the kernel will still be limited as it is outside the chroot environment.
mtdshare said:
Yes, running docker is absolutely possible on Android, doing the same on my device. The only thing you need is a custom kernel with all required docker features activated - so be prepared to compile a kernel. In addition, Termux offers a docker installation package which can be simply fetched and run (dockerd) afterwards.
Here you go with a step by step guide (not written by me): https://gist.github.com/FreddieOliveira/efe850df7ff3951cb62d74bd770dce27
These are the features which were sufficient for me:
CONFIG_NAMESPACES=y
CONFIG_NET_NS=y
CONFIG_PID_NS=y
CONFIG_IPC_NS=y
CONFIG_UTS_NS=y
CONFIG_CGROUPS=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_SCHED=y
CONFIG_CPUSETS=y
CONFIG_MEMCG=y
CONFIG_KEYS=y
CONFIG_VETH=y
CONFIG_BRIDGE=y
CONFIG_BRIDGE_NETFILTER=y
CONFIG_IP_NF_FILTER=y
CONFIG_IP_NF_TARGET_MASQUERADE=y
CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=y
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=y
CONFIG_NETFILTER_XT_MATCH_IPVS=y
CONFIG_NETFILTER_XT_MARK=y
CONFIG_IP_NF_NAT=y
CONFIG_NF_NAT=y
CONFIG_POSIX_MQUEUE=y
CONFIG_DEVPTS_MULTIPLE_INSTANCES=y
CONFIG_NF_NAT_IPV4=y
CONFIG_NF_NAT_NEEDED=y
CONFIG_OVERLAY_FS=y
Furthermore you need to apply a small and easy patch to "net/netfilter/xt_qtaguid.c", otherwise the kernel will panic now and then. But its really easy to apply, take a look at the guide. Modification of Makefile "kernel/Makefile" is not necessary.
Click to expand...
Click to collapse
Basically all features flagged as missing when I ran the check-config file need adding to the defconfig file right ? It's not quite clear on the guide (maybe because it's common sense though I lack some so I better ask lol)
Also, it seems he is talking about a program that just lets you tick the features you want where as I am opening the file with a text editor really because I don't know better. This file defconfig only shows what's enabled.
I think I did download everything I need to compile the kernel, but not sure of it all yet (arch linux).
savvyyy said:
Basically all features flagged as missing when I ran the check-config file need adding to the defconfig file right ? It's not quite clear on the guide (maybe because it's common sense though I lack some so I better ask lol)
Also, it seems he is talking about a program that just lets you tick the features you want where as I am opening the file with a text editor really because I don't know better. This file defconfig only shows what's enabled.
I think I did download everything I need to compile the kernel, but not sure of it all yet (arch linux).
Click to expand...
Click to collapse
For building the kernel you will need an arm toolchain.
The UI for ticking your kernel options is called menuconfig.
What I'm mostly doing is first copying an existing device kernel config to the project root folder:
Code:
cp arch/arm64/configs/codename_defconfig to .config
Afterwards you can run the menuconfig. I used the following command (probably not all of the params are required):
Code:
make -s -j$(nproc --all) ARCH=arm64 CC=clang CLANG_TRIPLE=aarch64-linux-gnu- CROSS_COMPILE=aarch64-linux-android- CROSS_COMPILE_ARM32=arm-linux-androideabi- menuconfig
Copy modified config file back to configs folder:
Code:
mv new.config --> arch/arm64/configs/new_defconfig
Clean output folders and build kernel configuration file:
Code:
make clean
make mrproper
rm -R out
mkdir out
make ARCH=arm64 SUBARCH=arm64 O=out new_defconfig
Afterwards you can try to actually build your kernel.