Database Info

[Q] building ASOP generic ROM and applications

Hi.
I'm a veteran Android app developer but i am new to building roms, especially in the whole android build system.
It all started when trying to build the secured version of su.... i'm still unable to build it, the build system seems to ignore hte includes and claims there are undefined references to functions like get_property etc...
My main question is, i saw in the android build tree under /external that there are tcpdum, pcap and many nice to have applications i'll be happy to load on my device and compile for it (yes it's rooted)
My main steps in building (after checking out using repo) are:
run the . build/envsetup.sh script
run lunch generic-eng from the root of the srouce
run make
now my question is how do i make sure ALL the applications in the external folder are built ? should i change the build/code.mk to include their LOCAL_SRC name ? or is there a parameter i can pass to 'make' that should do the trick ?
Another question regarding the su itself is that it's Android.mk file did not contain LOCAL_MODULE_TAGS := sentence and i had to add it myself, anyone managed to successfully compile that file ?
Also anybody got some documentation regarding how to use the sqlite3 functions (or is it the std library and the documentation exist in the sqlite3 site ?)
10x

it seems to be a small problem with 2.3, once i checked out 2.2, compiled it using:
. build/envsetup.sh
launch generic-eng
make
all the binaries were built, since target was arm then all the binaries should be ok with the devices that use 2.2.
After the build i downloaded the su-binary from the git to $source_root/external/su-binary
changed the name in the Android.mk to su-binary so it wont collide with the system/external/su and run mm in the $source_root/external/su-binary dir.
Everything was build nicely.
I'll check the sqlite3 docs for refference but i'll appreciate anyone that knows the IBInder interface and the like that can explain the intnet send code (why not use am binary like in prev versions ? )

[Q] Is there an easy way to apply the patch files from the OTA on a PC?

I'm looking for a way to apply the patch files to the TF system image on a PC. This is not about how to get the ota to flash- I got that covered. This is about repacking roms or making flashable updates that don't rely on stock firmware.
I found a few ways, but none of them seem "easy."
The dirty way- apply the OTA and grab the system image.
From the PC command line - Linux has diff/patch, but that doesn't work for some of the patches in "IMGDIFF2" format. Besides, I'm not sure I ever got the regular BSDIFF40 patches to work.
The git option- There is the apply_patch source code in github. This works great, but I have to run it on the TF. Is it possible to compile this to work on my ubuntu box?
Maybe there is some magic script out there called apply_ota?
Thanks in advance

gee one said:
I'm looking for a way to apply the patch files to the TF system image on a PC. This is not about how to get the ota to flash- I got that covered. This is about repacking roms or making flashable updates that don't rely on stock firmware.
I found a few ways, but none of them seem "easy."
The dirty way- apply the OTA and grab the system image.
From the PC command line - Linux has diff/patch, but that doesn't work for some of the patches in "IMGDIFF2" format. Besides, I'm not sure I ever got the regular BSDIFF40 patches to work.
The git option- There is the apply_patch source code in github. This works great, but I have to run it on the TF. Is it possible to compile this to work on my ubuntu box?
Maybe there is some magic script out there called apply_ota?
Thanks in advance
Click to expand...
Click to collapse
I recently wrote some code for a ARM basestation (basically like a Freescale cell-phone board), and had to be able to supply some updates over time. Before the project ended, I'd also found bsdiff and bspatch and tested them on linux, where they worked fine (as long as you'd created either an ARM bsdiff/patch version or somehow took the big/little endianness of the hardware into account).
At first I thought it wasn't working but it seemed to do OK on standard cross-compiled-for-ARM files. (well, they still ran and performed the same stunt..).
The thing that worried me about this method is that they'd never have the same md5sum before as after, likely because they had a different amount of bytes. (e.g. you make your own before and after and build each, then create the patch file using bsdiff, then patch the old file using the patch file, and create a 'newtestfile', but look at the number of bytes (or md5sum it) difference between the original stage 2 file and the stage 1 + the patch file via bspatch).
Sorry that is pretty confusing the way I just tried to explain it.. I'll look this over again tomorrow and figure out if I said anything worthwhile

For inputs, the applypatch binary takes the file names, sha1 checksums of the before and after, and the final target size. Chance are high that the patch would only work if everything went in and came out properly.
Code:
apply_patch("/system/framework/framework-res.apk", "-",
99b382b5886e505508fa3f730d778a1bed4bc04e, 9761707,
a4798fb8d61d08b6f067e522e67107f6dd556148, package_extract_file("patch/system/framework/framework-res.apk.p"));
I think that bsdiff/patch will generally work on most of the OTA (BSDIFF40 files), but some of the files are IMGDIFF2, which targets binary/data files.
The applypatch binary works on both and as far as I can tell, is the same binary that the TF uses. Here is a snippet from the AOSP recovery... TF binary included Use the binary on your TF, not applypatch.sh! I suppose you can put it in system/xbin/
It would resolve the issue if I could compile this to run on my desktop from ubuntu. Short of that, I'm working on a script for the TF to parse the updater script.

gee one said:
For inputs, the applypatch binary takes the file names, sha1 checksums of the before and after, and the final target size. Chance are high that the patch would only work if everything went in and came out properly.
Code:
apply_patch("/system/framework/framework-res.apk", "-",
99b382b5886e505508fa3f730d778a1bed4bc04e, 9761707,
a4798fb8d61d08b6f067e522e67107f6dd556148, package_extract_file("patch/system/framework/framework-res.apk.p"));
I think that bsdiff/patch will generally work on most of the OTA (BSDIFF40 files), but some of the files are IMGDIFF2, which targets binary/data files.
The applypatch binary works on both and as far as I can tell, is the same binary that the TF uses. Here is a snippet from the AOSP recovery... TF binary included Use the binary on your TF, not applypatch.sh! I suppose you can put it in system/xbin/
It would resolve the issue if I could compile this to run on my desktop from ubuntu. Short of that, I'm working on a script for the TF to parse the updater script.
Click to expand...
Click to collapse
I'm not sure I understand the intent. I've just awoken and that's probably part of the problem..
'applypatch' is part of the standard (say 10.04 ubuntu) repo and is in a package called makepatch. Did you want to be able to run it "from" a standard x86 Ubuntu dist but have it targeting "armle" files or just be able to run it both on Ubuntu & Android correctly?
The files I tested were a) both binaries in different stages of evolution, and b) both built for armle(droids) and x86_64 and didn't behave differently, worked, and the checksum could be pre-calculated.
I'm going to mess around with applypatch when I'm out of my morning coma and get back to this in a bit.
Hah -- I do wonder if there's already a script (maybe it's a binary though) that lives in the dm* app on the tf101 that just does this already. You may have been right about that. Something has to apply those OTA patches. There's a few mechanisms for detecting them but I hadn't thought about how they're applied.
I'm trying to think if there's any scenario where the market applies 'patches' rather than always downloads the next version in entirety. Not sure about that.
Anyway, I'll see if I can get anywhere with applypatch.
Edit:: After a little messing around :
I finally see what you are trying to do and why what I'm doing is totally useless for that. I'm going to spend a little time looking over the imgdiff2 (apk) deal, and probably end up now figuring out a thing.

I'll look into the makepatch package later today. Maybe that might be what I'm looking for.
Just to be clear, the intent is to be able to patch a "stock" system without it actually running on the TF. For example, to take the stock 8.6.5.9 image, decompress system/ to the PC, apply the patches from the OTA, and then end up with a stock 8.6.5.13 image. This would all happen on the PC, not the TF. It would streamline the ROM cooking process and make it easier and faster to spit out a complete, flashable update that won't fail sha1 checks because of mods.
sent from my cyanogen(mod) vision

Zombie Thread Alert!!!
You have been warned....
OK, so my C-fu isn't strong enough to figure out how to cross compile the apply patch binary to run on a x86 computer. However, I did notice that the applypatch binary gets built in the aosp/generic_x86 build, specifically in the system/bin folder.
This will be an exectuble file that will run on a x86 linux based system and do the same thing that the applypatch binary does to the ota package during the ota update. In other words, you can apply the ota patch to a system image from the comfort of the linux command line without having to actually flash the ota. This will allow you to be the first one on your block to have the new system image without having to mess up whatever rom you are currently running.
Of course, this might all be moot if we don't see another ota for our tf. Come on ASUS, just one more so I can test this out on a live update.
OK, so if you read this far, repo sync your aosp source and build the generic_x86 branch. The binary is aosp/target/product/generic_x86/system/bin/applypatch_static
edit: then after you've done that, use a script to parse the updater-script to extract the patching commands and then she-bang! I'll post mine a little later- I've got to get some flowers for tonight

Any luck with that? I'm also searching for a way to apply an OTA update to a ROM using Ubuntu.

bgcngm said:
Any luck with that? I'm also searching for a way to apply an OTA update to a ROM using Ubuntu.
Click to expand...
Click to collapse
Yeah, it works like a charm. The binary is compiled in the aosp/generic_x86 build. There is no need to flash, you can just pull it out of the system/bin folder after compiling the source. You can probably use an "mmm" command to make just files you want, but I'm not sure which package it's in. Then, I use a script that converts the OTA patch commands in the updater-script to a script file that runs on my linuxbox that calls the binary and patches the ROM under linux/ubuntu.

Do you mind sharing your applypatch binary? I don't have AOSP source on my Ubuntu installation right now, so if you don't mind sharing I wouldn't need to build it from sources...

bgcngm said:
Do you mind sharing your applypatch binary? I don't have AOSP source on my Ubuntu installation right now, so if you don't mind sharing I wouldn't need to build it from sources...
Click to expand...
Click to collapse
I attached the applypatch binary- it should work on most x86 linux distros, maybe.
As I stated in my earlier post, the binary is built with the x86 build. I'm not so good with the build system, but I think the code is from the bootable/recovery/applypatch. I included the relevant notices for that code.
In the OTA updater-script, there will be a bunch of applypatch commands- you might have to change the name/permissions of the binary, but that syntax should be what you are looking for.
Enjoy!

Thank you very much for your time. It seems we are almost the only one trying this. The thing is that I really need to get this done because OTA update is failing for me by running it the usual way.
I've tried your binary and it works under my Ubuntu installation, however I was trying to manually patch just one file and got this:
Code:
[email protected]:~/Desktop/system/app$ applypatch_static ./ApplicationsProvider.apk - d9023cd58fd055e1ca3c8f8492b2c36aba923c6a 6184 69bea5d5a62980c611b903de8243d980f30e5fb5:../../patch/system/app/ApplicationsProvider.apk.p
applying patch to ./ApplicationsProvider.apk
target 6184 bytes; free space 1080520704 bytes; retry 1; enough 1
chown of "./ApplicationsProvider.apk.patch" failed: Operation not permitted
I got that chown error message although ApplicationsProvider.apk.patch was created. Is this the patched file?
EDIT: Nevermind, I forgot to run the command as superuser.
EDIT2: Would it be too much if I ask you also to share the script that converts OTA commands from updater-script into the correct syntax to run on Ubuntu?

There is no guarantee that this will work on your machine. It's a work in progress and will probably require some editing. This just creates the scripts that extract the commands from the OTA updater-script.
I haven't used it in a while since ASUS hasn't put out any updates lately, but I would copy the updater-script and rename it delete_source. Then I would edit out everything in delete_source except the delete commands. There might be an issue with a trailing ; or similar character. The script runs on the copy of the updater-script to pull out the applypatch commands and the delete_source to pull out the files that need to be deleted from the rom.
Check the syntax, but it will be something like ./tf_file <head of stock rom> <head of patch source>
If you have the donor stock system folder and the patch folder from the OTA at the same dir level, it would be ./tf_file system patch
The script also assumes that the applypatch binary is in your path as apply_patch. You can adjust as needed.
Lastly, my bash skills are probably pretty weak, so if you see ways to improve this script or add error/path checking, feel free to post improvements, etc. There is probably a really slick way to do this with just one script and not a script that makes three other scripts. The historical reason for this is that I used to transfer the files to my transformer and run the script on there. Running in Ubuntu is much faster!
Good luck!
Code:
#!/bin/bash
# extract_source is used to pull files to be patched from source tree
# delete_source is ued to delete unneeded files from the source tree
# tf_file applies the patches to the extracted source files
# don't forget to copy the direct updates in the OTA
# usage is apply_ota
# assumes that updater-script and delete-script are at same directory level
# updater-script is from META-INF folder, delete-script is from updater-script
# add error checking to count number of patches and number of patched files
# parse trailing / in folder names
work_dir=`pwd`
tf_file=$work_dir"/tf_file"
extract_source=$work_dir"/extract_source"
delete_file=$work_dir"/delete_source"
work_file=$work_dir"/work_file"
echo "#!/bin/bash" > $tf_file
sed ':a;N;$!ba;s/\n/\ /g' updater-script | sed -e 's/;/\n/g' -e 's/\ \ */\ /g' | grep "apply_patch(" | sed -e 's/"//g' -e 's/,\ p/\:p/g' -e 's/,//g' -e 's/package_extract_file(/$2\//g' -e 's/(/\ $1/g' -e 's/))//g' >> $tf_file
chmod 755 $tf_file
echo "#!/bin/bash" > $extract_source
#sed ':a;N;$!ba;s/\n/\ /g' updater-script | sed -e 's/;/\n/g' -e 's/\ \ */\ /g' | grep "apply_patch(" | cut -d \" -f 2 |sed -e s'/^/cp\ --parents\ $1/g' -e s'/$/\ \$2"\/\"/g' >> $extract_source
sed ':a;N;$!ba;s/\n/\ /g' updater-script | sed -e 's/;/\n/g' -e 's/\ \ */\ /g' | grep "apply_patch(" | cut -d \" -f 2 |sed -e 's/^/cp\ --parents\ $1/g' -e 's/$/\ \$2"\/\"/g' -e 's:/system::g' >> $extract_source
chmod 755 $extract_source
sed -e 's:",:\n:g' -e 's:\ ::g' delete-script > $delete_file
sort -r $delete_file > $work_file
echo "#!/bin/bash" > $delete_file
grep / $work_file | grep -v /$ | sed -e 's:"/:rm\ \$1/:g' >> $delete_file
grep /$ $work_file | sed -e 's:"/:rmdir\ \$1/:g' >> $delete_file
chmod 755 $delete_file
exit 0

IMGDIFF Tool Needed...
Hello Sir, i am looking for a tool to generate IMGDIFF patches on windows... i already have a BSDIFF tool, but coudn't find IMGDIFF... Thank You

I have no idea about windows, but the source is here.
https://android.googlesource.com/platform/bootable/recovery/+/master
You could install linux and build it from there. Follow these directions http://source.android.com/source/building.html
There is also lots of great info here http://wiki.cyanogenmod.org/w/Development#Learning_To_Build_CM
Good luck!

gee one said:
Good luck!
Click to expand...
Click to collapse
I am very thankful to you for replying me sir... these guides are awesome, but i dont think my slow internet connection and PC with 1GB RAM will be allowing me to download or compile
actully i was making a tool to apply OTA Updates in PC, it is semi functional for now, it is able to read updater script and apply bsdiff patches, but can't handle imgdiff patches and now i am stuck at this point because of no imgdiff port for windows... :crying:
i will be very thankful if you or @bgcngm can compile imgdiff at least for cygwin? else i will have to postpond this project until someone else compile and port imgdiff for win/cygwin

Sorry- I'm clueless about cygwin. You could try dual booting to linux to compile from there.

gee one said:
You could try dual booting to linux to compile from there.
Click to expand...
Click to collapse
Thanks alot for taking your time to reply me sir now it seems like linux is the only option left
as you are a linux user will it be too much if i ask you to find a right guide to compile ota tools only? cause i don't know much about linux, and it will be difficult for me to find right guide to compile ota tools only...

jawad6233 said:
Hello Sir, i am looking for a tool to generate IMGDIFF patches on windows... i already have a BSDIFF tool, but coudn't find IMGDIFF... Thank You
Click to expand...
Click to collapse
Answer is here - [Tool]imgdiff executable for windows. BTW, anybody have applypatch for Win32?

Hoping this works for lollipop.

[GUIDE][HACK]Cross Compiling for OSX on Linux with AOSP

Hi Folks
I wasn't sure where this should belong but as it is a bit of an Hack this forum is probably the most appropriate
Introduction
This short tutorial will show you how to patch the Android Build System to allow you to cross-compile Android AOSP host tools ( adb, fastboot etc ) for OSX using a linux based machine. This is something Google said was impossible or at the very least unsupported.
Assumptions
You have a linux based machine and working copy of the AOSP source tree.
You can/have successfully compile(d) a full Android Release from this tree.
A basic idea of how the Android Build System works is beneficial.
Getting Started
I've set-up a git repository which contains a binary copy of the OSX SDK 10.6 and the apple-darwin10-gcc cross compiler. So first things first. open a terminal and set the root of the AOSP sources tree to the current directory.
STAGE 1: Copy the OSX SDK
Step 1.
Clone the repo with the SDK and toolchain
Code:
git clone https://github.com/trevd/android_platform_build2.git build2
Step 2.
Create /Developer directory at your filesystem root, this is a known location for the SDKs
Code:
sudo mkdir /Developer
sudo chown $USER.$USER /Developer
Step 3.
Copy and unpack the SDK package
Code:
cp build2/osxsdks10.6.tar.gz /Developer
cd /Developer
tar -zxvf osxsdks10.6.tar.gz
rm osxsdks10.6.tar.gz
cd - # back to aosp root
STAGE 2 : Swapping the Toolchain
This is where the fun begins :laugh:
The Android Build system has the majority of the infrastructure in place already to build for OSX, the only problem is that you need OSX to build for OSX. However we can remedy that with a couple of dirty hacks :laugh:.
The prebuilts/gcc/darwin-x86 directory contains a toolchain compatible with osx ( mach-o binaries ). We are going to swap this for a linux compatible ( elf ) executables.
Step 4:
Copy and unpack the elf compatible darwin cross toolchain
Code:
cp build2/i686-apple-darwin-4.2.1.tar.gz prebuilts/gcc/linux-x86/host
cd prebuilts/gcc/linux-x86/host
tar -zxvf i686-apple-darwin-4.2.1.tar.gz
cd - # back to aosp root
Step 5:
Remove the mach-o binaries and symlink the elf binaries in it's place
Code:
cd prebuilts/gcc
rm -rf darwin-x86
ln -s linux-x86 darwin-x86
cd - # back to aosp root
Step 6:
We also need to replace the mach-o version of the ccache executable which live in the prebuilt/misc directory
Code:
cd prebuilts/misc
rm -rf darwin-x86
ln -s linux-x86 darwin-x86
cd - # back to aosp root
STAGE 3: Patching the build system .mk files
We need to patch a couple of files in the build directory namely the build/core/combo/HOST_darwin-x86.mk the main crux of this is swapping the ar tool for libtool so static libraries can be created without error.
Code:
patch -p1 < build2/build.patch
If the patch has been applied successfully you should see the following
Code:
patching file system/core/adb/Android.mk
patching file build/core/combo/HOST_darwin-x86.mk
patching file build/core/combo/select.mk
patching file build/core/envsetup.mk
patching file build/envsetup.sh
You are now ready to cross compile!! :good: ..... well not quite, but nearly.... here's why!
The Android Build System will attempt to build both the Target and Host files for most modules so I'd advise using a lunch option which already has a full target built for it or alternatively you can build the generic sdk using the following commands at the AOSP source tree root.
Code:
. build/envsetup.sh
lunch sdk-eng
make sdk
This will stop target dependency errors occurring when you build individual modules.
NOW we're ready to cross compile.
STAGE 4: Building Modules
At present module build is very much a piecemeal process. To build adb for example we need to build the dependencies first. This is not too onerous as most host modules have very few dependencies.
Building adb
adb has dependencies on the following libraries
Code:
external/zlib
external/openssl
system/core/liblog
system/core/libcutils
system/core/libzipfile
I've found the easiest way to compile the dependencies is to navigate to each directory in turn an use to "mm" build system command to compile the individual module. the commands I run to compile adb are as follows.
From AOSP Source Root
Code:
cd external/zlib
USE_DARWIN=true mm -j8
cd ../openssl
USE_DARWIN=true mm -j8
croot # go back to the AOSP root
cd system/core/liblog
USE_DARWIN=true mm -j8
cd ../libcutils
USE_DARWIN=true mm -j8
cd ../libzipfile/
USE_DARWIN=true mm -j8
cd ../adb
USE_DARWIN=true mm -j8
All being well you should now have and adb binary located at out/host/darwin-x86/bin/adb. running the file command on this binary should produce the following output
Code:
adb: Mach-O executable i386
Conclusion
Although this method is a little rough and ready, it should produce the desired results if you need to cross compile for OSX. The eventual goal would be to compile a full OSX Android SDK on linux in a similar manner to the way the windows-sdk is currently compiled. This requires more investigation as compiling the windows sdk on linux employs a little bit of trickery on the part of the build system.
Final Notes and FAQs:
Why can't I just type make <module> from the root?
Doing this triggers building of additional modules such as LLVM and clang which are to deployed out/host/darwin-x86/bin the build system then attempts to use binary later on. These are obviously built for the Mach-o architecture and as such are incompatible with the linux. This results in a build error which can and would be resolved by the above mentioned trickery ( see conclusion )

I use OSX binaries (along with Windows and my native Linux) in one of my projects. Thanks, I have always relied on finding compiled binaries elsewhere. Lack of an OSX aapt held up an update at one point.
One of those things that you don't really use until you need it, but I will try to remember to give it a shot. I don't have any doubt that it works.

mateorod said:
I use OSX binaries (along with Windows and my native Linux) in one of my projects. Thanks, I have always relied on finding compiled binaries elsewhere. Lack of an OSX aapt held up an update at one point.
One of those things that you don't really use until you need it, but I will try to remember to give it a shot. I don't have any doubt that it works.
Click to expand...
Click to collapse
Thanks. Yes this really is an edge case. Hopefully It will help some folks out.
Regarding aapt in particular.... It's perfectly possible to build aapt, however, we need to do some slight of hand with the clang and clang++ executables as libpng on which aapt depends uses these 2 binaries as part of it's build process.
Here's the build list and the clang trick if you want to try it some time.
Code:
build/libs/host
external/expat
external/zlib
system/core/liblog
system/core/libcutils
mkdir out/host/darwin-x86/bin
cp out/host/linux-x86/bin/clang out/host/darwin-x86/bin
cp out/host/linux-x86/bin/clang++ out/host/darwin-x86/bin
external/libpng
frameworks/base/libs/androidfw
frameworks/native/libs/utils
frameworks/base/tools/aapt
I started off with a clean out/host/darwin-x86 directory so I didn't miss any dependencies.
like I mentioned the clang "swap out" is something the make win_sdk option does automatically so with it a little more research I should be able to get the mac build to do the same but you'll have to "fill yer boots" with the ghetto method for now
For reference here's a link to the sdk building instructions http://tools.android.com/build which describes how to cross compile the windows sdk on linux ( in case anyone was wondering what the hell i'm on about)

My use case has come up
I will be cross-compiling for OSX today...specifically with aapt in mind. I will report back, but I fully expect it to work as described.

mateorod said:
My use case has come up
I will be cross-compiling for OSX today...specifically with aapt in mind. I will report back, but I fully expect it to work as described.
Click to expand...
Click to collapse
Cheers Man!
Hopefully no bitrot has crept in since april and now. I know I've changed my OS version since to Lubuntu 13.04, not like the OS version really matters any.
mateorod said:
but I fully expect it to work as described.
Click to expand...
Click to collapse
Then you Sir, are either Drunk or a Fool! LOL Keep expectations Quantum and only decided when the result is observed a'la Schrodinger Cat

okay...So I was trying to compile SlimRom (so as to get an OSX aapt binary with the SlimRom changes) and things did not necessarily go as planned. There were enough changes to the SlimRom android_build that your build/build.patch does not apply cleanly. I spent some time and tried to modify the patch so that it would work for both SlimRom, AOSP and probably others, but each android_build repo has some differences in surrounding the HOST_AR line, so commenting that just was not portable between flavors. Not cool.
Anyway, turns out that this method does not quite work out of the box for non-AOSP versions (not that you claimed that it did). I got some unfamiliar errors related to (I believe) some OSX toolchains. But in both times I tried this, I actually had to pretty immediately swap out of that flavor and so I was unable to do much debugging. (I keep all the flavors I build {CM, AOKP, SlimRom, PAC, PA, OpenPDroid, etc, etc, etc} all layered in one android/system/jellybean directory. It saves a ton of space, but only allows me to do one thing at a time.)
So the only feedback I have is nothing...I even formatted my hard drive in-between and forgot to put up a paste, so the errors are currently lost to history.
Things that I noticed, for better or worse
You recommend putting the SDKs in the root dir. I believe the documentation is recommending the Developer be placed in home (as per the SDK/ADT bundle docs).
You might want a
Code:
mv android_platform_build2 build2
line. I normally wouldn't bother, but it looks like you are trying to post a line-by-line guide.
I would put the recommendation that a full build be available to the out folder (or a built generic sdk) right at the top, since it is a preliminary step. I had to revert my handwritten changes, then build, then reapply the changes and rebuild since I thought it needed a clean out dir.
Did you have any trouble with git reverting the toolchain swap? On two separate machines, I had to go so far as to delete .repo/projects/prebuilts/gcc/* and prebuilts/gcc/darwin-x86/arm/arm-eabi-4.6. It kept complaining of that the project in the .repo folder was a bad match. No amount of git trickery (which I am not terrible at) let me back out more easily.
I am willing to try again...but I have some other small things to attend to first. It is an admirable hack you have here sir. I will return to it soon and report back once more.

mateorod said:
okay...So I was trying to compile SlimRom (so as to get an OSX aapt binary with the SlimRom changes) and things did not necessarily go as planned. There were enough changes to the SlimRom android_build that your build/build.patch does not apply cleanly. I spent some time and tried to modify the patch so that it would work for both SlimRom, AOSP and probably others, but each android_build repo has some differences in surrounding the HOST_AR line, so commenting that just was not portable between flavors. Not cool.
Anyway, turns out that this method does not quite work out of the box for non-AOSP versions (not that you claimed that it did). I got some unfamiliar errors related to (I believe) some OSX toolchains. But in both times I tried this, I actually had to pretty immediately swap out of that flavor and so I was unable to do much debugging. (I keep all the flavors I build {CM, AOKP, SlimRom, PAC, PA, OpenPDroid, etc, etc, etc} all layered in one android/system/jellybean directory. It saves a ton of space, but only allows me to do one thing at a time.)
So the only feedback I have is nothing...I even formatted my hard drive in-between and forgot to put up a paste, so the errors are currently lost to history.
Things that I noticed, for better or worse
You recommend putting the SDKs in the root dir. I believe the documentation is recommending the Developer be placed in home (as per the SDK/ADT bundle docs).
You might want a
Code:
mv android_platform_build2 build2
line. I normally wouldn't bother, but it looks like you are trying to post a line-by-line guide.
I would put the recommendation that a full build be available to the out folder (or a built generic sdk) right at the top, since it is a preliminary step. I had to revert my handwritten changes, then build, then reapply the changes and rebuild since I thought it needed a clean out dir.
Did you have any trouble with git reverting the toolchain swap? On two separate machines, I had to go so far as to delete .repo/projects/prebuilts/gcc/* and prebuilts/gcc/darwin-x86/arm/arm-eabi-4.6. It kept complaining of that the project in the .repo folder was a bad match. No amount of git trickery (which I am not terrible at) let me back out more easily.
I am willing to try again...but I have some other small things to attend to first. It is an admirable hack you have here sir. I will return to it soon and report back once more.
Click to expand...
Click to collapse
Hi
Thanks for this, It sounds like you've suffered an exercise in frustration there. I wasn't aware that "SlimRom" had a different aapt ( just out of general ignorance and not having paid any attention )
SDK - My Tree last time I used this was /Developer directory in the root - I think It comes from what the toolchain is expecting, I just gave it what it wants
mv android_platform_build2 build2 - Yep I did mean that it's the git clone line which wants changing
Code:
git clone https://github.com/trevd/android_platform_build2.git build2
SDK Recommendation - I shall move that to the top, even though it is already in there, It should probably be highlighted better and possible it's own "Stage"
Reverting the toolchain - Ahh , It appears I work slightly different from most in this respect. I have a general mistrust of SCM's ( I lost too much code on too many different SCM's, Probably through my own inability to use them correctly but ) what I do to revert to change is
Code:
cd prebuilts/gcc/darwin-x86/host/
rm -rf i686-apple-darwin-4.2.1
repo sync i686-apple-darwin-4.2.1
You can do this "trick" on any project in the source tree it's only on rare occasions where I screwed up badly that I have to delete anything in .repo/projects but I also have my distro in their own individual directories with there own full git trees, which is a massive waste of space and has a ton of redundancy due to the AOSP repositories being mirror by every single one but switching between them is a lot easier
If SlimRom's changes are localized to aapt, I'd be more inclined to drop it into the AOSP build and try that... If you have a link to slimrom's frameworks/base repository I'll grab it and try it myself.
On a final note there's a "full version" of the HOST_darwin make file in the build2/core/combo directory the changes to envsetup.mk and select.mk are minimal and can easily be applied manually. You don't need to patch the adb makefile if your not building it.
Again Thanks for the feedback

[KERNEL] Enabling Netfilter/AFWall+ on Ainol Novo 7 Paladin (MIPS)

I spent entirely too much time on this, so I'll post my boot.img and my findings here in case it's useful to anyone else in the future.
Paladin ROMs:
This page and this thread have working links to CWM and Superuser zips.
This page has a link to an ICS 4.0.3 ROM running Linux 3.0.8. I installed this ROM, but was disappointed with the kernel's feature set. Important options like CONFIG_IP_NF_TARGET_REJECT and CONFIG_IP_NF_TARGET_LOG were disabled.
This page has a custom ROM by "pinkflozd". I wasn't able to find this image in the wild. It may have a newer kernel (3.0.23?) but AFAICT it doesn't have the Netfilter changes I needed either.
Custom kernel build:
I stumbled across this kernel and used it as a baseline. Then I enabled:
Various missing Netfilter features (e.g. LOG and REJECT targets)
CONFIG_TUN
CONFIG_CIFS
The process looks like:
Install CWM
Find the .zip file for the ROM you want to use, e.g. the ICS 4.0.3 link above
Install that ROM and make sure the stock kernel works
Create a new boot.img with the desired options enabled
Flash the new boot.img with CWM
Basic steps to create the new boot.img:
Code:
git clone -b paladin-3.0.8 git://github.com/cernekee/linux
cd linux
git submodule init
git submodule update
cd paladin
cp /path/to/boot.img orig.img
make NDK=/opt/android-ndk-r9b
# if desired, edit kernel code or .config, and rerun "make"
The scripts under linux/paladin/ will extract the ramdisk from orig.img, then create a new boot.img based on your kernel source tree. For more details on the kernel build, see paladin/README .
Kernel installation:
To flash the new image, reboot into CWM, then:
Code:
adb push boot.img /tmp/
adb shell "dd if=/tmp/boot.img bs=1048576 seek=3 count=8 of=/dev/block/mmcblk0 ; sync"
# hit the reset button again
Other random findings:
I did not have any luck with the "z4root" app, although ro.secure=0 on this ROM so "adb shell" always gives you a root shell. It did take some work to find Superuser/su binaries that worked on MIPS.
Enabling Netfilter's conntrack option seemed to cause dhd.ko to become unstable, possibly because it affected binary compatibility with existing kernel modules. So I left it alone.
This ROM is missing native iptables/ip6tables binaries, so AFWall needs to be set to use the builtin copies. The latest AFWall betas do ship with MIPS binaries now (the purpose of this exercise was to test them).
Checking the box to enable IPv6 crashes AFWall instantly. Need to investigate this.
AFWall's inbound connection option probably will not work, due to the lack of conntrack in the kernel.
This ROM appears to be missing some Java libraries needed for ICS+ VpnService apps to work.
NDK r9b is the first NDK release with support for the "MXU" SIMD instructions supported by the Ingenic JZ4770 chip. Prior to this, most people used Ingenic's special toolchain to build the kernel.
To forcibly boot into CWM, hold down VOL+ while pressing the reset button.

[GUIDE] [BIN] Compile busybox on Linux

About Busybox: busybox.net/about.html
More on Busybox: busybox.net
This is just for anyone who wants to try, and especially those without access to a PC.
:
: --------------- BUILD STEPS --------------
:
Things we'll need besides your phone:
- "Android Terminal Emulator" app
- "Complete Linux Installer" app , I also recommend trying "linux deploy" for more advanced usage
- internet/wifi
- latest "busybox" source
1) We need to get Ubuntu or Debian booted for a sufficient build environment (kali linux works well too). I've used them all on Android but I like the better stocked terminal in the Ubuntu images. I used the app "Complete Linux Installer" which is free and works beautifully, very simple and easy. In the app you want to follow the short instructions to download an Ubuntu image, rename it to ubuntu.img, and place it in a folder named ubuntu in /sdcard. Then hit menu in the app and click the terminal screen icon that says "Launch". An Ubuntu terminal will now open in Android Terminal Emulator. Super quick and easy.
2) Let's download some crucial build environment tools.
Code:
apt-get install -y gcc build-essential libncurses5-dev libpam0g-dev libsepol1-dev libselinux1-dev
--EDIT-(30AUG2014)--
For Selinux compatibility and loginutils, we need to also download a few extra packages. Already included in the code above.
3) Now the cool thing about this chroot Ubuntu environment is that we still have access to the sdcard to transfer files between Android environment and the chroot jail. Extract your downloaded busybox source to your Ubuntu home with something like:
Code:
cd
tar -xf /sdcard/Download/busybox*bz2
cd busybox*
4) Now we can build busybox statically. The first thing we do is generate a Makefile by running "make" with a "defconfig" (default configuration file) Usually you will run "./configure" with other programs, but busybox compiles more like a kernel, so it uses a config which has a huge checklist of options.
(After successfully compiling busybox, we can go back and customize the .config; this entails that for each "CONFIG ..." line we see, we can uncomment it and mark it "y" or "n" to configure some option... This can be more easily done from a terminal busybox menu interface with "make menuconfig". You just need to crank font down to 7 or use telnet/ssh)
Skip "make defconfig" if you use a customized ".config" file such as one I've attached.
Code:
make defconfig
If all goes well, we now have a Makefile and are ready to compile:
Code:
make clean && make LDFLAGS=-static
Let "make" crank out the binary for a couple minutes. The extra variable we set after make is to compile statically. When compiling is complete we'll have a few different busybox binaries at the root of the source directory. We use the one named "busybox" since we're not debugging.
5) Now let's copy it to /system/usr/bin to install for test usage.
Code:
cp ./busybox /android/data/media/0
(Open a new terminal tab to get into Android Environment)
mount -o remount,rw /system
mkdir -p /system/usr/bin
cp -f /sdcard/busybox /system/usr/bin
chmod 0555 /system/usr/bin/busybox
/system/usr/bin/busybox --install -s /system/usr/bin
mount -o remount,ro /system
PATH+=:/system/usr/bin
.. and done. Run some scripts and enjoy your static busybox!
:
: Extra steps for SELinux-enabled busybox
:
Here are the extra steps you need to take to compile busybox with SELinux features. Sorry it took so long to get this added to this first post.
First we need to download the source for libselinux and libsepol and compile it. (This is for use with the standard glibc toolchain.)
Code:
cd
apt-get source libselinux libsepol
cd libselinux*
make
cd
cd libsepol*
make
Now that we have those libraries compiled, we can proceed to the busybox compilation.
Code:
cd
cd busybox*
make clean && make LDFLAGS='-static -L ../libselinux*/src -L ../libsepol*/src' CFLAGS='-Os -I ../libselinux*/include -I ../libsepol*/include'
That's pretty much it. It initially seems more complicated than it actually is, but all we're really doing is including the libraries for libselinux and libsepol into the busybox compilation.
edit:
**Commands to run if you have compile errors:
Code:
apt-get build-dep busybox
apt-get install -y build-essential
apt-get -f update
dpkg --configure -a
:
: --------------- DOWNLOADS --------------
:
***** Attached are flash installers for busybox (v1.23.1 stable, non-SELinux, 374 applets included!, ~1.1MB size) or busybox (v1.23.1 stable, SELinux, 386 applets included!, ~1.6MB size) *****
Since it's up-to-date it has some nice extras most people haven't seen like a "-I" option for xargs! Yes, that is correct, busybox xargs has its testicles back.
Code:
e.g.
$ echo Hello | xargs -I{} echo {} world!
> Hello world!
: ---------- UPDATES ----------
-------------------EDIT-2-(30AUG2014)----------------------
Got a Selinux-enabled busybox attached now. This means Selinux flags are integrated into applets like ls, id, ps, etc, and there are now 12 extra Selinux applets to give a total of 386 applets, ~1.6MB in size. The previous one is more portable, but this one can completely replace toolbox and gives you Selinux control for Android 4.4+. Plus it's pure maxed-out awesomeness.
***I've also attached the .config files for each busybox I've compiled, so anybody can remake them (from their phone!) as I've posted. You just need to download and extract the .config file to the root of your busybox source directory and type "make".***
-------------------EDIT-3----------------------
YashdSaraf has made some very useful flash zips to install either the non-selinux- or selinux-enabled busybox 1.23.1 via recovery. Installation replaces the stock busybox in /system/xbin. I've attached the zips he made to the end of this OP.
(**Note: Thought I'd mention that there will be a handful of applets that don't work in "Android" environment such as su(don't worry this isn't linked with the installer) Part of this is because of the way Android's default file structure is an amputated modified version of linux. With almost all of them, slight modifications to environment and file structure can be made to get them to work. This is just normal behaviour of busybox in android. The su and init applets shouldn't be used in Android though. I keep them compiled into the binary anyway for completeness of the build and because they work and are needed for a root.gz initrd or some chroot environments. It also doesn't hurt keeping them compiled in. You just have to remember not to link them when installing busybox.
-------------------EDIT-4-(06SEPT2014)----------------------
:
: How to compile against(using) uclibc for a smaller binary!!
:
Download the attached arm-linux-uclibcgnueabi toolchain package that I pre-compiled. Extract to /data/media:
Code:
cd /data/media
zip='/sdcard/Download/2014-09-06__arm-buildroot-linux-uclibcgnueabi.tar.lz.zip'
unzip -op "$zip" | lzip -d | tar -xf -
Then let's open up the "Complete Linux Installer" or "Linux Deploy" terminal.
To use the toolchain with a busybox build, we just need to specify the parameter CROSS_COMPILE which is a prefix to the compiler tools. So if you extracted the toolchain to /data/media, you will use:
Code:
make clean && make LDFLAGS=-static CROSS_COMPILE=/android/data/media/arm-buildroot-linux-uclibcgnueabi/bin/arm-buildroot-linux-uclibcgnueabi-
When you're done you should have a busybox binary with 374 functions with size around 1.1MB. That's a 20% decrease in size from using the standard glibc toolchain!
**IMPORTANT Notes
- The toolchain can't be used with lollipop since it's not compiled with -fPIC. I'll fix this later. Busybox is fine since it's static, it's just the toolchain I uploaded.
- Selinux-enabled busybox .config errors out when building using the uclibc toolchain; I think this is a lack of support issue. In the "Complete Linux Installer" app you'll need to add the mount "/data/media" in options. This gives you access to it as "/android/data/media", very very useful for extra space needs.
Difference between SELinux and non-SELinux busybox
The SELinux (NSA security enhanced Linux) binary comes with the following extra utilities: chcon, getenforce, getsebool, load_policy, matchpathcon, restorecon, runcon, selinuxenabled, setenforce, setfiles, setsebool, and sestatus. There are also some selinux flags enabled for applets such as "ps" and "ls", e.g. "ps -Z" and "ls -Z" to show the context for processes or files. If you are using Android 4.3 or newer, then you probably want to use the SELinux-enabled busybox since Android 4.3 is when SELinux was introduced to Android. Using the SELinux busybox on older version of Android without SELinux file structure should probably work besides the SELinux applets, but I haven't tested this. The non-SELinux binary can be used on any version of Android. When it comes down to it, the system actually uses "/system/bin/toolbox" SELinux applets for SELinux operations, so unless you specifically want to use busybox's SELinux tools for personal use, the safest option is to go with the non-SELinux busybox. I use Android 4.3.1 and 5.x, so I use busybox's better featured SELinux tools.
Latest updates see post 2
Busybox 1.23.1 (2015-02-06) below

Busybox compilation on Linux
reserved

Great Info here!
But I would be interested to know how well this method works on Samsung Stock devices running AOS 4.2 and above? Any experience?

Awesome info, this thread came up #1 while googling busybox 1.23
I made a flashable zip of the attached binary in the op to clean the old one(if any) and install the new busybox in xbin, just in case if anyone needs it. :good:

Is it work on xperia sp on 4.3 fw yes ?

YashdSaraf said:
Awesome info, this thread came up #1 while googling busybox 1.23
I made a flashable zip of the attached binary in the op to clean the old one(if any) and install the new busybox in xbin, just in case if anyone needs it. :good:
Click to expand...
Click to collapse
Thanx,worx fine with Carbon 4.4.4 on my LG.
GREETZ FROM TEAM-OPTIMA!!!

E:V:A said:
Great Info here!
But I would be interested to know how well this method works on Samsung Stock devices running AOS 4.2 and above? Any experience?
Click to expand...
Click to collapse
Thanks man. I've been compiling tons of stuff with Debian and Ubuntu chroot no problem on top of 4.3.1 Vanir and also 4.4.4 Carbon, both are my daily drivers. "Complete Linux Installer" is pretty fast compared to some other chroot apps like GNUroot (no offense to GNUroot, it works but is way too slow). It runs real-time compared to non-chroot. When compared to my dual-core 2007 Pentium M laptop, it's about 2-3 times as slow which isn't too bad for compiling something like mksh or even busybox which takes up to 5 mins I'd say.
In terms of binary size, compiling natively is better than cross-compiling it seems. I used gcc with no size optimizations here, so 1.37MB is pretty nice compared to some others around 2MB with full configs. With this method and klcc (gcc wrapper) I got mksh compiled to 192KB. I'm currently trying to build a uclibc toolchain on my laptop that will give me a mksh binary under 300KB..
YashdSaraf said:
Awesome info, this thread came up #1 while googling busybox 1.23
I made a flashable zip of the attached binary in the op to clean the old one(if any) and install the new busybox in xbin, just in case if anyone needs it. :good:
Click to expand...
Click to collapse
Cool thanks man! That is really useful, glad to hear from CALIBAN that it works. Could I add this to the OP with credit to you?
Hamidreza2010 said:
Is it work on xperia sp on 4.3 fw yes ?
Click to expand...
Click to collapse
Yes, xperia sp uses armv7 so you should be good to go.

7175 said:
Cool thanks man! That is really useful, glad to hear from CALIBAN that it works. Could I add this to the OP with credit to you?
Click to expand...
Click to collapse
Sure bro go ahead
Edit: Went ahead and made one for selinux enabled busybox :silly: , you can add this one in the op too.

Hey guys I was able to get an entire uClibc toolchain built the other day (using buildroot). I tested it and it makes some nice small binaries with about 20%+ smaller size than the standard glibc. Man that took hours to compile but was well worth it. It really put the stability of Android OS to the test as well. Kitkat absolutely couldn't finish compiling with multiple oom's and oops's, but Vanir 4.3.1 stuck it out real nice. Once I had the huge amount of required buildroot packages downloaded, I was able to compile in TWRP as well with good stability. (I have the "Complete Linux Installer" startup chroot script aliased in my mkshrc so I can pull up an ubuntu terminal without starting the app. )
So I got 3 new attachments to OP:
- arm-linux-uclibc toolchain for anyone who wants to compile stuff with it (host=arm AND target=arm)
- busybox (374 fcns, non-selinux) 1116KB
- lzip binary (in case you need it to unzip the toolchain, which is highly compressed from 64MB to 14MB with lzip's lzma)
**As I mentioned in the updated OP, I wasn't able to get selinux-enabled busybox compiled with uclibc. This may be something unsupported, or there may be a patch fix. I'll check it out. I'll try with musl libc and musl-gcc as well.

I have another approach, I try aboriginal cross compiler toolchain in archLinux it produced small binary, but I can't compile busybox for android. For Linux it work. Maybe need bionic lib?

ndrancs said:
I have another approach, I try aboriginal cross compiler toolchain in archLinux it produced small binary, but I can't compile busybox for android. For Linux it work. Maybe need bionic lib?
Click to expand...
Click to collapse
Sounds interesting. I honestly haven't given this a try yet, but I'm very interested in taking a look at it. At this point I'm pretty much addicted to making the smallest binaries I can and testing out different toolchains. I'll give it a good search on duckduckgo, and if you have any insightful links that would be great.
Edit: Alright cool I found the source for Aboriginal Linux at landley.net/aboriginal and am building on Android now. I'm also trying this on my x86_64 laptop so that I can compare the differences like I have with glibc, uclibc, musl, klibc binary builds in a native environment and a cross-compile environment.
I see from my laptop's build that a busybox was generated, but it was dynamic and has a libc.so.6 dependency. @ndrancs : this might be what you were talking about. Did you try compiling static? Also see if "make allnoconfig && make clean && make LDFLAGS=-static" works for compiling busybox with Aboriginal Linux.

7175 said:
Edit: Alright cool I found the source for Aboriginal Linux at landley.net/aboriginal and am building on Android now. I'm also trying this on my x86_64 laptop so that I can compare the differences like I have with glibc, uclibc, musl, klibc binary builds in a native environment and a cross-compile environment.
I see from my laptop's build that a busybox was generated, but it was dynamic and has a libc.so.6 dependency. @ndrancs : this might be what you were talking about. Did you try compiling static? Also see if "make allnoconfig && make clean && make LDFLAGS=-static" works for compiling busybox with Aboriginal Linux.
Click to expand...
Click to collapse
I preferred to use uclibc I think it easy to setup and produced small binary.. Aboriginal cross-compiler use uclibc as default. Btw I don't use cmd : LDFLAGS=-static instead I set it in .config.. Maybe I try this later..

ndrancs said:
I preferred to use uclibc mk it easy to setup and produced small binary.. Aboriginal cross-compiler use uclibc as default. Btw I don't use cmd : LDFLAGS=-static instead I set it in .config.. Maybe I try this later..
Click to expand...
Click to collapse
Ok yeah I like how aboriginal set up with uclibc, and it has scripts for each build stage, so you can stop at the toolchain. I'll be interested to see their future releases with the musl libc as well.
Also for anyone interested, I figured out how to run dynamic binaries in android:
- make the directories "/lib" and "/lib/arm-linux-gnueabihf"
Code:
mkdir -p /lib/arm-linux-gnueabihf
- copy the linker "ld-linux-armhf.so.3" to "/lib"
- find a specific binary's dependencies: e.g. for dynamic mksh do:
Code:
strings mksh | grep \\.so
- copy the listed libs to "/lib/arm-linux-gnueabihf": e.g. for mksh that would be libc.so.6. The libs/linker you copy over will come from the mounted ubuntu/debian/... image you have mounted like with "Complete Linux Installer".
- adjust your LD_LIBRARY_PATH:
Code:
LD_LIBRARY_PATH=/lib:/lib/arm-linux-gnueabihf:$LD_LIBRARY_PATH

Any plan to update the busybox to current version. Thanks.

@7175 can you update flashable zip to 1.23.0 stable ?

@ndrancs @exodius48 : Thanks for notifying me guys, I needed to get around to updating to 1.23.0 stable. I updated the original post with no-edify installers for busybox 1.23.0 stable. There's a non-SELinux uclibc compiled version and a full 386-applet SELinux glibc compiled version. They're included in this post too for ease.

7175 said:
@ndrancs @exodius48 : Thanks for notifying me guys, I needed to get around to updating to 1.23.0 stable. I updated the original post with no-edify installers for busybox 1.23.0 stable. There's a non-SELinux uclibc compiled version and a full 386-applet SELinux glibc compiled version. They're included in this post too for ease.
Click to expand...
Click to collapse
Great..been waiting for this release.. :good:
Btw, can i use busybox_full_selinux.zip on android 4.2.2 MIUI rom?

exodius48 said:
Great..been waiting for this release.. :good:
Btw, can i use busybox_full_selinux.zip on android 4.2.2 MIUI rom?
Click to expand...
Click to collapse
Yeah that should work just fine. I'm pretty sure any SELinux tools or applet flags should work since 4.2 introduced SELinux to its filesystem. Let me know if there are any issues.

7175 said:
Yeah that should work just fine. I'm pretty sure any SELinux tools or applet flags should work since 4.2 introduced SELinux to its filesystem. Let me know if there are any issues.
Click to expand...
Click to collapse
Great release..busybox_full_selinux.zip working fine so far on MIUI rom V5 android 4.2.2. :victory:

7175 said:
@ndrancs @exodius48 : Thanks for notifying me guys, I needed to get around to updating to 1.23.0 stable. I updated the original post with no-edify installers for busybox 1.23.0 stable. There's a non-SELinux uclibc compiled version and a full 386-applet SELinux glibc compiled version. They're included in this post too for ease.
Click to expand...
Click to collapse
Hey @7175
Great guide. I am able to compile just fine on my device using your guide. However, is there any way to compile the selinux applets support using a Linux PC (or NDK)? I am not able to find a selinux supported toolchain. May be you can help.