Database Info

[GUIDE] Kernel Build Guide

This is a basic kernel build guide, to help those that keep coming into IRC asking for help.
If you follow this, you should have a working kernel that will work with ext4 or RFS.
NOTE::I wrote this at 1am, when Im kinda falling asleep, so there might be errors. Please let me know, and Ill fix it up
What you need:
A Linux system, with about 2gb free (This is written for 32bit systems, 64bit will need a compatible toolchain)
Development libraries
The kernel source
An ARM toolchain
An initramfs from an existing kernel
Downloads
EB13 source - search for D700 under mobile phone
Code Sourcery 2009q3 ARM toolchain
Daemon's Ext4/RFS Initramfs
Steps:
Set up the build environment
The first thing you need to do is set up the build environment.
For Ubuntu or other Debian-based system, you need the build-essential package and libncurses.
You can install this by running this command:
Code:
sudo apt-get install build-essential libncurses5-dev
If your not running Ubuntu/Debian, the packages are gcc, g++, libc-dev, and make.
Next, your going to need a directory for all your kernel build. You can't have spaces in any folder name. Im going to use /home/tortel/build/ for the rest of this guide.
Now you need to extract the kernel source to your directory. In the Samsung source file, its SPH-D700_kernel.tar
Next, extract the ARM toolchain into your build directory.
Last, extract your initramfs into the directory.
After this, your build directory should contain 4 things:
Code:
arm-2009q3/
eb13init/
Kernel/
build.sh
For this guide, Im not going to use the build.sh. If you want to, you'll need to configure that yourself.
Configure your kernel
Now the fun begins.
You need to edit the Kernel/Makefile file, and change line 184 to match where your toolchain is. My line 184:
Code:
CROSS_COMPILE ?= /home/tortel/build/arm2009q3/bin/arm-none-linux-gnueabi-
Now you need to configure the kernel.
The kernel configuration file is called .config, which is hidden by default (Ctrl+H shows hidden files in GNOME). You need to set lines 81-91, so it uses your initramfs. My lines:
Code:
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE="/home/tortel/kernel/eb13init/"
CONFIG_INITRAMFS_ROOT_UID=0
CONFIG_INITRAMFS_ROOT_GID=0
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
# CONFIG_INITRAMFS_COMPRESSION_NONE is not set
CONFIG_INITRAMFS_COMPRESSION_GZIP=y
# CONFIG_INITRAMFS_COMPRESSION_BZIP2 is not set
# CONFIG_INITRAMFS_COMPRESSION_LZMA is not set
This will build your initramfs into the kernel image.
Now, your set to build.
Build the kernel
This is the easy part. Open a terminal, cd to build/Kernel and run make.
The commands:
Code:
cd build/Kernel/
make
If you have more than one core, you should add -j {1+ # of CPU cores} to make so it finishes faster. It only takes about 5min on my 2ghz C2D laptop.
Make it flashable!
Congrats, its built!
The file you want is Kernel/arch/arm/boot/zImage in your kernel source directory. After make finishes, run:
Code:
cp arch/arm/boot/zImage ..
This copies your kernel image file into your main build directory. Now, you need to put it into a flashable zip. To make it easy, here. Just throw the zImage into the root of that zip, put it on your SD card, and flash it.

Might want later
Advanced config stuff to come

Might want this too

Thank you for this, I'm glad too see this and others who have contributed great guides like this.
sent from "The Other Woman"

I was one of those people who kept bothering lol. I learned how to compile a kernal in two day though, now im on my way making a OC/UV Voodoo color kernal (coming soon) :]
Thanks alot

globalninja said:
I was one of those people who kept bothering lol. I learned how to compile a kernal in two day though, now im on my way making a OC/UV Voodoo color kernal (coming soon) :]
Thanks alot
Click to expand...
Click to collapse
what should I look into for making the changes necessary for an overclocking kernel?

Thanks for this helpful thread, and I had a question previously, that someone pm'ed me for. OP, please remove this reply if you feel inclined to. I had a question about the initramfs because my computer was having problems loading the page for the link.

Hey guys, kindof a noob to this.
How can I get an initramfs for DI18? Ideally, I could just use the one in the stock ROM unmodified, but I have no idea how to obtain it.
I figured it out! Used the scripts here to extract it from zImage:
http://forum.xda-developers.com/showthread.php?t=901152

[Tool]File(1) command for Android (ver 5.11)

After wandering around looking for an android build of the file command, I finally went ahead and compiled it myself.
The attached package contains the file binary (v5.11), the libmagic.so shared library, the associated magic files and the build specific files that I needed (over and above the source) to build it. The paths of the files are relative to '/'.
Refs:
source: file website (www_darwinsys_com/file/)
compiling: Native Vim on Android (gdr_geekhood_net/gdrwpl/vim-android.php)
Note: Add the location of libmagic.so to LD_LIBRARY_PATH to get rid of the "CANNOT LINK EXECUTABLE" error. (I use /data/local/lib)

Compiled a dynamically linked library and associated binary instead of the monolithic binary built earlier.

samveen said:
After wandering around looking for an android build of the file command, I finally went ahead and compiled it myself.
The attached package contains the file binary (v5.11), the libmagic.so shared library, the associated magic files and the build specific files that I needed (over and above the source) to build it. The paths of the files are relative to '/'.
Refs:
source: file website (www_darwinsys_com/file/)
compiling: Native Vim on Android (gdr_geekhood_net/gdrwpl/vim-android.php)
Note: Add the location of libmagic.so to LD_LIBRARY_PATH to get rid of the "CANNOT LINK EXECUTABLE" error. (I use /data/local/lib)
Click to expand...
Click to collapse
I followed your instructions and it works perfectly.
Code:
$ echo $LD_LIBRARY_PATH
/data/local/lib:/data/local/lib:/data/local/lib:/system/lib:/vendor/lib
$ file
Usage: file [-bchikLlNnprsvz0] [--apple] [--mime-encoding] [--mime-type]
[-e testname] [-F separator] [-f namefile] [-m magicfiles] file ...
file -C [-m magicfiles]
file [--help]
$ getprop | grep device
[ro.cm.device]: [jordan]
[ro.product.device]: [umts_jordan]
I have been looking for this for a while. Man!! This post needs some serious attention. Great work.
Sent from my MB525 usiIng XDA

@samveen Any chance you could compile it with libmagic statically linked again to make it portable? Thanks for your time.

osm0sis said:
@samveen Any chance you could compile it with libmagic statically linked again to make it portable? Thanks for your time.
Click to expand...
Click to collapse
@osm0sis Sure. I'll add in a statically linked binary, but it'll take a little time (2-3 days).

samveen said:
@osm0sis Sure. I'll add in a statically linked binary, but it'll take a little time (2-3 days).
Click to expand...
Click to collapse
I was just looking for this. Great tool.
Thanks for your work.

samveen said:
@osm0sis Sure. I'll add in a statically linked binary, but it'll take a little time (2-3 days).
Click to expand...
Click to collapse
Any luck? I think it only needs libmagic statically linked to be portable; the other NDK/Bionic stuff can probably stay dynamic to keep the filesize down. I see file 5.17 source is available now too if you wanted to update your builds to the latest.

osm0sis said:
Any luck? I think it only needs libmagic statically linked to be portable; the other NDK/Bionic stuff can probably stay dynamic to keep the filesize down. I see file 5.17 source is available now too if you wanted to update your builds to the latest.
Click to expand...
Click to collapse
If you want only the system file type, the workaround i use is this (eval $(/sbin/blkid /dev/block/mmcblk0p11 | /sbin/awk ' { print $3 } '); /sbin/busybox echo $TYPE)
In a script just add a var like this
TYPE=$(eval $(/sbin/blkid /dev/block/mmcblk0p11 | /sbin/awk ' { print $3 } '); /sbin/busybox echo $TYPE) and you will have the FS type in $TYPE.
Phone: Samsung Galaxy SII - GT-I9100
Kernel: Dorimanx kernel v10.43v99-mv3
1st ROM : SlimSaber 4.4.2 Maliv3 by fusionjack build of 20140320 (Online)
2nd ROM :
MODEM: UHMS1
MODS: Partition Fix, Hue Blue v4.4 by Kroz :good:

Computoncio said:
If you want only the system file type, the workaround i use is this (eval $(/sbin/blkid /dev/block/mmcblk0p11 | /sbin/awk ' { print $3 } '); /sbin/busybox echo $TYPE)
In a script just add a var like this
TYPE=$(eval $(/sbin/blkid /dev/block/mmcblk0p11 | /sbin/awk ' { print $3 } '); /sbin/busybox echo $TYPE) and you will have the FS type in $TYPE.
Click to expand...
Click to collapse
Interesting but nope; I need file to determine a filetype. Compressed archives via a script, to be specific.

@osm0sis I got the code compiled (both with a dynamic libmagic.so, and libmagic compiled into file) but I need a little more time to compile the magic data (it requires running file on the target device to compile the magic data into a binary datafile). Once I have that figured out (either by compiling it or by creating a script to allow the users to do it themselves, I'll post it all in one go (need one more day).

samveen said:
@osm0sis I got the code compiled (both with a dynamic libmagic.so, and libmagic compiled into file) but I need a little more time to compile the magic data (it requires running file on the target device to compile the magic data into a binary datafile). Once I have that figured out (either by compiling it or by creating a script to allow the users to do it themselves, I'll post it all in one go (need one more day).
Click to expand...
Click to collapse
Awesome! Thanks so much. The magic file isn't terribly important since you can grab one a lot of places and specify it with the -m parameter; keeps the file size down too if you were thinking of including it in the binary somehow!

Alright. I just wanted to thank you again for your work but I managed to make a static native compile (~440kb) of file-5.17 using my N7 and a kbox build environment I set up. It's available in my ARM Android Image Kitchen linked in my sig. Cheers!

osm0sis said:
Alright. I just wanted to thank you again for your work but I managed to make a static native compile (~440kb) of file-5.17 using my N7 and a kbox build environment I set up. It's available in my ARM Android Image Kitchen linked in my sig. Cheers!
Click to expand...
Click to collapse
@osm0sis - Can you please share how did you static compile this? I have setup the Ubuntu 13.10 image on Android using Complete Linux Installer (with all the necessary build tools setup).
I am able to compile this, but it's linking the shared library "libmagic.la".

Need to pass --static to ./configure and have a cross compiler set up, for ARMv7.

ericlnu said:
Need to pass --static to ./configure and have a cross compiler set up, for ARMv7.
Click to expand...
Click to collapse
I don't think that's a valid option.
Code:
$ ./configure --static
configure: error: unrecognized option: `--static'
Try `./configure --help' for more information

amit.bagaria said:
I don't think that's a valid option.
Code:
$ ./configure --static
configure: error: unrecognized option: `--static'
Try `./configure --help' for more information
Click to expand...
Click to collapse
Nope, seems not ^_^. Sorry, it usually is with autoconf derived configure scripts.

amit.bagaria said:
@osm0sis - Can you please share how did you static compile this? I have setup the Ubuntu 13.10 image on Android using Complete Linux Installer (with all the necessary build tools setup).
I am able to compile this, but it's linking the shared library "libmagic.la".
Click to expand...
Click to collapse
Haven't tried it using a full Linux image on Android, just made it native compile in KBox2 on my N7'13 for awhile by editing in the gettext functions it was complaining about, and have since moved on to cross-compiling on my Windows 7 desktop in Cygwin using the NDK. There were still a few tricks to getting it to be a true static compile. It's all linked in my post here:
http://forum.xda-developers.com/showthread.php?p=54510825
My latest compile included in my AIK-mobile package is file 5.20. :good:

I don't suppose you could do a quick compile with the -fPIE flag? Trying to use this on 5.0 + and don't want to patch my linker.
Tried to compile it myself (it sort-of worked, file --version is operational) but when I try to actually use it for something my terminal borks out and switches to some weird font after about 20 warnings. (configure also seems to be using the -rpath flag apparently, linker complains about it) hmm
EDIT: I've got it working after some trial and error. If anyone requests I'll post it
EDIT 2: android 6.0's linker seems less forgiving than 5.0/5.1, need to compile without -rpath (so no just using ./configure) will keep posted

Can you please post the file binary you built with -pie ?

[CLOSED][Guide] How-to Automate Your ROM Build Process Using Jenkins - Setup Nightlies

HOW TO AUTOMATE YOUR ROM BUILDING PROCESS
... using
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Special thanks to @chasmodo for all his support, as well as his wonderful How-To build CM thread located HERE.
I. Donations and Support...
Feel free to post any question here, as well as provide your own scripts. As far as I'm concerned, there are no stupid questions; you are, however, required to search before posting.
Please do not donate to me, donate to the Testicular Cancer Society (Click Here).
II. Introduction...
This guide will direct you on how to setup your android build environment to automate your builds using Jenkins and various shell/bash scripts. As this was initially drafted, it was targeted towards CyanogenMod 10.2, but I am quite confident it can easily be easily modified to work with anything you choose, including LP builds.
This can be a useful process if you want to setup nightly/weekly builds, or if you just want to have a easy-to-use web interface to run your builds without entering the shell.
This guide assumes you have an active build environment (preferably Ubuntu-based), and general linux knowledge.
III. Initializing your build environment...
You will first have to have your build environment setup, and you should already be able to successfully build CM, AOSP, AOKP, or other variants, before moving forward. Furthermore, you are expected to have simple scripting knowledge, or you will not be able to do this. If you need assistance with setting up an android build environment (which is not the focus of this thread), you can get started by Clicking Here, or by looking at @chasmodo's thread by Clicking Here.
It is important to note that your Jenkins home directory is at "/var/lib/jenkins" (not /home/jenkins). As such, to get your build directory in the proper location, you will login normally to your shell, and "su jenkins" (you may have to use su policy, if you don't know how to do this, you may just want to login as root). Do Not login to the system-jenkins-user shell by changing the jenkins user password (I am referring to the System User named jenkins, not your jenkins web-interface). Once logged in as jenkins, do a "cd ~", create your directory (e.g., "cm10.2"), and repo init there (e.g., in /var/lib/jenkins/cm10.2 directory).
IV. Installing Jenkins and Apache Daemons...
Once your build environment has been initialized, you will have to get Jenkins/Hudson setup and running. You can read more about Jenkins software by Clicking Here.
Further, you will need to install and setup apache web server in order to access the Jenkins web interface. Click Here for more information on apache.
The jenkins install process can easily be setup by following the guide Located Here.
After you have successfully installed Jenkins and apache, you may proceed to setting up your automated builds.
V. Setting Up Jenkins Security...
Be sure to setup a Jenkins user account, or a remote user can get your setup and screw up your machine.
Follow the article here:
https://wiki.jenkins-ci.org/display/JENKINS/Standard+Security+Setup
Continued in Post #2

V. Configuring your first build...
.... A. Creating the new build...
Once you have setup your jenkins server (by going to http://localhost:8080, or port 80 if you set it that way), log in with the username/password you configured.
After you have logged in, click "New Job" in the left toolbar.
Screenshot:
Then, enter the "Job name". Please Note to NOT use spaces or slashes in the name, as it causes build issues. If you need a space, use an underscore ("_"). Check the radio-box "Build a free-style software project", and hit "OK".
Screenshot:
Now, continue to the procedure listed directly below (i.e., in section "B", "How to have Jenkins call your local scripts...")
.... B. How to have Jenkins call your local scripts...
As opposed to attempting to have Jenkins' propriety interface try to run your build process directly, it is best to have it call out for your individualized shell/bash scripts.
To do so, under the "Execute Shell" Field in the "Build" section, you will call out your script by entering the following code:
Code:
#!/bin/bash
cd ~/[COLOR="Red"]FOLDER[/COLOR]/
source [COLOR="Red"]SCRIPT_NAME[/COLOR].sh
Screenshot:
Once your code has been input, the next step is to create your script in your build directory. In this instance, the script is called "buildscriptclobber.sh" and the directory is "~/cm10.2" (~ is home). It is important to remember that your home directory is the Jenkins user home directory, which gets installed at /var/lib/jenkins (NOT /home/jenkins).
As above mentioned, you will login normally to your shell, and "su jenkins". Once logged in as jenkins, do a "cd ~/FOLDER" (e.g., "~/cm10.2"). Then, use your favorite text editor (e.g., pico, installed by sudo apt-get install pico), and enter your script.
A good script I wrote will make clobber and repo sync every time. It is as follows:
Code:
#!/bin/bash
cd ~/[COLOR="red"]FOLDER[/COLOR]/
make clobber
cd ~/[COLOR="red"]FOLDER[/COLOR]/
export PATH=~/bin:$PATH
export USE_CCACHE=1
repo sync
vendor/cm/get-prebuilts
source build/envsetup.sh && time brunch [COLOR="red"]DEVICE[/COLOR]
You may notice above that I call out the folder ("cd"), and set the "PATH" multiple times, especially when its already set in your ".bashrc". This is because the script will give errors if you do not do so, because Jenkins won't consistently read your ".bashrc" or other external configurations. It is important to always do this type of thing when shell scripting with Jenkins. The "time" command may be omitted, because Jenkins log will show the total build time throughout, so its partially redundant, but I like it in there.
Once the script has been setup, do not forget to "chmod a+x" or it will not run.
VI. Automating your first build...
WIP SETTING UP H H H TRIGGER
VII. Configuring and automating additional builds...
Follow the same steps as above, just be sure the Cron Triggers are set to not overlap. For Example, if you have setup H H H(0-5) for one, and you know your builds may take up to two hours, then setup H H H(2-7) for the next.
Continued in Post #3

VIII. Scripting your builds in general...
.... A. Use individual shell scripts for each process...
The beauty of having a Jenkins environment is your scripting capabilities. There are many possibilities, and many plugins available within the Jenkins interface. I, however, prefer just executing Shell Scripts, since you will have more control over what you are doing via bash scripting. If you are a bash scripting novice, you can Click Here to view a nice guide.
When scripting, realize that every change you make must be changed in your other scripts. For example, if I use the change filename script below, obviously I will need to modify my upload script to reflect the changed filename. I think of it in a way that "one thing breaks another". This is why its great to have an independent script for every process you do, so that only one file requires modification if changes are made.
.... B. How to set your scripts to run after the initial compile process has completed...
In order to run ANY OTHER PROCESS after the initial build script is complete, you must run another script after build. To do this, you will use the "Add build step" feature to "Execute Shell", located in the "Build" section. As such, for ANY new script you want to run after the build script, you must add another build step. Consequently, every single script below requires a new build script.
Screenshot:
Like above when you setup your initial build script, you will call a script out. This can be done the same way, using:
Code:
#!/bin/bash
cd ~/[COLOR="Red"]FOLDER[/COLOR]/
source [COLOR="Red"]SCRIPT_NAME[/COLOR].sh
Screenshot:
Continued in Post #4

IX. Specific post-build scripts...
.... A. Automating Filename Changes, and/or Moving The Build to a Different Directory Automatically...
This essentially moves the build .zip to the same folder, but renames it to the name you choose. Thus, when it uploads, you don't have to manually rename the ROM Firmware file.
Code:
#!/bin/bash
cd /var/lib/jenkins/[COLOR="red"]FOLDER[/COLOR]/out/target/product/[COLOR="red"]DEVICE[/COLOR]/
mv cm-10*.zip [COLOR="red"]BUILD-NAME[/COLOR].zip
If you want to add the date to the filename, you can add the following string within the filename:
Code:
$(date +%Y%m%d)
You are free to modify the ordering of %Y%m%d as you see fit.
Therefore, if I wanted my build to say "SGT7-DATE-n7000-NIGHTLY.zip", I would use the following as the filename:
Code:
SGT7-[COLOR="Red"]$(date +%Y%m%d)[/COLOR]-n7000-NIGHTLY.zip
As such, you're final code would be:
Code:
#!/bin/bash
cd /var/lib/jenkins/[COLOR="red"]FOLDER[/COLOR]/out/target/product/[COLOR="red"]DEVICE[/COLOR]/
mv cm-10*.zip [COLOR="red"]BUILD-NAME[/COLOR]-$(date +%Y%m%d).zip
Alternatively, if you want to move to a local directory, such as your public ftp folder, you can use the following code instead, adding the date string above if you want:
Code:
#!/bin/bash
cd /var/lib/jenkins/[COLOR="red"]FOLDER[/COLOR]/out/target/product/[COLOR="red"]DEVICE[/COLOR]/
mv cm-10*.zip /[COLOR="red"]PATH_TO_PUB_FTP_FOLDER[/COLOR]/[COLOR="red"]BUILD-NAME[/COLOR].zip
.... B. Automating uploads to "goo.im" or any other FTP/SFTP server...
First, you will have to install yafc in ubuntu. This is done via "sudo apt-get install yafc".
Then, create a script using the following code:
Code:
#!/bin/bash
cd /var/lib/jenkins/[COLOR="Red"]FOLDER[/COLOR]/out/target/product/[COLOR="red"]DEVICE[/COLOR]/
yafc <<**
open ftp://[COLOR="Red"]USER[/COLOR]:[COLOR="red"]PASS[/COLOR]@upload.goo.im:21/
cd public_html/[COLOR="red"]FOLDER[/COLOR]
put -f -r /var/lib/jenkins/[COLOR="red"]FOLDER[/COLOR]/out/target/product/[COLOR="red"]DEVICE[/COLOR]/cm-10*.zip
close
.... C. Automating build.prop modifications...
MARKER
.... D. Automating changelogs...
MARKER
Continued in Post #5

X. Troubleshooting...
Here are some common solutions to your issues.
Reboot your system ("reboot -n") and try again. You may need a daily, or even weekly reboot depending on your RAM, software causing memory leaks, etc. You can setup a CRONJOB (Click Here for more info) to do a reboot daily. If you choose to setup a job in the crontab, be ensure it doesn't conflict with the timing of your Jenkins cron builds.
If you keep getting java out of memory errors, you probably need more RAM.
Other errors may be caused by your CCACHE being set to low. If you continue having issues caused by ccache, even when its high, you may want to install a global ccache in /usr/bin or the like.
Did you remember to set permissions on your scripts ("chmod a+x")
need to reset your username? See this post: http://forum.xda-developers.com/showthread.php?p=51248853
having ota jenkins.zip NOT FOUND ERROR (or no such file or directory)? See this post: http://forum.xda-developers.com/showthread.php?p=48454597; see also this post: http://forum.xda-developers.com/showthread.php?p=55954963

Subscribed!
Sent from my Nexus 4 using Tapatalk 4

MikeyCriggz said:
Subscribed!
Sent from my Nexus 4 using Tapatalk 4
Click to expand...
Click to collapse
I have lots of great scripts and stuff I will be posting! just have to juggle my real job with throwing in some time every day to build this up...

Well done, mate. This is something that hasn't been covered so far methinks. :good:

chasmodo said:
Well done, mate. This is something that hasn't been covered so far methinks. :good:
Click to expand...
Click to collapse
Thanks buddy. I wanted to PM you a question, but you don't accept PMs...
I'll G+ you later on.

I have become your slave
Very good
#KitKat - Having one right now.

This is what you call, "Taking it to next level"
:thumbup:
Sent from Asylum HQ

Absolutely awesome! I'm subscribed this fred, man!
Sent from my N7000 using cm10.2-Asyluminator by chasmodo

with my new root - 16GB RAM - everything works like it should thread subscribed!
is there any way to make the build faster? yes, using ccache.. but are there some more ways to get everything out of the box ?

dschense said:
with my new root - 16GB RAM - everything works like it should thread subscribed!
is there any way to make the build faster? yes, using ccache.. but are there some more ways to get everything out of the box ?
Click to expand...
Click to collapse
What type of hard drive do you have?
How long do your Jenkins builds take to run?
The first run always fails (flaw in Jenkins) I need to add that to the guide.
The second run always takes extra long
The third run should be quicker...
My builds take about 30-45 minutes including repo sync everything
If you have a Solid State hard drive you can have builds going much quicker....

Okay.. I have tried many ways of setting up environment with Jenkins for compiling for paranoid android. But I had no luck at all..
I am running everything on a Ubuntu 12.04 headless server.
I did everything for setting up environment with installing nessesary apps and stuff with user who has sudo rights.
Than I installed Jenkins and logged in to the Jenkins user with sudo su und from root to Jenkins with su Jenkins without pw.
Home of Jenkins is var/lib/jenkins.
I downloaded android sdk to Jenkins home and installed sdk over x11 ssh connection in var/lib/jenkins/android-sdk.
Installed java jdk 6u45 to opt/java/64/java installation
Jdk path is set in Jenkins settings for default.
Logged in to Jenkins Web. Setup security with new user, created freestyle project und created workings pace. Repo init to this directory and sync everything.
In the freestyle project shell with Repo sync command and. /rom-build.sh i9300 clean
#!/bin/bash
cd ~/
export PATH=~/bin:$PATH
export USE_CCACHE=1
cd ~/workspace/pa43-legacy-modded/
repo sync -j16
./rom-build.sh i9300 clean
Build is syncing and building.. But I get Damn error all the time..
Do I have to change permissions or is the setup of my environment not the right way?
Hope anybody can help me...
Kaaaaaaaaatsching... Tapatalked!

@dschense
I'm confused as to what your Jenkins shell command is...
Also are you calling another script within a script? Or do you have two separate Jenkins execute shells?
Does that question make sense?
Also if you went into that particular build directory logged in via su jenkins, can you compile it manually without error?
your setup should look like this:
is that what you have?
You should Not be calling another script from inside your jenkins script. that will cause issues. you need to do them incrementally in chronological order and do a new execute shell for each script.
Also the "./" command won't work inside a script, or in a jenkins command
so your "./rom-build.sh i9300 clean" should be a whole new execute shell
and instead of "./" you need to use the "source" command like I have above

Also can you post the link of your error? I know you PM'd Me maybe u can paste that here
Sent from my GT-N7000 using Tapatalk 2

I think my biggest problem is how I can do a right shell script for building. If I build ParanoidAndroid from source manually I use the command. /rom-build.sh i9300... But I don't know exactly what commands to use for just using the shell window in Jenkins without using the script by ParanoidAndroid.. Any help?
Kaaaaaaaaatsching.... Tapatalked!

dschense said:
I think my biggest problem is how I can do a right shell script for building. If I build ParanoidAndroid from source manually I use the command. /rom-build.sh i9300... But I don't know exactly what commands to use for just using the shell window in Jenkins without using the script by ParanoidAndroid.. Any help?
Kaaaaaaaaatsching.... Tapatalked!
Click to expand...
Click to collapse
I'm trying to help but you don't respond to my questions.
Can you (1) post the PA build script and (2) post exactly what is in your execute shell
Sent from my GT-N7000 using Tapatalk 2

I am at work atm.. I will post everything when I have a little more time.. Next week I have much free time to use for setting up everything at all.. But maybe I can post some things like the build script tomorrow after work.
BTW. Big big the for helping me!
EDIT:
This is the rom-build.sj script:
http://pastebin.com/UzFFKqYM
EDIT2:
Now everything is clean.. i fetched my repo stuff and finished syncing.
Job is also clean.. any help for how to do next ?
Kaaaaaaaaatsching... Tapatalked!

[Scripting][ARM][Installer][Port] Python (static) 2.7.9 and 3.4.2

(Check out https://www.python.org for information on what python is.)
Yes you read that correctly, static python for Android! It took a while to figure out how to get this compiled, but I finally did it. I present you with an installer and a little bit of testing. The second post contains information on compilation and the sources. Btw, any cool python scripts are welcomed!
INSTALLATION
Recovery flash installers are attached that will install python 2.7.9 and/or 3.4.2 to /system/pythonX.X.X. You can install both if you want. Installation size is about 41MB, 43MB, and 49MB respectively for python 2.7.8, python 2.7.9, and python 3.4.2.
After installation, the python director(ies) in /system will contain the static python binary, some scripts, and a bunch of modules and documents. Separate scripted executables will be installed to /system/bin/python or /system/bin/python3 depending on which one is installed. These basically just set the PYTHONHOME environment variable and execute the python binary.
TESTING
When you boot up to Android after installation, you should be able to just open up a terminal and fire off some python commands.
Test using python 2.7.9:
Code:
python -c 'print "Hello World!"'
Test using python 3.4.2:
Code:
python3 -c 'print("Hello World!")'
You can also write scripts shelled with python:
Code:
#!/system/bin/python
print "Hello World!"
Make sure to set them as executable with "chmod +x".
Adjust your screen brightness via /sys (root required, tested on Galaxy Nexus):
Code:
python -c 'f=open("/sys/devices/omapdss/display0/backlight/s6e8aa0/brightness","w"); f.write("40"); f.close()'
A cool script I made to tweak file system I/O like rq_affinity, rotational, etc for I/O blocks:
Code:
#!/system/bin/python
import os,re,sys
list=[]
# find all directories containing rq_affinity
for roots, dirs, files in os.walk('/sys'):
for file in files:
match=re.search(r'\S+/rq_affinity',os.path.join(roots,file))
if match:
list.append(match.group().replace('rq_affinity',''))
# write specific values to files in each directory found before
for dir in list:
for name in 'rq_affinity', 'rotational', 'read_ahead_kb', 'nr_requests', 'iostats', 'nomerges', 'add_random':
try:
f=open(dir+name,'w')
if name is 'rq_affinity': f.write('1')
elif name is 'read_ahead_kb': f.write('512')
elif name is 'nr_requests': f.write('512')
else: f.write('0')
f.close()
except IOError:
sys.stderr.write('Problem writing to ' + dir+name + '\n')
Or execute "python" or "python3" without any parameters to open up the interpreter and go from there. When you start for example "python" (python 2.7.8), you should see something like:
Code:
Python 2.7.8 (default, Dec 2 2014, 05:15:18)
[GCC 4.9.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
***I haven't fully tested this yet, so let me know how it goes. I know basic commands work, but there are still some complexities in the compilation that may need to be figured out.***
(update) -- Install Extra Packages/Modules via "easy_install" -- (root required)
1) In a shell, remount /system read-writable with:
Code:
mount -o remount,rw /system
2) Change directory to /system/pythonX.X.X, i.e.,
Code:
cd /system/pythonX.X.X
3) Execute easy_install followed by a package name, i.e.,
Code:
./easy_install [i]package_name[/i]
4) Let it download and install! Then test it out.
5) Remount /system read-only with:
Code:
mount -o remount,ro /system
Also, to remove a package, execute:
Code:
./easy_install -m [i]package_name[/i]
rm -r ../lib/pythonX.X/site-packages/[i]package_name[/i]*.egg
*** Flashing the resolv.conf patch may be required to make an internet connection to download modules.
Also, a few things need to be done to get easy_install working:
1) For python 2.7.8 or 2.7.9 you need to link python2.7 to python like this:
Code:
ln -s /system/python2.7.9/bin/python2.7 /system/python2.7.9/bin/python
I'll make sure this is automatically done in future installs.
2) SSL certificates need to be added to "/etc/pki/tls/certs/" to use SSL with easy_install.
Code:
mkdir -p /etc/pki/tls/certs
curl http://curl.haxx.se/ca/cacert.pem -o /etc/pki/tls/certs/ca-bundle.crt
ISSUES
The python installations have the following modules compiled in with external dependencies such as libraries not included at this point:
Code:
--------------PYTHON 2.7.8-2.7.9----------------------
_hashlib _multiprocessing _ssl
_testcapi bz2 crypt
dbm
Code:
--------------PYTHON 3.4.2-----------------------------
_crypt _dbm _decimal
_hashlib _multiprocessing _ssl
_testcapi
I will look into this more.
As @bubbleguuum points out, name resolution with python 3.4.2's urllib wasn't working. I found a workaround by adding the line
Code:
options single-request-reopen
to /system/etc/resolv.conf. This still needs some testing but seemed to work for me. I've included a recovery flash zip to patch /system/etc/resolv.conf if the line is not there since this is a root operation.
UPDATES
* 2014-12-02 * Compiled in more modules for each installation. This makes the installation a bit bigger, but it's worth it. You get a more complete python! To get stuff like help functions and math and readline modules, you need the larger installations linked below under "DOWNLOADS".
* 2014-12-03 * Thanks @cybojenix for pointing out some incapatibility issues with the original edify installer I was using. I have now updated the installers to use SuperSU's great non-edify sh-script installer.
* 2014-12-05 * Some updates to the installers to include more modules, plus size reduction for python 3.4.2. Modules added to python 2.7.8 : _bsddb _ctypes, _ctypes_test, _hotshot, _json, _lsprof, _sqlite3, future_builtins, and ossaudiodev, easy_install, pip(broken for now). Modules added to python 3.4.2: _bz2, _ctypes, _ctypes_test, _gdbm, _json, _lsprof, _opcode, _sqlite3, _testbuffer, _testimportmultiple, ossaudiodev, xxlimited
* 2014-12-05 * Some big updates to the installer to include almost the rest of the modules that wouldn't compile. Also easy_install is working on my end with these new experimental builds. With a little effort, should be able to get pip working as well. I had to use some hacky flags to get it to compile, ignore unresolved-symbols and such, so it definitely needs some testing. Another thing I did was tweak the install script to write over previous installations from these installations. I recommend downloading the experimental builds if you read this. The worst thing that might happen is one of the new modules I've included won't work completely.
* 2014-12-14 * Added python 2.7.9. Trimmed the installation sizes down to the much more reasonable 41MB, 43MB, and 49MB respectively for python 2.7.8, python 2.7.9, and python 3.4.2.
* 2014-12-24 * Attached a recovery flash.zip to patch resolv.conf to try and fix a urllib name resolution error.
* 2014-12-26 * A couple updates: Tweaked python installations so that modules looks for "/system/bin/sh" instead of "/bin/sh" when needed. Also a few "#!" corrections for scripts in the python bin directory. This shouldn't break anything from what I've tested, only make Android python more capable, but I'll keep the old installations attached for now just in case since the modifications involved using "sed". The second update is nice. I've added a little section on installing extra modules with easy_install(included and working with all installations).

Compiling
The environment I compiled in was a debian "wheezy" image mounted as a loop device on Android. The reason I compile this way is so I have full access to the arm environment tools, gcc compiler, etc. without having to go through the trouble of using a cross-compiler (which doesn't work in this case from my testing with uClibc, or eglibc. Code sourcery's might work but I didn't feel like booting an x86 linux installation to try). You can recreate the same environment pretty easily by using either Complete Linux Installer or Linux Deploy apps. I use both.
To get a static python compiled, I first downloaded the following packages with APT:
Code:
[i]apt-get build-dep python python3[/i]
build-essential gcc make # building tools
binutils-gold # awesome gold linker
zlib1g-dev # zlib
libreadline-dev, libncurses5-dev # readline, curses
libbz2-dev # bz2
libsqlite3-dev # sqlite3
python-bsddb3, python3-bsddb3
libgdbm-dev # gdb
libssl-dev # ssl
python-tk, python3-tk # tkinter
libdb-dev # db
python-gdbm python-bsddb3
libffi-dev # _ctypes
tcl8.6-dev # tkinter
libx11-dev # tkinter
libmpdec-dev # decimal
Then I manually installed binutils-gold by unpacking the rpm for armv7h (this isn't available with APT yet). You can get it here though. (Update: this should be available in the debian repositories now.)
For static compilation, you need to make some changes to Modules/Setup from the python source directory. Specifically, you need to add
Code:
*static*
to the top of the file. Then you need to uncomment any commented modules you would like compiled in. When you compile and see a list of failed modules, search for each of these in Modules/Setup and uncomment them. For example, change
Code:
#math mathmodule.c _math.c
to
Code:
math mathmodule.c _math.c
. Then recompile. There may be some shared dependencies and packages as well you need to figure out to get them to compile. Take a look at the attached "Setup" files for more detail.
The actually compile steps look like this:
(1)
Code:
./configure --build=arm --prefix="$PWD"/out LDFLAGS="-static -static-libgcc -Wl,--unresolved-symbols=ignore-all -Wl,--export-dynamic" CPPFLAGS=-static CXXFLAGS=-static CFLAGS="-Os -static" LDFLAGS=-static LD=ld.gold
(2) Modify the Modules/Setup file.
(3)
Code:
make clean; make install
After compiling, there are some things you can do to make your installation in "./out" smaller. Strip the large binaries in "out/bin", i.e.
Code:
strip -s ./out/bin/python2.7
Also remove the python archive library, *.o's, etc. with:
Code:
make clean
This might seem kind of weird, but it works and cleans out unnecessary files from your installation.
**Make sure to backup your Modules/Setup file if you run "make distclean". I lost my 3.4.2 Modules/Setup file after doing this. :silly: Now I gotta recreate it, dammit.
**I should also mention that compile time is very fast given the amount of data generated. It takes about 10 minutes to get python and all it's modules compiled on my Galaxy Nexus.**
:EDITS:
: Updated some configure parameters. Removed unnecessary, extra "./configure". Also added some updates to the included "Setup" files.
: Hacky update to "./conifgure" to ignore unresolved symbols, export dynamic. This allows certain modules to be compiled which have shared dependencies.
: Update to trimming down the installation size after "make". Use another "make clean".
Downloads - Extract in the source directory
Setup (python 2.7.8)
Setup (python 3.4.2)
Setup (Experimental) (python 2.7.8)
Setup (Experimental) (python 3.4.2)

Something that would be interesting to look in to would be to have pip/easy_install running, so you can install extra non c packages onto the sdcard/data/wherever.
Would you also consider opening up the sources please? There is a project I'm interested in doing involving python on Android, however a lack of time meant I couldn't finish building it.
Give me a shout if you need anything. I hope to see even more come out of this thread
Edit:
I've just looked at the updater-script. The mount command won't work on the majority of devices. Can you turn the update-binary in to a shell script please? See the SuperSU updater for reference

cybojenix said:
Something that would be interesting to look in to would be to have pip/easy_install running, so you can install extra non c packages onto the sdcard/data/wherever.
Would you also consider opening up the sources please? There is a project I'm interested in doing involving python on Android, however a lack of time meant I couldn't finish building it.
Give me a shout if you need anything. I hope to see even more come out of this thread
Edit:
I've just looked at the updater-script. The mount command won't work on the majority of devices. Can you turn the update-binary in to a shell script please? See the SuperSU updater for reference
Click to expand...
Click to collapse
Thanks for your reply! I've changed the installer to use SuperSU's no-edify sh-scripted updater. I tested on my device, but let me know if there are any issues though. I kept it pretty basic.
I added some stuff on the source modifications and compilation to the 2nd post. The only file I've actually modified so far in the source is the generated Modules/Setup file, and I've included the one I used for python 2.7.8. Unfortunately, I over-cleaned the python 3.4.2 directory, which deleted my modified Modules/Setup there, so I'll have to add it later when I re-edit it.
That's a great idea on pip/easy_install. I did download the pip_installer and tried out installing on the static python I made. Got some errors though pointing to a few modules I haven't gotten compiled into the static python installation (listed in the OP near the bottom, specifically _ctypes). I'll see what I can do about that. The line for compiling the module might just need to be added to Modules/Setup. Hopefully that's the case.

@7175
Thank you very much for these binaries.
There is however a problem: name resolution (DNS) doesn't seem to work at all (with both python 2 and 3 downloads).
All attempts to use urllib.request.urlopen('http://somehost.com') fail with "<urlopen error [Errno -2] Name or service not known>" (running python as root but it doesn't matter, and a rooted Nexus 4 running 4.4.4 stock ROM):
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 153, in urlopen
return opener.open(url, data, timeout)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 455, in open
response = self._open(req, data)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 473, in _open
'_open', req)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 433, in _call_chain
result = func(*args)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 1202, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 1176, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>
Click to expand...
Click to collapse
it works if replacing somehost.com by its ip address.
To reproduce above stack trace with python 3:
import urllib.request
urllib.request.urlopen('http://www.google.com')
while this work:
urllib.request.urlopen('http://173.194.45.229')
nslookup and general name resolution work anywhere else:
[email protected]:/ # nslookup www.google.com
nslookup wwwgoogle.com
Server: 8.8.4.4
Address 1: 8.8.4.4 google-public-dns-b.google.com
Name: wwwgoogle.com
Address 1: 2a00:1450:4007:80c::1014 par03s15-in-x14.1e100.net
Address 2: 173.194.45.242 par03s15-in-f18.1e100.net
Address 3: 173.194.45.240 par03s15-in-f16.1e100.net
Address 4: 173.194.45.244 par03s15-in-f20.1e100.net
Address 5: 173.194.45.241 par03s15-in-f17.1e100.net
Address 6: 173.194.45.243 par03s15-in-f19.1e100.net
Click to expand...
Click to collapse
This issue has probably something to do with how python was compiled to do name resolution. I remember vaguely a similar issue
on Ubuntu 12.04, with a ffmpeg compile that would always fail to resolve hostnames in URLs. No sure what the exact fix was but it had something to do with the libc and its name resolution mechanism. Or maybe something ipv6 related ?

@bubbleguuum : Thanks for checking out that important issue.
I did a little research and it seems there is some issue with DNS name resolution related to ipv6. I recompiled with "--disable-ipv6" and that seemed to fix the issue. I also found that adding the line
Code:
options single-request-reopen
to /etc/resolv.conf did the trick as well. Apparently this tells the resolver to use a new socket for ipv6 resolution instead of same one as ipv4. It thereby reduces wait-time as well. Maybe check on your end and see if that works. Otherwise I'll upload ipv4-only installations since this issue at least needs to be side-stepped.
EDIT: Hmm, looks like re-compiling with "--disable-ipv6" didn't fix the problem. Will have to look more into this issue and hold off on uploading ipv4-only python3 since it didn't fix it. In the meantime, I've attached a recovery flash.zip to the OP to add the above mentioned line to /system/etc/resolv.conf if needed, which seems to fix the issue for me.

7175 said:
@bubbleguuum : Thanks for checking out that important issue.
I did a little research and it seems there is some issue with DNS name resolution related to ipv6. I recompiled with "--disable-ipv6" and that seemed to fix the issue. I also found that adding the line
Code:
options single-request-reopen
to /etc/resolv.conf did the trick as well. Apparently this tells the resolver to use a new socket for ipv6 resolution instead of same one as ipv4. It thereby reduces wait-time as well. Maybe check on your end and see if that works. Otherwise I'll upload ipv4-only installations since this issue at least needs to be side-stepped.
Click to expand...
Click to collapse
Thank you for looking into this so fast.
My intended usage is for running the python binary from a regular non-root app (which works great!), so editing resolv.conf is not an option.

I've been researching that DNS resolving issue and could not find a clear explanation on why it fails on Android and not on other systems, and why exactly the added line in resolv.conf fixes (or rather workaround) it.
All seems to point to socket.getaddrinfo() failing for an unknown reason.
Is Python compiled with HAVE_GETADDRINFO defined ? If that's the case it uses the libc definition of getaddrinfo, otherwise
it uses an emulation function found in getaddrinfo.c.
My guess is that it is the latter, and what is causing this issue on Android.
It's probable this issue has already been solved on Android since other Python binaries exists, but Googling around do not give
much answer.

I get this error when trying to install anything with easy_install (using /system/python3.4.2/bin/easy_install-3.4 because /system/python3.4.2/easy_install doesn't seem to exist like in the OP):
Code:
# ./easy_install-3.4 feedparser
Searching for feedparser
Reading https://pypi.python.org/simple/feedparser/
Download error on https://pypi.python.org/simple/feedparser/: [Errno -2] Name or service not known -- Some packages may not be found!
Couldn't find index page for 'feedparser' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Download error on https://pypi.python.org/simple/: [Errno -2] Name or service not known -- Some packages may not be found!
No local packages or download links found for feedparser
error: Could not find suitable distribution for Requirement.parse('feedparser')
i got curl and the certs installed in /etc/pki/tls/certs/ as per instrucions in the OP, any help?

Thoughts on PIL?

Hmm... new problems here. Android Lollipop have no /system/etc/resolv.conf by-default and urllib patch is no more works.
How it can be fixed?

http://qpython.net/index.php

Where I can find source code and related documents for project

Pratik Raj said:
Where I can find source code and related documents for project
Click to expand...
Click to collapse
I would like to know the same, as well as what license it is under... Also, I don't care about the network, but does it work on Oreo? Also, do you think I could customise the installer to install to a non-system location (/tmp) so that my flashable zip can use python?

hackintosh5 said:
I would like to know the same, as well as what license it is under... Also, I don't care about the network, but does it work on Oreo? Also, do you think I could customise the installer to install to a non-system location (/tmp) so that my flashable zip can use python?
Click to expand...
Click to collapse
Idk about OP, but I can surely post my source for python 2.7 on arm/arm64. I compiled it like a year ago, but it wasn't too bad, just required a bunch of patches and code for dlopen

I just want to say thank you so much.

[DEV][PROOF OF CONCEPT] APK-Patcher - Easily Mod APKs from Recovery [Flashable Zip]

APK-Patcher -- Flashable Zip Template for Modifying System APKs On-Device from Recovery
APK modification can be a tedious task. If you want to provide modified APKs you copy it to PC to decompile using Java applets like baksmali and apktool, make changes and recompile using another Java applet, smali (or apktool again). Then there's the pitfall of different recompiled APKs being required for the various different ROMs, etc., not to mention across Android versions. So, considering Android runs a version of Java runtime and the primary tools are Java-based I wondered why we couldn't do all of this on-device and on-the-fly so modifications could be applied via recovery, to whatever ROM APK was present, and take the constant update burden off of the modification creator.
I have taken these commandline Java applets, dexed them to allow them to run on-device, and found/built static ARM compiles of zip, zipalign and aapt to run on the various Android versions and devices. These binaries would need to be recompiled for other architecture support. The zip is smart and automated, using the APK name to run all the various parts involved for complicated patches and is extensible to any number of APKs in a single zip. The modification creator needs to only add the APK name(s) to a list, figure out how to make the changes universally (i.e. sed, awk, etc.) then add the commands to scripts of the same name, and add any whole-file additions/changes in subdirectories of the same name.
A proof-of-concept working script automating the long known Facebook Contacts Sync modifications to ContactsProvider.apk is included for reference.
My development work on my many projects comes out of my free time, so if you enjoy this project or anything else I've done on xda, please consider sponsoring my ongoing work using my GitHub Sponsors profile. For a one-time donation you can hit the donate link from my profile. Thank you for your support!
Source: https://github.com/osm0sis/APK-Patcher/
Status: No Longer Updated
Instructions
1a) Place any required script to alter the decompiled APK classes.dex smali files in /script as a -smali.sh file with the name of the APK
-b) Place any required/additional updated whole smali files in /patch under the name of the APK with "-smali" and the relative path within the classes.dex file (including com)
2a) Place any required script to alter the decoded APK resources.asrc files in /script as a -res.sh file with the name of the APK
-b) Place any required/additional updated whole resource files in /patch under the name of the APK with "-res" and the relative path within the zip/resources.asrc file (including res)
3) Modify the envvar.sh to add your banner, apklist, backup and cleanup options
4) Modify the extracmd.sh to add any additional commands to be performed at the end of the patching process that aren't patch-related (/data file changes, etc.)
5) zip -r9 UPDATE-APK-Patcher.zip * -x README UPDATE-APK-Patcher.zip
As a general rule, whole-file adding is best used only for including a file that doesn't already exist, and the more surgical script-work should be used to keep things more universal.
If supporting a recovery that forces zip signature verification (like Cyanogen Recovery) then you will need to also sign your zip using the method I describe here:
[DEV][TEMPLATE] Complete Shell Script Flashable Zip Replacement + Signing [SCRIPT]
Enjoy!
Questions, comments and feedback welcome.
Credits & Thanks: JesusFreke for bak/smali, iBotPeaches for apktool, Surge1223 for help working out how to build zipalign and aapt, and all authors of the included binaries and those who ported them over for their amazing work.
Disclaimer: Naturally, you take all the responsibility for what happens to your device when you start messing around with things.

Script/Patch Reference
Properties / Variables (envvar.sh)
Code:
banner="Facebook Contacts Sync Enabler";
apklist="ContactsProvider.apk";
apkbak=/data/media/APK-Backup;
backup=1;
cleanup=1;
banner is the name of your patch zip, usually suggestive of what it does, to be displayed at the beginning of the zip flash. You should include your name/handle here like "by osm0sis @ xda-developers" for credit purposes.
apklist is a string containing the list of APKs to be patched included in the patch zip, separated by spaces between the quotes. Each APK is automatically found recursively in /system, then copied to the working directory to be decompiled and acted on, then copied back to /system.
apkbak is the location to place backups of the untouched APKs in apklist if backup=1 is set.
backup=1 will store backups of the untouched APKs in the location specified in apkbak.
cleanup=0 will keep the zip from removing it's working directory in /tmp/apkpatcher or any of the files resulting from the smali process - this can be useful if trying to debug in adb shell whether the patches worked correctly. cleanup=1 is necessary on multi-APK patching zips to clean the baksmali classout directory for the next APK, so it's recommended each APK to be patched be tested on their own with cleanup=0 before combining into a single zip.

osm0sis said:
Properties / Variables (envvar.sh)
Code:
banner="Facebook Contacts Sync Enabler";
apklist="ContactsProvider.apk";
apkbak=/data/media/APK-Backup;
backup=1;
cleanup=1;
banner is the name of your patch zip, usually suggestive of what it does, to be displayed at the beginning of the zip flash. You should include your name/handle here like "by osm0sis @ xda-developers" for credit purposes.
apklist is a string containing the list of APKs to be patched included in the patch zip, separated by spaces between the quotes. Each APK is automatically found recursively in /system, then copied to the working directory to be decompiled and acted on, then copied back to /system.
apkbak is the location to place backups of the untouched APKs in apklist if backup=1 is set.
backup=1 will store backups of the untouched APKs in the location specified in apkbak.
cleanup=0 will keep the zip from removing it's working directory in /tmp/apkpatcher or any of the files resulting from the smali process - this can be useful if trying to debug in adb shell whether the patches worked correctly. cleanup=1 is necessary on multi-APK patching zips to clean the baksmali classout directory for the next APK, so it's recommended each APK to be patched be tested on their own with cleanup=0 before combining into a single zip.
Click to expand...
Click to collapse
Did you happen to figure out how to cross compile jdk for arm? (different from the arm sources available from oracle etc)
..and nice work!

Surge1223 said:
Did you happen to figure out how to cross compile jdk for arm? (different from the arm sources available from oracle etc)
..and nice work!
Click to expand...
Click to collapse
Nah, just forced bak/smali and apktool to run by building/adding classes.dex for each of them and figuring out how to call a functional dalvikvm from recovery.
The first part was mentioned in passing (and quite awesomely) by JesusFreke back in 2011.
That got it running in booted Android. Then running from recovery was a whole other issue, but I stumbled upon Jim Huang's work on executing dalvikvm on Linux, and that lead me to figuring out a solution.
As it stands, apktool should technically be able to do both the classes and resources parts of things (since it incorporates baksmali and smali), but for some reason it bails in recovery on decompiling the classes.dex where baksmali standalone doesn't. So perhaps in future revisions apktool could be the only jar required.
Similarly, apktool also bails when rebuilding resources.asrc because it can't call its internal(?) aapt, so I've included aapt compiled as PIE since that's the only working one I could find; in the future this could go too if apktool resolves it, or at the very least it would be more universal if we had a static compile of aapt to go with the other static binaries included.
Other possible items on the to-do list would include adding zip re-signing (currently it uses the apktool /system APK trick of keeping META-INF the same), reflashing to revert to the backup APKs, and adding busybox to always have an expected execution environment like I recently did on AnyKernel2 to work around Cyanogen Recovery and toybox missing basic stuff like awk at the moment. These aren't something I'm going to be actively working on since it works and APK modding isn't really a focus for me, but since I figured out how to automate the process I wanted to provide the WIP template for the modding community. :good:

osm0sis said:
Nah, just forced bak/smali and apktool to run by building/adding classes.dex for each of them and figuring out how to call a functional dalvikvm from recovery.
The first part was mentioned in passing (and quite awesomely) by JesusFreke back in 2011:
http://forum.xda-developers.com/showpost.php?p=10424700&postcount=2
That got it running in booted Android. Then running from recovery was a whole other issue, but I stumbled upon Jim Huang's work on executing dalvikvm on Linux, and that lead me to figuring out a solution.
As it stands, apktool should technically be able to do both the classes and resources parts of things (since it incorporates baksmali and smali), but for some reason it bails in recovery on decompiling the classes.dex where baksmali standalone doesn't. So perhaps in future revisions apktool could be the only jar required.
Similarly, apktool bails when rebuilding resources.asrc because it can't call its internal(?) aapt, so I've included aapt compiled as PIE since that's the only working one I could find; in the future this could go too if apktool resolves it or at the very least it would be more universal if we had a static compile of aapt to go with the other static binaries included.
Other possible items on the to-do list would include adding zip re-signing (currently it uses the apktool /system APK trick of keeping META-INF the same), and adding busybox to always have an expected execution environment like I recently did on AnyKernel2 to work around Cyanogen Recovery and toybox missing basic stuff like awk. These aren't something I'm going to be actively working on since it works and APK modding isn't really a focus for me, but since I figured out how to automate the process I wanted to provide the WIP template for the modding community. :good:
Click to expand...
Click to collapse
Nice! I tried to compile aapt from my device a few mins ago just for s&g's got pretty far actually before it couldn't compile some external android libs because I didn't have them downloaded.
Have you ever seen the Android apktool apk by pqy330? It's not on git anymore for some reason, but the guy figured out how to get java, javac, aapt, aidl, and everything working on arm, only required a LD_LIBRARY_PATH export. I wish I could get a hold of the guy to ask him about getting some of those tougher arm static binaries compiled.
And yeah I wouldn't actively spend hours working on this if it's already in a working state like you said, it's just interesting compared to the other mundane things I read of xda lol. Especially new developments. Great job man.

APK-Patcher: minor fixes, binary updates:
- allow for recoveries without busybox
- update tool compiles to my newer zipalign and static(!) aapt builds
- strip zip binary to compensate for larger aapt and zipalign binaries
- simplify aapt usage now that we have a static arm binary
- configure zipalign for Marshmallow system APK handling
https://github.com/osm0sis/APK-Patcher/commit/af24805f06e2e2b92f2bfeeb1490d6022073c4b5
Thanks @Surge1223 for the help with figuring out compiling zipalign and aapt statically for arm. :highfive:

Wow, i wish that i know this patcher before
so im trying to add two lines after line #24 in "res/xml/tuner_prefs.xml"
i tried
Code:
sed -i '24 i\> <Preference android:title="@string/nav_bar" android:key="nav_bar" android:fragment="com.android.systemui.tuner.NavBarTuner" />' res/xml/tuner_prefs.xml;
sed -i '25 i\> <Preference android:title="@string/color_and_appearance" android:key="color_transform" android:fragment="com.android.systemui.tuner.ColorAndAppearanceFragment" />' res/xml/tuner_prefs.xml;
and this also
Code:
sed -i '24 i\> <Preference android:title="@string/nav_bar" android:key="nav_bar" android:fragment="com.android.systemui.tuner.NavBarTuner" />' res/xml/tuner_prefs.xml;
sed -i '25 i\> <Preference android:title="@string/color_and_appearance" android:key="color_transform" android:fragment="com.android.systemui.tuner.ColorAndAppearanceFragment" />' res/xml/tuner_prefs.xml;
and it doesn't work, im really sorry i never used sed commands before
do i have to add original file in "patch/SystemUI/res/xml" and then the script should change it ?

AL_IRAQI said:
Wow, i wish that i know this patcher before
so im trying to add two lines after line #24 in "res/xml/tuner_prefs.xml"
i tried
Code:
sed -i '24 i\> <Preference android:title="@string/nav_bar" android:key="nav_bar" android:fragment="com.android.systemui.tuner.NavBarTuner" />' res/xml/tuner_prefs.xml;
sed -i '25 i\> <Preference android:title="@string/color_and_appearance" android:key="color_transform" android:fragment="com.android.systemui.tuner.ColorAndAppearanceFragment" />' res/xml/tuner_prefs.xml;
and this also
Code:
sed -i '24 i\> <Preference android:title="@string/nav_bar" android:key="nav_bar" android:fragment="com.android.systemui.tuner.NavBarTuner" />' res/xml/tuner_prefs.xml;
sed -i '25 i\> <Preference android:title="@string/color_and_appearance" android:key="color_transform" android:fragment="com.android.systemui.tuner.ColorAndAppearanceFragment" />' res/xml/tuner_prefs.xml;
and it doesn't work, im really sorry i never used sed commands before
do i have to add original file in "patch/SystemUI/res/xml" and then the script should change it ?
Click to expand...
Click to collapse
Nah, per the OP, the "patch" directory is only for replacing files whole, which you shouldn't do to keep it universal.
You just need a working sed command in the script directory.
You can borrow this script function from my AnyKernel2 to make the insertion easier for you as a sed beginner:
Code:
# insert_line <file> <if search string> <before|after> <line match string> <inserted line>
insert_line() {
if [ -z "$(grep "$2" $1)" ]; then
case $3 in
before) offset=0;;
after) offset=1;;
esac;
line=$((`grep -n "$4" $1 | head -n1 | cut -d: -f1` + offset));
sed -i "${line}s;^;${5}\n;" $1;
fi;
}
Then follow that with an insert_line command using the syntax provided. "if search string" tests whether the mod has been completed already, then "line match string" is the line you want to put the inserted lines before/after.
You can insert both lines at once with a \n between as well.

osm0sis said:
Nah, per the OP, the "patch" directory is only for replacing files whole, which you shouldn't do to keep it universal.
You just need a working sed command in the script directory.
You can borrow this script function from my AnyKernel2 to make the insertion easier for you as a sed beginner:
Code:
# insert_line <file> <if search string> <before|after> <line match string> <inserted line>
insert_line() {
if [ -z "$(grep "$2" $1)" ]; then
case $3 in
before) offset=0;;
after) offset=1;;
esac;
line=$((`grep -n "$4" $1 | head -n1 | cut -d: -f1` + offset));
sed -i "${line}s;^;${5}\n;" $1;
fi;
}
Then follow that with an insert_line command using the syntax provided. "if search string" tests whether the mod has been completed already, then "line match string" is the line you want to put the inserted lines before/after.
You can insert both lines at once with a \n between as well.
Click to expand...
Click to collapse
im such a noob, even this basic code is hard for me to understand :crying:
do i have to keep grep commands ?, because i dont need to replace or find something in the file, only inserting after line #24
$1 is the file here right ?, and your trying to find $2 and add someting, and in case of 3$ there is another case, and edits will probably be before/after $4 ?

AL_IRAQI said:
im such a noob, even this basic code is hard for me to understand :crying:
do i have to keep grep commands ?, because i dont need to replace or find something in the file, only inserting after line #24
$1 is the file here right ?, and your trying to find $2 and add someting, and in case of 3$ there is another case, and edits will probably be before/after $4 ?
Click to expand...
Click to collapse
What is on line 24? Wouldn't it be more universal to match against some part of the contents of the current line 24 to ensure its always in the right place, even if the number changes?
The $1 $2 $3 $4 $5 correspond to each argument supplied as per the <syntax> in the #commented line.

osm0sis said:
What is on line 24? Wouldn't it be more universal to match against some part of the contents of the current line 24 to ensure its always in the right place, even if the number changes?
The $1 $2 $3 $4 $5 correspond to each argument supplied as per the <syntax> in the #commented line.
Click to expand...
Click to collapse
i modded systemui for N5X,N6/6P,N9 and android one device, all of them has the same layout
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
or i can search for "<Preference android:title="@string/other"" and add it before it
oh, now its easier to understand ;P

Okay, so.. Probably:
Code:
insert_line res/xml/tuner_prefs.xml "com.android.systemui.tuner.NavBarTuner" before "com.android.systemui.tuner.OtherPrefs" "<Preference android:title=\"@string/nav_bar\" android:key=\"nav_bar\" android:fragment=\"com.android.systemui.tuner.NavBarTuner\" />\n<Preference android:title=\"@string/color_and_appearance\" android:key=\"color_transform\" android:fragment=\"com.android.systemui.tuner.ColorAndAppearanceFragment\" />";

osm0sis said:
Okay, so.. Probably:
Code:
insert_line res/xml/tuner_prefs.xml "com.android.systemui.tuner.NavBarTuner" before "com.android.systemui.tuner.OtherPrefs" "<Preference android:title=\"@string/nav_bar\" android:key=\"nav_bar\" android:fragment=\"com.android.systemui.tuner.NavBarTuner\" />\n<Preference android:title=\"@string/color_and_appearance\" android:key=\"color_transform\" android:fragment=\"com.android.systemui.tuner.ColorAndAppearanceFragment\" />";
Click to expand...
Click to collapse
i removed /patch and added the script to SystemUI-res.sh, but it doesn't seems to work
screenshot:

AL_IRAQI said:
i removed /patch and added the script to SystemUI-res.sh, but it doesn't seems to work
screenshot:
Click to expand...
Click to collapse
You shouldn't see that message for what you're doing unless you've got things set up weird. Can you just attach your WIP zip and I'll take a look?

osm0sis said:
You shouldn't see that message for what you're doing unless you've got things set up weird. Can you just attach your WIP zip and I'll take a look?
Click to expand...
Click to collapse
here you are, link
Thanks

osm0sis said:
You shouldn't see that message for what you're doing unless you've got things set up weird. Can you just attach your WIP zip and I'll take a look?
Click to expand...
Click to collapse
AL_IRAQI said:
here you are, link
Thanks
Click to expand...
Click to collapse
Okay, try this. I've got you about 90% of the way there; fixed all the weirdness with the unnecessary patch folder, extra files and extracmd. The sed insertion is still untested, so you'll have to check that out, and from what you and others have told me you'll need to update the included tools for Nougat support.
Download the newer bak/smali and apktool, then that's done with
Code:
dx --dex --output=classes.dex <file>.jar
from within the latest SDK. Add the classes.dex to the jar (it's a zip). Name them like the current ones (version numbers, etc.) and put them in tools and remove the old ones. :good:
[ Attachment removed. ]

osm0sis said:
Okay, try this. I've got you about 90% of the way there; fixed all the weirdness with the unnecessary patch folder, extra files and extracmd. The sed insertion is still untested, so you'll have to check that out, and from what you've told me you'll need to update the included tools for Nougat support.
Download the newer bak/smali and apktool, then that's done with
Code:
dx --dex --output=classes.dex <file>.jar
from within the latest NDK. Add the classes.dex to the jar (it's a zip). Name them like the current ones (version numbers, etc.) and put them in tools and remove the old ones. :good:
Click to expand...
Click to collapse
Awesome, ill try it today :fingers-crossed:
Edit:
i already downloaded newer tools, and now i have to run this code ?
Code:
dx --dex --output=classes.dex apktool_2.0.3.jar
dx --dex --output=classes.dex baksmali-2.1.1.jar
dx --dex --output=classes.dex smali-2.1.1.jar

AL_IRAQI said:
Awesome, ill try it today :fingers-crossed:
Edit:
i already downloaded newer tools, and now i have to run this code ?
Code:
dx --dex --output=classes.dex apktool_2.0.3.jar
dx --dex --output=classes.dex baksmali-2.1.1.jar
dx --dex --output=classes.dex smali-2.1.1.jar
Click to expand...
Click to collapse
@AL_IRAQI, sorry, just saw your edit. The latest bak/smali is 2.1.3, and latest apktool is 2.2.0, but otherwise yes, you run that command on each jar separately from the build-tools directory of the latest Android SDK, and put the resulting classes.dex inside the jar (which is a zip file). Then it's able to be run on Android as in my zip.

New commit!
tools: binary updates:
https://github.com/osm0sis/APK-Patcher/commit/e4f6e3e90e13c862652183460a86e49cb0514fd9
Since I felt bad @AL_IRAQI didn't appear to be able to figure it out for his mods. :angel:

From an incorrect thread:
aj20010319 said:
Can I combine APK Patcher with aroma? Like use the standard updater-binary and adding all these functions from updater-binary into a different shell script and executing it later?
Click to expand...
Click to collapse
aj20010319 said:
Okay so I somehow managed to get aroma and apk patcher together, made separate zips and then made one zip flash the other.
I am trying to decompile framework-res.apk, but it's failing, logs show this error,
Code:
framework-res.apk
I: Using Apktool 2.2.0 on framework-res.apk
I: Loading resource table...
I: Decoding AndroidManifest.xml with resources...
I: Regular manifest package...
I: Decoding file-resources...
java.lang.NoClassDefFoundError: Failed resolution of: Ljavax/imageio/ImageIO;
at brut.androlib.res.decoder.Res9patchStreamDecoder.decode(Res9patchStreamDecoder.java:39)
at brut.androlib.res.decoder.ResStreamDecoderContainer.decode(ResStreamDecoderContainer.java:33)
at brut.androlib.res.decoder.ResFileDecoder.decode(ResFileDecoder.java:120)
at brut.androlib.res.decoder.ResFileDecoder.decode(ResFileDecoder.java:87)
at brut.androlib.res.AndrolibResources.decode(AndrolibResources.java:262)
at brut.androlib.Androlib.decodeResourcesFull(Androlib.java:131)
at brut.androlib.ApkDecoder.decode(ApkDecoder.java:108)
at brut.apktool.Main.cmdDecode(Main.java:163)
at brut.apktool.Main.main(Main.java:81)
Caused by: java.lang.ClassNotFoundException: Didn't find class "javax.imageio.ImageIO" on path: DexPathList[[zip file "/tmp/apkpatcher/tools/apktool_2.2.0-dexed.jar"],nativeLibraryDirectories=[/system/lib]]
at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java)
at java.lang.ClassLoader.loadClass(ClassLoader.java)
at java.lang.ClassLoader.loadClass(ClassLoader.java)
... 9 more
Suppressed: java.lang.ClassNotFoundException: javax.imageio.ImageIO
at java.lang.Class.classForName(Native Method)
at java.lang.BootClassLoader.findClass(ClassLoader.java)
at java.lang.BootClassLoader.loadClass(ClassLoader.java)
... 11 more
Caused by: java.lang.NoClassDefFoundError: Class not found using the boot class loader; no stack trace available
Decoding APK resources failed. Aborting...
Click to expand...
Click to collapse
Seems like you're in the wrong thread? I'm not sure. You can try reverting to the previous builds of tools, but beyond that I'm not sure at the moment. It's on my to-do to test everything out again in the near future and resolve any issues I can come up with. Please reply in the APK-Patcher thread.