Related
Manually Deodexing (Windows/Linux/OSX)
Let's start first with deodexing manually in order to learn about how the whole process works. Deodexing manually can be very useful if you only want to deodex a couple apks, or if some apks are giving tools such as xUltimate trouble. So here's what you'll need:
Basic knowledge of odex vs. deodex... READ THIS
Smali/baksmali installed correctly (if you are on Linux/OSX you can get the wrapper script as well, otherwise just use the .jars)
Java JRE installed correctly
The apks you want to deodex
ALL of the files in /system/framework from your ROM
This is also an excellent read for understanding the process of deodexing an apk
The first thing we want to do is find the bootclasspath for the ROM you are using. You will have to get the file /init.rc from your phone in some way; if you prefer using root explorer then use root explorer to copy it to your sd card, I prefer using adb. Once you have init.rc on your computer, open it up in a text editor. Search for the line that begins with "export BOOTCLASSPATH" and copy the rest of the line. It should look like this:
Code:
/system/framework/core.jar:/system/framework/core-junit.jar:/system/framework/bouncycastle.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/framework-ext.jar:/system/framework/android.policy.jar:/system/framework/services.jar:/system/framework/apache-xml.jar:/system/framework/filterfw.jar:/system/framework/com.motorola.android.frameworks.jar:/system/framework/com.motorola.android.widget.jar:/system/framework/com.motorola.frameworks.core.addon.jar:/system/framework/kafdex.jar:/system/framework/com.motorola.orange.simauth.jar
The bootclasspath is simply a list of dependencies that the system apps need to run - i.e. jars/apks that contain code that the system apps use. It is necessary to have the correct bootclasspath when deodexing.
Now that we have the bootclasspath we can start to set up for the actual deodexing. All the files listed in the bootclasspath are found in /system/framework, so grab all the jars/apks in /system/framework from your ROM (adb pull them or take them from the flashable zip if available) and put them in a folder named "framework" somewhere that you can remember, such as your desktop or at the root of your hard drive (i.e. C:\framework). We will use the bootclasspath to find all the files in this folder that are needed to deodex the apps properly.
Now we have to modify the bootclasspath in order to get the baksmali command to work correctly. The current bootclasspath that we have is saying that all the frameworks are in the /system/framework folder of your computer, but most of you have the frameworks somewhere like C:\framework or /home/me/framework. So open your text editor again and paste the current bootclasspath. Hopefully your editor will have a find/replace function; if so, just keep replacing all instances of "/system/framework" with the path to your frameworks. For example, this is what it would look like if your frameworks are in C:\framework:
Code:
C:\framework\core.jar:C:\framework\core-junit.jar:C:\framework\bouncycastle.jar:C:\framework\ext.jar:C:\framework\framework.jar:C:\framework\framework-ext.jar:C:\framework\android.policy.jar:C:\framework\services.jar:C:\framework\apache-xml.jar:C:\framework\filterfw.jar:C:\framework\com.motorola.android.frameworks.jar:C:\framework\com.motorola.android.widget.jar:C:\framework\com.motorola.frameworks.core.addon.jar:C:\framework\kafdex.jar:C:\framework\com.motorola.orange.simauth.jar
Finally we are ready to actually deodex the apks. Place the apks and odex's (odexes?) you want to deodex anywhere... your desktop, a folder somewhere, the root of your hard drive... it does not matter. Place the smali/baksmali .jar's in the same directory as the apps you wish to deodex. Open up a command prompt/terminal/etc and "cd" to the directory that the .apk's are in. For example if they are in C:\apps_to_deodex then I would type
Code:
cd C:\apps_to_deodex
The process for deodexing is a little dance like this:
Use baksmali to decompile the .odex file to smali
Use smali to compile the .smali files to a .dex
Use an archive manager (ex. 7zip) to put the newly created .dex file into the original .apk
Delete the old .odex file and all the decompiled smali.
So, let's begin.
BAKSMALI
Since baksmali/smali are .jar's, we will be using java to execute them. The syntax for the baksmali command is something like this:
Code:
java -jar baksmali.jar -a [COLOR="Red"]api_level[/COLOR] -c [COLOR="red"]bootclasspath[/COLOR] -x [COLOR="red"]file.odex[/COLOR]
The -a argument is not necessary with ICS. If you are on GB then you must use "-a 10" in order for it to deodex properly.
The bootclasspath is the bootclasspath that we determined earlier
The file.odex is, of course, the name of the odex file you want to decompile.
So let's say I want to decompile abc.odex, which was built for ICS. The command would be:
Code:
java -jar baksmali.jar -c C:\framework\core.jar:C:\framework\core-junit.jar:C:\framework\bouncycastle.jar:C:\framework\ext.jar:C:\framework\framework.jar:C:\framework\framework-ext.jar:C:\framework\android.policy.jar:C:\framework\services.jar:C:\framework\apache-xml.jar:C:\framework\filterfw.jar:C:\framework\com.motorola.android.frameworks.jar:C:\framework\com.motorola.android.widget.jar:C:\framework\com.motorola.frameworks.core.addon.jar:C:\framework\kafdex.jar:C:\framework\com.motorola.orange.simauth.jar -x abc.odex
This command creates a directory called "out" containing the decompiled odex file.
SMALI
Now that we have "baksmalid" the odex file, we now need to "smali" it to turn the .smali code into the .dex format. The syntax for this command is something like this:
Code:
java -jar smali.jar out -o classes.dex
We are calling the newly generated file "classes.dex" because that is the file that Android looks for in deodexed apks.
When the command completes you will have a file called "classes.dex" sitting in the directory holding all the deodexed apps.
ARCHIVE MANAGER
Hopefully by now you understand the difference between an odexed app and a deodexed app...
Odexed -> abc.apk AND abc.odex
Deodexed -> abc.apk ONLY which contains classes.dex
So, since we have the classes.dex, all we have to do is drag-n-drop it into the respective.apk using an archive manager such as 7zip (Windows). The result is an .apk which contains all the files necessary for it to install and run... it is deodexed.
CLEANUP
If you are going to repeat the process multiple times you will begin to get confused with filenames and everything. Since you now have a fully funtional abc.apk, you can delete:
-The abc.odex file as it is no longe necessary
-The "out" folder created in step 1
-The classes.dex because it is already copied into abc.apk
TROUBLESHOOTING
Sometimes, no matter what you try, you just can not get an app to deodex because it just isn't finding the right dependencies to load. If you come across some troublesome apps, you can use the "-I" argument when baksmaling to force it to decompile:
Code:
java -jar baksmali.jar -a [COLOR="Red"]api_level[/COLOR] -c [COLOR="red"]bootclasspath[/COLOR] -x -I [COLOR="red"]file.odex[/COLOR]
A warning... if you use the -I option on an app that is normally able to deodex fine, you will most likely get force closes and other random errors. Only use -I as a last resort.
For some reason our BOOTCLASSPATH on the ICS leak seems to list some files that don't exist in /system/framework. This means that when you try to baksmali you will get an error about not being able to load class files. If you get this error, look for the .jar that it references, and delete it from the BOOTCLASSPATH and try again. In the end it should look something like this in Windows:
Code:
C:\path_to_frameworks\core.jar:C:\path_to_frameworks\core-junit.jar:C:\path_to_frameworks\bouncycastle.jar:C:\path_to_frameworks\ext.jar:C:\path_to_frameworks\framework.jar:C:\path_to_frameworks\framework-ext.jar:C:\path_to_frameworks\android.policy.jar:C:\path_to_frameworks\services.jar:C:\path_to_frameworks\apache-xml.jar:C:\path_to_frameworks\filterfw.jar:C:\path_to_frameworks\com.motorola.android.frameworks.jar:C:\path_to_frameworks\com.motorola.android.widget.jar:C:\path_to_frameworks\com.motorola.frameworks.core.addon.jar
Thanks alteredlikeness for the tip!
Deodexing Automatically with xUltimate (Windows)
Coming soon...
Hope this helps clear up some questions that you've had... enjoy!
Nice work!
Mods, stickify please?
Sent from my SAMSUNG-SGH-I747 using xda premium
Grrreat! Thank you for putting this together! I swear I dreamed about this last nite...
But, a note on bootclasspath: For ICS, I have run into problems using it directly from the init.rc - we don't physically have all of those jar files in our framework. So, I removed a couple and have not ran into problems since. Here's what I got:
Code:
/system/framework/core.jar:/system/framework/core-junit.jar:/system/framework/bouncycastle.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/framework-ext.jar:/system/framework/android.policy.jar:/system/framework/services.jar:/system/framework/apache-xml.jar:/system/framework/filterfw.jar:/system/framework/com.motorola.android.frameworks.jar:/system/framework/com.motorola.android.widget.jar:/system/framework/com.motorola.frameworks.core.addon.jar
Edit: edited.
alteredlikeness said:
Grrreat! Thank you for putting this together! I swear I dreamed about this last nite...
But, a note on bootclasspath: For ICS, I have run into problems using it directly from the init.rc - we don't physically have all of those jar files in our framework. So, I removed a couple and have not ran into problems since. Here's what I got:
Code:
/system/framework/core.jar:/system/framework/core-junit.jar:/system/framework/bouncycastle.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/framework-ext.jar:/system/framework/android.policy.jar:/system/framework/services.jar:/system/framework/apache-xml.jar:/system/framework/filterfw.jar:/system/framework/com.motorola.android.frameworks.jar:/system/framework/com.motorola.android.widget.jar:/system/framework/com.motorola.frameworks.core.addon.jar
Click to expand...
Click to collapse
Thanks! I haven't manually done anything in ICS so I forgot about that... modified OP
hello i got classes.dex and put it in Bluetooth.apk , i put it on system/app and removed old apk, .odex files but after reboot i can't find bluetooth.odex in system/app , and not working what i changed in apk...
Error when attempting first deodex
I get this error not matter how I do the command in CMD.
Error occured while loading boot class path files. Aborting.
org.jf.dexlib.Util.ExceptionWithContext: Cannot locate boot class path file C
at org.jf.dexlib.Code.Analysis.ClassPath.loadBootClassPath(ClassPath.java:218)
at org.jf.dexlib.Code.Analysis.ClassPath.initClassPath(ClassPath.java:146)
at org.jf.dexlib.Code.Analysis.ClassPath.InitializeClassPath(ClassPath.java:131)
at org.jf.baksmali.baksmali.disassembleDexFile(baksmali.java:101)
at org.jf.baksmali.main.main(main.java:308)
I am a noob when it comes to this stuff...do you have any insight on this? Thanks.
jcvermillion
HTC Droid Incredible 4G LTE
Rooted
Unlocked Bootloader
Ready to Deodex the old fashion way.
First: I got this error:
Error occured while loading boot class path files. Aborting.
org.jf.dexlib.Util.ExceptionWithContext: Cannot locate boot class path file C
at org.jf.dexlib.Code.Analysis.ClassPath.loadBootClassPath(ClassPath.java:218)
at org.jf.dexlib.Code.Analysis.ClassPath.initClassPath(ClassPath.java:146)
at org.jf.dexlib.Code.Analysis.ClassPath.InitializeClassPath(ClassPath.java:131)
at org.jf.baksmali.baksmali.disassembleDexFile(baksmali.java:101)
at org.jf.baksmali.main.main(main.java:308)
Click to expand...
Click to collapse
after writing this
Code:
java -jar baksmali-1.4.2.jar -a 10 -c \framework\core.jar:\framework\bouncycastle.jar:\framework\ext.jar:\framework\framework.jar:\framework\android.policy.jar:\framework\services.jar:\framework\core-junit.jar -x framework.odex
(I am using mac os x)
Second: what should I use to put classes.dex files inside .jar files??
I have the same error and where i must put your solution
"java -jar baksmali-1.4.2.jar -a 10 -c \framework\core.jar:\framework\bouncycastle.jar:\framework\ext.jar:\framework\framework.jar:\framework\android.policy.jar:\framework\services.jar:\framework\core-junit.jar -x framework.odex"
Where i must put this?
Thanks.
Edit: i used this topic to deodex my pre-rooted ROM: http://forum.xda-developers.com/showthread.php?t=2374008
Help-Me Memo.odex Samsung GT-S5360B
Help-me
C:\framework>java -jar baksmali.jar -a 10 -c c:\framework\core.jar:c:\framework\
bouncycastle.jar:c:\framework\ext.jar:c:\framework\framework.jar:c:\framework\an
droid.policy.jar:c:\framework\services.jar:c:\framework\core-junit.jar -x Memo.o
dex
Error occured while loading boot class path files. Aborting.
org.jf.dexlib.Util.ExceptionWithContext: Cannot locate boot class path file c
at org.jf.dexlib.Code.Analysis.ClassPath.loadBootClassPath(ClassPath.jav
a:237)
at org.jf.dexlib.Code.Analysis.ClassPath.initClassPath(ClassPath.java:14
5)
at org.jf.dexlib.Code.Analysis.ClassPath.InitializeClassPath(ClassPath.j
ava:131)
at org.jf.baksmali.baksmali.disassembleDexFile(baksmali.java:111)
at org.jf.baksmali.main.main(main.java:293)
anexo: Memo.apk, Memo.odex, init.rc, BOOTCLASSPATH.txt
Anyone got solution for these 'cannot find boot class path in C' error ?
EDIT:FOUND IT!!!
Just copy all framework files in C:/framework and instead of -c use -d
and command should look like
Code:
java -jar baksmali.jar -a [COLOR="Red"]##(api lvl)[/COLOR] -d C:/framework/ -x [COLOR="red"][.odex file][/COLOR]
No need to type the whole bootclasspath he just needs the path to framework folder
Hope it helps
I am trying to deodex telephony-common.jar and telephony-common.odex
I tried first
Code:
java -jar baksmali.jar -x telephony-common.odex
and got
Code:
Error occured while loading boot class path files. Aborting.
org.jf.dexlib.Code.Analysis.ClassPath$ClassNotFoundException: Could not find superclass Landroid/app/Service;
at org.jf.dexlib.Code.Analysis.ClassPath$ClassDef.loadSuperclass(ClassPath.java:784)
at org.jf.dexlib.Code.Analysis.ClassPath$ClassDef.<init>(ClassPath.java:668)
at org.jf.dexlib.Code.Analysis.ClassPath.loadClassDef(ClassPath.java:280)
at org.jf.dexlib.Code.Analysis.ClassPath.initClassPath(ClassPath.java:163)
at org.jf.dexlib.Code.Analysis.ClassPath.InitializeClassPathFromOdex(ClassPath.java:110)
at org.jf.baksmali.baksmali.disassembleDexFile(baksmali.java:98)
at org.jf.baksmali.main.main(main.java:278)
Error while loading class Landroid/accessibilityservice/AccessibilityService; from file ./framework.zip
Error while loading ClassPath class Landroid/accessibilityservice/AccessibilityService;
then
Code:
java -jar baksmali.jar -c "/home/matthew/temp/X9006&X9076ColorOS_V1.2.7i_full/system/framework"/core.jar:"/home/matthew/temp/X9006&X9076ColorOS_V1.2.7i_full/system/framework"/core-junit.jar:"/home/matthew/temp/X9006&X9076ColorOS_V1.2.7i_full/system/framework"/bouncycastle.jar:"/home/matthew/temp/X9006&X9076ColorOS_V1.2.7i_full/system/framework"/ext.jar:"/home/matthew/temp/X9006&X9076ColorOS_V1.2.7i_full/system/framework"/framework.jar:"/home/matthew/temp/X9006&X9076ColorOS_V1.2.7i_full/system/framework"/framework-ext.jar:"/home/matthew/temp/X9006&X9076ColorOS_V1.2.7i_full/system/framework"/android.policy.jar:"/home/matthew/temp/X9006&X9076ColorOS_V1.2.7i_full/system/framework"/services.jar:"/home/matthew/temp/X9006&X9076ColorOS_V1.2.7i_full/system/framework"/apache-xml.jar:"/home/matthew/temp/X9006&X9076ColorOS_V1.2.7i_full/system/framework"/filterfw.jar:"/home/matthew/temp/X9006&X9076ColorOS_V1.2.7i_full/system/framework"/com.motorola.android.frameworks.jar:"/home/matthew/temp/X9006&X9076ColorOS_V1.2.7i_full/system/framework"/com.motorola.android.widget.jar:"/home/matthew/temp/X9006&X9076ColorOS_V1.2.7i_full/system/framework"/com.motorola.frameworks.core.addon.jar:"/home/matthew/temp/X9006&X9076ColorOS_V1.2.7i_full/system/framework"/kafdex.jar:"/home/matthew/temp/X9006&X9076ColorOS_V1.2.7i_full/system/framework"/com.motorola.orange.simauth.jar -x telephony-common.odex
and got
Code:
Error occured while loading boot class path files. Aborting.
org.jf.dexlib.Util.ExceptionWithContext: Cannot locate boot class path file /home/matthew/temp/X9006&X9076ColorOS_V1.2.7i_full/system/framework/core.jar
at org.jf.dexlib.Code.Analysis.ClassPath.loadBootClassPath(ClassPath.java:237)
at org.jf.dexlib.Code.Analysis.ClassPath.initClassPath(ClassPath.java:145)
at org.jf.dexlib.Code.Analysis.ClassPath.InitializeClassPath(ClassPath.java:131)
at org.jf.baksmali.baksmali.disassembleDexFile(baksmali.java:105)
at org.jf.baksmali.main.main(main.java:278)
This is a step by step guide to compiling Rootbox on your computer.
Requirements:
willingness to learn as you go
Ubuntu 12.04 64bit, dual-booted or virtual machine
some space on your Ubuntu partition, I'd say about 50GB to be very safe
an internet connection
You will need to setup the android environment on your computer first. The next step will depend on what version of ubuntu you use. The steps are laid out for version 12.10 of Ubuntu. If you have 12.04 you will have to search to make sure the files you are installing are the same version. I use a program called synaptic package manager to check to make sure the correct files are being installed. Most Ubuntu OS do not install it by default. To install it type this into a terminal:
Code:
sudo apt-get install synaptic
enter this command into the terminal.
Code:
sudo apt-get install git-core gnupg flex bison gperf libsdl1.2-dev libesd0-dev libwxgtk2.8-dev squashfs-tools build-essential zip curl libncurses5-dev zlib1g-dev pngcrush schedtool g++-multilib lib32z1-dev lib32ncurses5-dev lib32readline6-dev gcc-4.7-multilib g++-4.7-multilib libxml2-utils libxml2 xsltproc
After that is all installed you will need to get rid of any java installed on your system.
to get rid of it type this into the terminal
Code:
sudo apt-get remove openjdk* icedtea* default-jre
This should remove all instances of java. Now to get the java that you want.
to get java add this ppa.
Code:
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java6-installer
Now to start building the rom.
in terminal type
Code:
mkdir ~/android/rootbox/
then type
Code:
cd ~/android/rootbox
in terminal type
Code:
repo init -u git://github.com/Root-Box/platform_manifest.git -b jb-mr1
in terminal type
Code:
gedit .repo/local_manifest.xml
place this inside file and save file
Code:
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<project path="device/htc/ville" name="atrus5/device_htc_ville" remote="gh" revision="master" />
<project name="android_kernel_htc_msm8960" path="kernel/htc/msm8960" remote="cm" revision="cm-10.1" />
<project path="device/htc/msm8960-common" name="android_device_htc_msm8960-common" remote="cm" revision="cm-10.1" />
<project path="vendor/htc/ville" name="atrus5/vendor_htc_ville" remote="gh" revision="master" />
</manifest>
in terminal type
Code:
repo sync
Go have some food, maybe watch a movie, go golfing, whatever you want to do cause this will take quite awhile. When that is done in terminal type
Code:
gedit vendor/rootbox/vendorsetup.sh
add this to end of file and save
Code:
add_lunch_combo rootbox_ville-userdebug
in terminal type
Code:
gedit vendor/rootbox/products/ville.mk
add this to file and save
Code:
# Specify phone tech before including full_phone
$(call inherit-product, vendor/rootbox/configs/gsm.mk)
PRODUCT_RELEASE_NAME := ville
# Boot animation
TARGET_SCREEN_HEIGHT := 960
TARGET_SCREEN_WIDTH := 540
# Inherit some common CM stuff.
$(call inherit-product, vendor/rootbox/configs/common.mk)
# Inherit device configuration
$(call inherit-product, device/htc/ville/device_ville.mk)
# Ville Overlay
PRODUCT_PACKAGE_OVERLAYS += vendor/rootbox/overlay/common
# PA OVERLAY_TARGET
OVERLAY_TARGET := pa_hdpi
# Setup device specific product configuration.
PRODUCT_DEVICE := ville
PRODUCT_NAME := rootbox_ville
PRODUCT_BRAND := HTC
PRODUCT_MODEL := HTC One S
PRODUCT_MANUFACTURER := HTC
# Copy Bootanimation
PRODUCT_COPY_FILES += \
vendor/rootbox/prebuilt/hdpi/bootanimation.zip:system/media/bootanimation.zip
in terminal type
Code:
gedit vendor/rootbox/products/AndroidProducts.mk
add this to end of file and save. (note: you will need to add a ' \' to the line before it, without the quotations. that is a space before the \). For example here is what my last two lines look like. $(LOCAL_DIR)/t0lte.mk \
$(LOCAL_DIR)/ville.mk
Code:
$(LOCAL_DIR)/ville.mk
This part is optional. I do these steps just to make sure that I have all the necessary files to compile the rom.
plug in your phone
in terminal type
Code:
cd device/htc/ville
and
Code:
./extract-files.sh
Ok now back to building the rom. You don't have to do the above steps, it's just to make sure.
in terminal type
Code:
cd ~/android/rootbox
or wherever you have placed your rootbox build. This is where I store mine.
in terminal type
Code:
./build_rootbox.sh ville
This should compile the rom and you can flash the nightly build onto your phone. The file is in the out/target/product/ville directory.
I hope this helps.
Why not just share what you build?
Or are we not allowed to share what we build, like what happened for the Baked Black Bean ROM?
Sent from my HTC One S using Tapatalk 2
RockR172 said:
Why not just share what you build?
Or are we not allowed to share what we build, like what happened for the Baked Black Bean ROM?
Sent from my HTC One S using Tapatalk 2
Click to expand...
Click to collapse
Good point. I will ask them and see if I can.
atrus5 said:
Good point. I will ask them and see if I can.
Click to expand...
Click to collapse
I am going to try this right after Baked!
Bowmanspeer said:
I am going to try this right after Baked!
Click to expand...
Click to collapse
I'm guessing permissions were granted...
Bowmanspeer said:
I'm guessing permissions were granted...
Click to expand...
Click to collapse
Rootbox is being built by $droyd$ now. If you want the rom without having to build it you can go to the one s development section and download it from there. I know that you will not be able to get the files needed from the proprietary repo as well as the device/htc/ville repo. If you need them let me know.
atrus5 said:
Rootbox is being built by $droyd$ now. If you want the rom without having to build it you can go to the one s development section and download it from there. I know that you will not be able to get the files needed from the proprietary repo as well as the device/htc/ville repo. If you need them let me know.
Click to expand...
Click to collapse
Yeah. I actually tried and it aborted because of errors. I just want to learn.
I did Build Baked from the other How to. Its a little different when build it yourself... But I want to know more...
Bowmanspeer said:
Yeah. I actually tried and it aborted because of errors. I just want to learn.
I did Build Baked from the other How to. Its a little different when build it yourself... But I want to know more...
Click to expand...
Click to collapse
What were the errors? I felt the same way when trying to compile roms too. Got a lot of errors but struggled through it.
Sent from my One S using xda premium
atrus5 said:
What were the errors? I felt the same way when trying to compile roms too. Got a lot of errors but struggled through it.
Click to expand...
Click to collapse
2 main sources wouldn't download. One was a 404 page not found and the other had to do with permissions I think. I've closed the terminal so its gone. I didn't have this problem when I did Baked. The problem there was something to do with the Lunch menu till $droyd$ stepped in and provided a different command that compiled and produced the flashable zip and boot.img.
Bowmanspeer said:
2 main sources wouldn't download. One was a 404 page not found and the other had to do with permissions I think. I've closed the terminal so its gone. I didn't have this problem when I did Baked. The problem there was something to do with the Lunch menu till $droyd$ stepped in and provided a different command that compiled and produced the flashable zip and boot.img.
Click to expand...
Click to collapse
I have switched the post to reflect the changes needed for the error, or at least what I think is the error. These are changes to the local_manifest.xml file.
Thanks. I'll give it another whirl.
Sent.
Another one trying to compile...
Hello artus5,
thanks for your guide!! As I would like to learn how to build Roms from Source too, I just tested your manual.
Unfortunatelly I ran into problems, wright from the start. I got this error and don't know how to get over it...
HTML:
Lunching your device
build/core/product_config.mk:234: *** Can not locate config makefile for product "rootbox_". Schluss.
** Don't have a product spec for: 'rootbox_'
** Do you have the right repo manifest?
Starting RootBox build for
including vendor/rootbox/vendorsetup.sh
build/core/product_config.mk:234: *** Can not locate config makefile for product "rootbox_". Schluss.
** Don't have a product spec for: 'rootbox_'
** Do you have the right repo manifest?
No such item in brunch menu. Try 'breakfast'
Maybe you could help me with that.
Thanks and happy eastern!
rum0s said:
Hello artus5,
thanks for your guide!! As I would like to learn how to build Roms from Source too, I just tested your manual.
Unfortunatelly I ran into problems, wright from the start. I got this error and don't know how to get over it...
HTML:
Lunching your device
build/core/product_config.mk:234: *** Can not locate config makefile for product "rootbox_". Schluss.
** Don't have a product spec for: 'rootbox_'
** Do you have the right repo manifest?
Starting RootBox build for
including vendor/rootbox/vendorsetup.sh
build/core/product_config.mk:234: *** Can not locate config makefile for product "rootbox_". Schluss.
** Don't have a product spec for: 'rootbox_'
** Do you have the right repo manifest?
No such item in brunch menu. Try 'breakfast'
Maybe you could help me with that.
Thanks and happy eastern!
Click to expand...
Click to collapse
What are you typing for the command to start it? I have made some changes to the tutorial. Could you please reread it and tell me if you have typed anything wrong. I just tested it on mine and everything worked for me. It sounds like you have entered something wrong in the ville.mk file or else you typed ./build-rootbox.sh rootbox_ville instead of just ville. I also didn't put how to set up the android environment into the tutorial. I should probably do that as well.
atrus5 said:
It sounds like you have entered something wrong in the ville.mk file or else you typed ./build-rootbox.sh rootbox_ville instead of just ville. I also didn't put how to set up the android environment into the tutorial. I should probably do that as well.
Click to expand...
Click to collapse
Oh my fault... I entered ./build_rootbox.sh instead of ./build_rootbox.sh ville. Sorry!
As I'm very new to this and would really like to learn it, enhancing your tutorial would be very much appreciated :laugh:
Therefore I have a question.
The part where I add the local_manifest.xml. Is it for getting the devices files that are not included in the rootbox git repository as my device is not supported? Why do I have then to extract files from my device with the ./extract-files.sh script?
rum0s said:
Oh my fault... I entered ./build_rootbox.sh instead of ./build_rootbox.sh ville. Sorry!
As I'm very new to this and would really like to learn it, enhancing your tutorial would be very much appreciated :laugh:
Therefore I have a question.
The part where I add the local_manifest.xml. Is it for getting the devices files that are not included in the rootbox git repository as my device is not supported? Why do I have then to extract files from my device with the ./extract-files.sh script?
Click to expand...
Click to collapse
I did this just to make sure that all the files that were needed were taken from the phone. I don't like to leave anything out of the rom. I will change that to reflect the fact that all the files are already added. Thank you. I have changed to tutorial to be a full tutorial.
edit: thank you for making this into a better tutorial.
edit: the extra lines added to the local_manifest.xml file is for your device kernel as well as any device files that your phone will need.
@atrus5: you got PM and thanks
ok here's another question, as the compiling process has been aborted
it tells me something like (my terminal is in german):
HTML:
make: *** no rule to make target »vendor/rootbox/prebuilt/rootbox_ville.conf«,
from »/media/..../rootbox/out/target/product/ville/system/etc/paranoid/properties.conf«. end.
make: *** Waiting for processes not yet finished...
I couldn't find a file called rootbox_ville.conf in the prebuilt folder. Is this the problem?
Futhermore there were some files not found but the message was just skipped without any interuption/problem...
HTML:
find: "../../vendor/unbundled_google/libs/gcm/gcm-client/src": file or folder not found
find: "../../vendor/unbundled_google/libs/gcm/gcm-server/src": file or folder not found
find: "../../vendor/unbundled_google/libs/gcm/gcm-client/src": file or folder not found
find: "../../vendor/unbundled_google/libs/gcm/gcm-server/src": file or folder not found
rum0s said:
ok here's another question, as the compiling process has been aborted
it tells me something like (my terminal is in german):
HTML:
make: *** no rule to make target »vendor/rootbox/prebuilt/rootbox_ville.conf«,
from »/media/..../rootbox/out/target/product/ville/system/etc/paranoid/properties.conf«. end.
make: *** Waiting for processes not yet finished...
I couldn't find a file called rootbox_ville.conf in the prebuilt folder. Is this the problem?
Futhermore there were some files not found but the message was just skipped without any interuption/problem...
HTML:
find: "../../vendor/unbundled_google/libs/gcm/gcm-client/src": file or folder not found
find: "../../vendor/unbundled_google/libs/gcm/gcm-server/src": file or folder not found
find: "../../vendor/unbundled_google/libs/gcm/gcm-client/src": file or folder not found
find: "../../vendor/unbundled_google/libs/gcm/gcm-server/src": file or folder not found
Click to expand...
Click to collapse
That looks like something that rootbox has just added to there files in the last 9 hours. I don't know how to fix that yet. I will see what I can do.
rum0s said:
ok here's another question, as the compiling process has been aborted
it tells me something like (my terminal is in german):
HTML:
make: *** no rule to make target »vendor/rootbox/prebuilt/rootbox_ville.conf«,
from »/media/..../rootbox/out/target/product/ville/system/etc/paranoid/properties.conf«. end.
make: *** Waiting for processes not yet finished...
I couldn't find a file called rootbox_ville.conf in the prebuilt folder. Is this the problem?
Futhermore there were some files not found but the message was just skipped without any interuption/problem...
HTML:
find: "../../vendor/unbundled_google/libs/gcm/gcm-client/src": file or folder not found
find: "../../vendor/unbundled_google/libs/gcm/gcm-server/src": file or folder not found
find: "../../vendor/unbundled_google/libs/gcm/gcm-client/src": file or folder not found
find: "../../vendor/unbundled_google/libs/gcm/gcm-server/src": file or folder not found
Click to expand...
Click to collapse
You will need to change your ville.mk file to this as they have made changes onto the rootbox build. Replace everything in your ville.mk file with this.
Code:
# Specify phone tech before including full_phone
$(call inherit-product, vendor/rootbox/configs/gsm.mk)
PRODUCT_RELEASE_NAME := ville
# Boot animation
TARGET_SCREEN_HEIGHT := 960
TARGET_SCREEN_WIDTH := 540
# Inherit some common CM stuff.
$(call inherit-product, vendor/rootbox/configs/common.mk)
# Inherit device configuration
$(call inherit-product, device/htc/ville/device_ville.mk)
# Ville Overlay
PRODUCT_PACKAGE_OVERLAYS += vendor/rootbox/overlay/common
# PA OVERLAY_TARGET
OVERLAY_TARGET := pa_hdpi
# Setup device specific product configuration.
PRODUCT_DEVICE := ville
PRODUCT_NAME := rootbox_ville
PRODUCT_BRAND := HTC
PRODUCT_MODEL := HTC One S
PRODUCT_MANUFACTURER := HTC
# Copy Bootanimation
PRODUCT_COPY_FILES += \
vendor/rootbox/prebuilt/hdpi/bootanimation.zip:system/media/bootanimation.zip
edit: I would do a repo sync after changing the ville.mk file and also type in the terminal:
Code:
rm -rf out
This will start the build process all over again.
Okay fantastic, thanks for your immediate answer! compiling has been started again
So how did you figure it out, what the problem was? As I can see, there has been inserted an overlay...I just looked into the pa_hdpi.conf it's telling the system how various apps shall look like in terms of DPI. Was this the missing part? Or has an "old overlay" been removed?
Moreover I created a directory for the local_manifets.xml file in the .repo folder, as the compilier told me to do that. Maybe you could add this to your OP too.
(Check out https://www.python.org for information on what python is.)
Yes you read that correctly, static python for Android! It took a while to figure out how to get this compiled, but I finally did it. I present you with an installer and a little bit of testing. The second post contains information on compilation and the sources. Btw, any cool python scripts are welcomed!
INSTALLATION
Recovery flash installers are attached that will install python 2.7.9 and/or 3.4.2 to /system/pythonX.X.X. You can install both if you want. Installation size is about 41MB, 43MB, and 49MB respectively for python 2.7.8, python 2.7.9, and python 3.4.2.
After installation, the python director(ies) in /system will contain the static python binary, some scripts, and a bunch of modules and documents. Separate scripted executables will be installed to /system/bin/python or /system/bin/python3 depending on which one is installed. These basically just set the PYTHONHOME environment variable and execute the python binary.
TESTING
When you boot up to Android after installation, you should be able to just open up a terminal and fire off some python commands.
Test using python 2.7.9:
Code:
python -c 'print "Hello World!"'
Test using python 3.4.2:
Code:
python3 -c 'print("Hello World!")'
You can also write scripts shelled with python:
Code:
#!/system/bin/python
print "Hello World!"
Make sure to set them as executable with "chmod +x".
Adjust your screen brightness via /sys (root required, tested on Galaxy Nexus):
Code:
python -c 'f=open("/sys/devices/omapdss/display0/backlight/s6e8aa0/brightness","w"); f.write("40"); f.close()'
A cool script I made to tweak file system I/O like rq_affinity, rotational, etc for I/O blocks:
Code:
#!/system/bin/python
import os,re,sys
list=[]
# find all directories containing rq_affinity
for roots, dirs, files in os.walk('/sys'):
for file in files:
match=re.search(r'\S+/rq_affinity',os.path.join(roots,file))
if match:
list.append(match.group().replace('rq_affinity',''))
# write specific values to files in each directory found before
for dir in list:
for name in 'rq_affinity', 'rotational', 'read_ahead_kb', 'nr_requests', 'iostats', 'nomerges', 'add_random':
try:
f=open(dir+name,'w')
if name is 'rq_affinity': f.write('1')
elif name is 'read_ahead_kb': f.write('512')
elif name is 'nr_requests': f.write('512')
else: f.write('0')
f.close()
except IOError:
sys.stderr.write('Problem writing to ' + dir+name + '\n')
Or execute "python" or "python3" without any parameters to open up the interpreter and go from there. When you start for example "python" (python 2.7.8), you should see something like:
Code:
Python 2.7.8 (default, Dec 2 2014, 05:15:18)
[GCC 4.9.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
***I haven't fully tested this yet, so let me know how it goes. I know basic commands work, but there are still some complexities in the compilation that may need to be figured out.***
(update) -- Install Extra Packages/Modules via "easy_install" -- (root required)
1) In a shell, remount /system read-writable with:
Code:
mount -o remount,rw /system
2) Change directory to /system/pythonX.X.X, i.e.,
Code:
cd /system/pythonX.X.X
3) Execute easy_install followed by a package name, i.e.,
Code:
./easy_install [i]package_name[/i]
4) Let it download and install! Then test it out.
5) Remount /system read-only with:
Code:
mount -o remount,ro /system
Also, to remove a package, execute:
Code:
./easy_install -m [i]package_name[/i]
rm -r ../lib/pythonX.X/site-packages/[i]package_name[/i]*.egg
*** Flashing the resolv.conf patch may be required to make an internet connection to download modules.
Also, a few things need to be done to get easy_install working:
1) For python 2.7.8 or 2.7.9 you need to link python2.7 to python like this:
Code:
ln -s /system/python2.7.9/bin/python2.7 /system/python2.7.9/bin/python
I'll make sure this is automatically done in future installs.
2) SSL certificates need to be added to "/etc/pki/tls/certs/" to use SSL with easy_install.
Code:
mkdir -p /etc/pki/tls/certs
curl http://curl.haxx.se/ca/cacert.pem -o /etc/pki/tls/certs/ca-bundle.crt
ISSUES
The python installations have the following modules compiled in with external dependencies such as libraries not included at this point:
Code:
--------------PYTHON 2.7.8-2.7.9----------------------
_hashlib _multiprocessing _ssl
_testcapi bz2 crypt
dbm
Code:
--------------PYTHON 3.4.2-----------------------------
_crypt _dbm _decimal
_hashlib _multiprocessing _ssl
_testcapi
I will look into this more.
As @bubbleguuum points out, name resolution with python 3.4.2's urllib wasn't working. I found a workaround by adding the line
Code:
options single-request-reopen
to /system/etc/resolv.conf. This still needs some testing but seemed to work for me. I've included a recovery flash zip to patch /system/etc/resolv.conf if the line is not there since this is a root operation.
UPDATES
* 2014-12-02 * Compiled in more modules for each installation. This makes the installation a bit bigger, but it's worth it. You get a more complete python! To get stuff like help functions and math and readline modules, you need the larger installations linked below under "DOWNLOADS".
* 2014-12-03 * Thanks @cybojenix for pointing out some incapatibility issues with the original edify installer I was using. I have now updated the installers to use SuperSU's great non-edify sh-script installer.
* 2014-12-05 * Some updates to the installers to include more modules, plus size reduction for python 3.4.2. Modules added to python 2.7.8 : _bsddb _ctypes, _ctypes_test, _hotshot, _json, _lsprof, _sqlite3, future_builtins, and ossaudiodev, easy_install, pip(broken for now). Modules added to python 3.4.2: _bz2, _ctypes, _ctypes_test, _gdbm, _json, _lsprof, _opcode, _sqlite3, _testbuffer, _testimportmultiple, ossaudiodev, xxlimited
* 2014-12-05 * Some big updates to the installer to include almost the rest of the modules that wouldn't compile. Also easy_install is working on my end with these new experimental builds. With a little effort, should be able to get pip working as well. I had to use some hacky flags to get it to compile, ignore unresolved-symbols and such, so it definitely needs some testing. Another thing I did was tweak the install script to write over previous installations from these installations. I recommend downloading the experimental builds if you read this. The worst thing that might happen is one of the new modules I've included won't work completely.
* 2014-12-14 * Added python 2.7.9. Trimmed the installation sizes down to the much more reasonable 41MB, 43MB, and 49MB respectively for python 2.7.8, python 2.7.9, and python 3.4.2.
* 2014-12-24 * Attached a recovery flash.zip to patch resolv.conf to try and fix a urllib name resolution error.
* 2014-12-26 * A couple updates: Tweaked python installations so that modules looks for "/system/bin/sh" instead of "/bin/sh" when needed. Also a few "#!" corrections for scripts in the python bin directory. This shouldn't break anything from what I've tested, only make Android python more capable, but I'll keep the old installations attached for now just in case since the modifications involved using "sed". The second update is nice. I've added a little section on installing extra modules with easy_install(included and working with all installations).
Compiling
The environment I compiled in was a debian "wheezy" image mounted as a loop device on Android. The reason I compile this way is so I have full access to the arm environment tools, gcc compiler, etc. without having to go through the trouble of using a cross-compiler (which doesn't work in this case from my testing with uClibc, or eglibc. Code sourcery's might work but I didn't feel like booting an x86 linux installation to try). You can recreate the same environment pretty easily by using either Complete Linux Installer or Linux Deploy apps. I use both.
To get a static python compiled, I first downloaded the following packages with APT:
Code:
[i]apt-get build-dep python python3[/i]
build-essential gcc make # building tools
binutils-gold # awesome gold linker
zlib1g-dev # zlib
libreadline-dev, libncurses5-dev # readline, curses
libbz2-dev # bz2
libsqlite3-dev # sqlite3
python-bsddb3, python3-bsddb3
libgdbm-dev # gdb
libssl-dev # ssl
python-tk, python3-tk # tkinter
libdb-dev # db
python-gdbm python-bsddb3
libffi-dev # _ctypes
tcl8.6-dev # tkinter
libx11-dev # tkinter
libmpdec-dev # decimal
Then I manually installed binutils-gold by unpacking the rpm for armv7h (this isn't available with APT yet). You can get it here though. (Update: this should be available in the debian repositories now.)
For static compilation, you need to make some changes to Modules/Setup from the python source directory. Specifically, you need to add
Code:
*static*
to the top of the file. Then you need to uncomment any commented modules you would like compiled in. When you compile and see a list of failed modules, search for each of these in Modules/Setup and uncomment them. For example, change
Code:
#math mathmodule.c _math.c
to
Code:
math mathmodule.c _math.c
. Then recompile. There may be some shared dependencies and packages as well you need to figure out to get them to compile. Take a look at the attached "Setup" files for more detail.
The actually compile steps look like this:
(1)
Code:
./configure --build=arm --prefix="$PWD"/out LDFLAGS="-static -static-libgcc -Wl,--unresolved-symbols=ignore-all -Wl,--export-dynamic" CPPFLAGS=-static CXXFLAGS=-static CFLAGS="-Os -static" LDFLAGS=-static LD=ld.gold
(2) Modify the Modules/Setup file.
(3)
Code:
make clean; make install
After compiling, there are some things you can do to make your installation in "./out" smaller. Strip the large binaries in "out/bin", i.e.
Code:
strip -s ./out/bin/python2.7
Also remove the python archive library, *.o's, etc. with:
Code:
make clean
This might seem kind of weird, but it works and cleans out unnecessary files from your installation.
**Make sure to backup your Modules/Setup file if you run "make distclean". I lost my 3.4.2 Modules/Setup file after doing this. :silly: Now I gotta recreate it, dammit.
**I should also mention that compile time is very fast given the amount of data generated. It takes about 10 minutes to get python and all it's modules compiled on my Galaxy Nexus.**
:EDITS:
: Updated some configure parameters. Removed unnecessary, extra "./configure". Also added some updates to the included "Setup" files.
: Hacky update to "./conifgure" to ignore unresolved symbols, export dynamic. This allows certain modules to be compiled which have shared dependencies.
: Update to trimming down the installation size after "make". Use another "make clean".
Downloads - Extract in the source directory
Setup (python 2.7.8)
Setup (python 3.4.2)
Setup (Experimental) (python 2.7.8)
Setup (Experimental) (python 3.4.2)
Something that would be interesting to look in to would be to have pip/easy_install running, so you can install extra non c packages onto the sdcard/data/wherever.
Would you also consider opening up the sources please? There is a project I'm interested in doing involving python on Android, however a lack of time meant I couldn't finish building it.
Give me a shout if you need anything. I hope to see even more come out of this thread
Edit:
I've just looked at the updater-script. The mount command won't work on the majority of devices. Can you turn the update-binary in to a shell script please? See the SuperSU updater for reference
cybojenix said:
Something that would be interesting to look in to would be to have pip/easy_install running, so you can install extra non c packages onto the sdcard/data/wherever.
Would you also consider opening up the sources please? There is a project I'm interested in doing involving python on Android, however a lack of time meant I couldn't finish building it.
Give me a shout if you need anything. I hope to see even more come out of this thread
Edit:
I've just looked at the updater-script. The mount command won't work on the majority of devices. Can you turn the update-binary in to a shell script please? See the SuperSU updater for reference
Click to expand...
Click to collapse
Thanks for your reply! I've changed the installer to use SuperSU's no-edify sh-scripted updater. I tested on my device, but let me know if there are any issues though. I kept it pretty basic.
I added some stuff on the source modifications and compilation to the 2nd post. The only file I've actually modified so far in the source is the generated Modules/Setup file, and I've included the one I used for python 2.7.8. Unfortunately, I over-cleaned the python 3.4.2 directory, which deleted my modified Modules/Setup there, so I'll have to add it later when I re-edit it.
That's a great idea on pip/easy_install. I did download the pip_installer and tried out installing on the static python I made. Got some errors though pointing to a few modules I haven't gotten compiled into the static python installation (listed in the OP near the bottom, specifically _ctypes). I'll see what I can do about that. The line for compiling the module might just need to be added to Modules/Setup. Hopefully that's the case.
@7175
Thank you very much for these binaries.
There is however a problem: name resolution (DNS) doesn't seem to work at all (with both python 2 and 3 downloads).
All attempts to use urllib.request.urlopen('http://somehost.com') fail with "<urlopen error [Errno -2] Name or service not known>" (running python as root but it doesn't matter, and a rooted Nexus 4 running 4.4.4 stock ROM):
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 153, in urlopen
return opener.open(url, data, timeout)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 455, in open
response = self._open(req, data)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 473, in _open
'_open', req)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 433, in _call_chain
result = func(*args)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 1202, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 1176, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>
Click to expand...
Click to collapse
it works if replacing somehost.com by its ip address.
To reproduce above stack trace with python 3:
import urllib.request
urllib.request.urlopen('http://www.google.com')
while this work:
urllib.request.urlopen('http://173.194.45.229')
nslookup and general name resolution work anywhere else:
[email protected]:/ # nslookup www.google.com
nslookup wwwgoogle.com
Server: 8.8.4.4
Address 1: 8.8.4.4 google-public-dns-b.google.com
Name: wwwgoogle.com
Address 1: 2a00:1450:4007:80c::1014 par03s15-in-x14.1e100.net
Address 2: 173.194.45.242 par03s15-in-f18.1e100.net
Address 3: 173.194.45.240 par03s15-in-f16.1e100.net
Address 4: 173.194.45.244 par03s15-in-f20.1e100.net
Address 5: 173.194.45.241 par03s15-in-f17.1e100.net
Address 6: 173.194.45.243 par03s15-in-f19.1e100.net
Click to expand...
Click to collapse
This issue has probably something to do with how python was compiled to do name resolution. I remember vaguely a similar issue
on Ubuntu 12.04, with a ffmpeg compile that would always fail to resolve hostnames in URLs. No sure what the exact fix was but it had something to do with the libc and its name resolution mechanism. Or maybe something ipv6 related ?
@bubbleguuum : Thanks for checking out that important issue.
I did a little research and it seems there is some issue with DNS name resolution related to ipv6. I recompiled with "--disable-ipv6" and that seemed to fix the issue. I also found that adding the line
Code:
options single-request-reopen
to /etc/resolv.conf did the trick as well. Apparently this tells the resolver to use a new socket for ipv6 resolution instead of same one as ipv4. It thereby reduces wait-time as well. Maybe check on your end and see if that works. Otherwise I'll upload ipv4-only installations since this issue at least needs to be side-stepped.
EDIT: Hmm, looks like re-compiling with "--disable-ipv6" didn't fix the problem. Will have to look more into this issue and hold off on uploading ipv4-only python3 since it didn't fix it. In the meantime, I've attached a recovery flash.zip to the OP to add the above mentioned line to /system/etc/resolv.conf if needed, which seems to fix the issue for me.
7175 said:
@bubbleguuum : Thanks for checking out that important issue.
I did a little research and it seems there is some issue with DNS name resolution related to ipv6. I recompiled with "--disable-ipv6" and that seemed to fix the issue. I also found that adding the line
Code:
options single-request-reopen
to /etc/resolv.conf did the trick as well. Apparently this tells the resolver to use a new socket for ipv6 resolution instead of same one as ipv4. It thereby reduces wait-time as well. Maybe check on your end and see if that works. Otherwise I'll upload ipv4-only installations since this issue at least needs to be side-stepped.
Click to expand...
Click to collapse
Thank you for looking into this so fast.
My intended usage is for running the python binary from a regular non-root app (which works great!), so editing resolv.conf is not an option.
I've been researching that DNS resolving issue and could not find a clear explanation on why it fails on Android and not on other systems, and why exactly the added line in resolv.conf fixes (or rather workaround) it.
All seems to point to socket.getaddrinfo() failing for an unknown reason.
Is Python compiled with HAVE_GETADDRINFO defined ? If that's the case it uses the libc definition of getaddrinfo, otherwise
it uses an emulation function found in getaddrinfo.c.
My guess is that it is the latter, and what is causing this issue on Android.
It's probable this issue has already been solved on Android since other Python binaries exists, but Googling around do not give
much answer.
I get this error when trying to install anything with easy_install (using /system/python3.4.2/bin/easy_install-3.4 because /system/python3.4.2/easy_install doesn't seem to exist like in the OP):
Code:
# ./easy_install-3.4 feedparser
Searching for feedparser
Reading https://pypi.python.org/simple/feedparser/
Download error on https://pypi.python.org/simple/feedparser/: [Errno -2] Name or service not known -- Some packages may not be found!
Couldn't find index page for 'feedparser' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Download error on https://pypi.python.org/simple/: [Errno -2] Name or service not known -- Some packages may not be found!
No local packages or download links found for feedparser
error: Could not find suitable distribution for Requirement.parse('feedparser')
i got curl and the certs installed in /etc/pki/tls/certs/ as per instrucions in the OP, any help?
Thoughts on PIL?
Hmm... new problems here. Android Lollipop have no /system/etc/resolv.conf by-default and urllib patch is no more works.
How it can be fixed?
http://qpython.net/index.php
Where I can find source code and related documents for project
Pratik Raj said:
Where I can find source code and related documents for project
Click to expand...
Click to collapse
I would like to know the same, as well as what license it is under... Also, I don't care about the network, but does it work on Oreo? Also, do you think I could customise the installer to install to a non-system location (/tmp) so that my flashable zip can use python?
hackintosh5 said:
I would like to know the same, as well as what license it is under... Also, I don't care about the network, but does it work on Oreo? Also, do you think I could customise the installer to install to a non-system location (/tmp) so that my flashable zip can use python?
Click to expand...
Click to collapse
Idk about OP, but I can surely post my source for python 2.7 on arm/arm64. I compiled it like a year ago, but it wasn't too bad, just required a bunch of patches and code for dlopen
I just want to say thank you so much.
APK-Patcher -- Flashable Zip Template for Modifying System APKs On-Device from Recovery
APK modification can be a tedious task. If you want to provide modified APKs you copy it to PC to decompile using Java applets like baksmali and apktool, make changes and recompile using another Java applet, smali (or apktool again). Then there's the pitfall of different recompiled APKs being required for the various different ROMs, etc., not to mention across Android versions. So, considering Android runs a version of Java runtime and the primary tools are Java-based I wondered why we couldn't do all of this on-device and on-the-fly so modifications could be applied via recovery, to whatever ROM APK was present, and take the constant update burden off of the modification creator.
I have taken these commandline Java applets, dexed them to allow them to run on-device, and found/built static ARM compiles of zip, zipalign and aapt to run on the various Android versions and devices. These binaries would need to be recompiled for other architecture support. The zip is smart and automated, using the APK name to run all the various parts involved for complicated patches and is extensible to any number of APKs in a single zip. The modification creator needs to only add the APK name(s) to a list, figure out how to make the changes universally (i.e. sed, awk, etc.) then add the commands to scripts of the same name, and add any whole-file additions/changes in subdirectories of the same name.
A proof-of-concept working script automating the long known Facebook Contacts Sync modifications to ContactsProvider.apk is included for reference.
My development work on my many projects comes out of my free time, so if you enjoy this project or anything else I've done on xda, please consider sponsoring my ongoing work using my GitHub Sponsors profile. For a one-time donation you can hit the donate link from my profile. Thank you for your support!
Source: https://github.com/osm0sis/APK-Patcher/
Status: No Longer Updated
Instructions
1a) Place any required script to alter the decompiled APK classes.dex smali files in /script as a -smali.sh file with the name of the APK
-b) Place any required/additional updated whole smali files in /patch under the name of the APK with "-smali" and the relative path within the classes.dex file (including com)
2a) Place any required script to alter the decoded APK resources.asrc files in /script as a -res.sh file with the name of the APK
-b) Place any required/additional updated whole resource files in /patch under the name of the APK with "-res" and the relative path within the zip/resources.asrc file (including res)
3) Modify the envvar.sh to add your banner, apklist, backup and cleanup options
4) Modify the extracmd.sh to add any additional commands to be performed at the end of the patching process that aren't patch-related (/data file changes, etc.)
5) zip -r9 UPDATE-APK-Patcher.zip * -x README UPDATE-APK-Patcher.zip
As a general rule, whole-file adding is best used only for including a file that doesn't already exist, and the more surgical script-work should be used to keep things more universal.
If supporting a recovery that forces zip signature verification (like Cyanogen Recovery) then you will need to also sign your zip using the method I describe here:
[DEV][TEMPLATE] Complete Shell Script Flashable Zip Replacement + Signing [SCRIPT]
Enjoy!
Questions, comments and feedback welcome.
Credits & Thanks: JesusFreke for bak/smali, iBotPeaches for apktool, Surge1223 for help working out how to build zipalign and aapt, and all authors of the included binaries and those who ported them over for their amazing work.
Disclaimer: Naturally, you take all the responsibility for what happens to your device when you start messing around with things.
Script/Patch Reference
Properties / Variables (envvar.sh)
Code:
banner="Facebook Contacts Sync Enabler";
apklist="ContactsProvider.apk";
apkbak=/data/media/APK-Backup;
backup=1;
cleanup=1;
banner is the name of your patch zip, usually suggestive of what it does, to be displayed at the beginning of the zip flash. You should include your name/handle here like "by osm0sis @ xda-developers" for credit purposes.
apklist is a string containing the list of APKs to be patched included in the patch zip, separated by spaces between the quotes. Each APK is automatically found recursively in /system, then copied to the working directory to be decompiled and acted on, then copied back to /system.
apkbak is the location to place backups of the untouched APKs in apklist if backup=1 is set.
backup=1 will store backups of the untouched APKs in the location specified in apkbak.
cleanup=0 will keep the zip from removing it's working directory in /tmp/apkpatcher or any of the files resulting from the smali process - this can be useful if trying to debug in adb shell whether the patches worked correctly. cleanup=1 is necessary on multi-APK patching zips to clean the baksmali classout directory for the next APK, so it's recommended each APK to be patched be tested on their own with cleanup=0 before combining into a single zip.
osm0sis said:
Properties / Variables (envvar.sh)
Code:
banner="Facebook Contacts Sync Enabler";
apklist="ContactsProvider.apk";
apkbak=/data/media/APK-Backup;
backup=1;
cleanup=1;
banner is the name of your patch zip, usually suggestive of what it does, to be displayed at the beginning of the zip flash. You should include your name/handle here like "by osm0sis @ xda-developers" for credit purposes.
apklist is a string containing the list of APKs to be patched included in the patch zip, separated by spaces between the quotes. Each APK is automatically found recursively in /system, then copied to the working directory to be decompiled and acted on, then copied back to /system.
apkbak is the location to place backups of the untouched APKs in apklist if backup=1 is set.
backup=1 will store backups of the untouched APKs in the location specified in apkbak.
cleanup=0 will keep the zip from removing it's working directory in /tmp/apkpatcher or any of the files resulting from the smali process - this can be useful if trying to debug in adb shell whether the patches worked correctly. cleanup=1 is necessary on multi-APK patching zips to clean the baksmali classout directory for the next APK, so it's recommended each APK to be patched be tested on their own with cleanup=0 before combining into a single zip.
Click to expand...
Click to collapse
Did you happen to figure out how to cross compile jdk for arm? (different from the arm sources available from oracle etc)
..and nice work!
Surge1223 said:
Did you happen to figure out how to cross compile jdk for arm? (different from the arm sources available from oracle etc)
..and nice work!
Click to expand...
Click to collapse
Nah, just forced bak/smali and apktool to run by building/adding classes.dex for each of them and figuring out how to call a functional dalvikvm from recovery.
The first part was mentioned in passing (and quite awesomely) by JesusFreke back in 2011.
That got it running in booted Android. Then running from recovery was a whole other issue, but I stumbled upon Jim Huang's work on executing dalvikvm on Linux, and that lead me to figuring out a solution.
As it stands, apktool should technically be able to do both the classes and resources parts of things (since it incorporates baksmali and smali), but for some reason it bails in recovery on decompiling the classes.dex where baksmali standalone doesn't. So perhaps in future revisions apktool could be the only jar required.
Similarly, apktool also bails when rebuilding resources.asrc because it can't call its internal(?) aapt, so I've included aapt compiled as PIE since that's the only working one I could find; in the future this could go too if apktool resolves it, or at the very least it would be more universal if we had a static compile of aapt to go with the other static binaries included.
Other possible items on the to-do list would include adding zip re-signing (currently it uses the apktool /system APK trick of keeping META-INF the same), reflashing to revert to the backup APKs, and adding busybox to always have an expected execution environment like I recently did on AnyKernel2 to work around Cyanogen Recovery and toybox missing basic stuff like awk at the moment. These aren't something I'm going to be actively working on since it works and APK modding isn't really a focus for me, but since I figured out how to automate the process I wanted to provide the WIP template for the modding community. :good:
osm0sis said:
Nah, just forced bak/smali and apktool to run by building/adding classes.dex for each of them and figuring out how to call a functional dalvikvm from recovery.
The first part was mentioned in passing (and quite awesomely) by JesusFreke back in 2011:
http://forum.xda-developers.com/showpost.php?p=10424700&postcount=2
That got it running in booted Android. Then running from recovery was a whole other issue, but I stumbled upon Jim Huang's work on executing dalvikvm on Linux, and that lead me to figuring out a solution.
As it stands, apktool should technically be able to do both the classes and resources parts of things (since it incorporates baksmali and smali), but for some reason it bails in recovery on decompiling the classes.dex where baksmali standalone doesn't. So perhaps in future revisions apktool could be the only jar required.
Similarly, apktool bails when rebuilding resources.asrc because it can't call its internal(?) aapt, so I've included aapt compiled as PIE since that's the only working one I could find; in the future this could go too if apktool resolves it or at the very least it would be more universal if we had a static compile of aapt to go with the other static binaries included.
Other possible items on the to-do list would include adding zip re-signing (currently it uses the apktool /system APK trick of keeping META-INF the same), and adding busybox to always have an expected execution environment like I recently did on AnyKernel2 to work around Cyanogen Recovery and toybox missing basic stuff like awk. These aren't something I'm going to be actively working on since it works and APK modding isn't really a focus for me, but since I figured out how to automate the process I wanted to provide the WIP template for the modding community. :good:
Click to expand...
Click to collapse
Nice! I tried to compile aapt from my device a few mins ago just for s&g's got pretty far actually before it couldn't compile some external android libs because I didn't have them downloaded.
Have you ever seen the Android apktool apk by pqy330? It's not on git anymore for some reason, but the guy figured out how to get java, javac, aapt, aidl, and everything working on arm, only required a LD_LIBRARY_PATH export. I wish I could get a hold of the guy to ask him about getting some of those tougher arm static binaries compiled.
And yeah I wouldn't actively spend hours working on this if it's already in a working state like you said, it's just interesting compared to the other mundane things I read of xda lol. Especially new developments. Great job man.
APK-Patcher: minor fixes, binary updates:
- allow for recoveries without busybox
- update tool compiles to my newer zipalign and static(!) aapt builds
- strip zip binary to compensate for larger aapt and zipalign binaries
- simplify aapt usage now that we have a static arm binary
- configure zipalign for Marshmallow system APK handling
https://github.com/osm0sis/APK-Patcher/commit/af24805f06e2e2b92f2bfeeb1490d6022073c4b5
Thanks @Surge1223 for the help with figuring out compiling zipalign and aapt statically for arm. :highfive:
Wow, i wish that i know this patcher before
so im trying to add two lines after line #24 in "res/xml/tuner_prefs.xml"
i tried
Code:
sed -i '24 i\> <Preference android:title="@string/nav_bar" android:key="nav_bar" android:fragment="com.android.systemui.tuner.NavBarTuner" />' res/xml/tuner_prefs.xml;
sed -i '25 i\> <Preference android:title="@string/color_and_appearance" android:key="color_transform" android:fragment="com.android.systemui.tuner.ColorAndAppearanceFragment" />' res/xml/tuner_prefs.xml;
and this also
Code:
sed -i '24 i\> <Preference android:title="@string/nav_bar" android:key="nav_bar" android:fragment="com.android.systemui.tuner.NavBarTuner" />' res/xml/tuner_prefs.xml;
sed -i '25 i\> <Preference android:title="@string/color_and_appearance" android:key="color_transform" android:fragment="com.android.systemui.tuner.ColorAndAppearanceFragment" />' res/xml/tuner_prefs.xml;
and it doesn't work, im really sorry i never used sed commands before
do i have to add original file in "patch/SystemUI/res/xml" and then the script should change it ?
AL_IRAQI said:
Wow, i wish that i know this patcher before
so im trying to add two lines after line #24 in "res/xml/tuner_prefs.xml"
i tried
Code:
sed -i '24 i\> <Preference android:title="@string/nav_bar" android:key="nav_bar" android:fragment="com.android.systemui.tuner.NavBarTuner" />' res/xml/tuner_prefs.xml;
sed -i '25 i\> <Preference android:title="@string/color_and_appearance" android:key="color_transform" android:fragment="com.android.systemui.tuner.ColorAndAppearanceFragment" />' res/xml/tuner_prefs.xml;
and this also
Code:
sed -i '24 i\> <Preference android:title="@string/nav_bar" android:key="nav_bar" android:fragment="com.android.systemui.tuner.NavBarTuner" />' res/xml/tuner_prefs.xml;
sed -i '25 i\> <Preference android:title="@string/color_and_appearance" android:key="color_transform" android:fragment="com.android.systemui.tuner.ColorAndAppearanceFragment" />' res/xml/tuner_prefs.xml;
and it doesn't work, im really sorry i never used sed commands before
do i have to add original file in "patch/SystemUI/res/xml" and then the script should change it ?
Click to expand...
Click to collapse
Nah, per the OP, the "patch" directory is only for replacing files whole, which you shouldn't do to keep it universal.
You just need a working sed command in the script directory.
You can borrow this script function from my AnyKernel2 to make the insertion easier for you as a sed beginner:
Code:
# insert_line <file> <if search string> <before|after> <line match string> <inserted line>
insert_line() {
if [ -z "$(grep "$2" $1)" ]; then
case $3 in
before) offset=0;;
after) offset=1;;
esac;
line=$((`grep -n "$4" $1 | head -n1 | cut -d: -f1` + offset));
sed -i "${line}s;^;${5}\n;" $1;
fi;
}
Then follow that with an insert_line command using the syntax provided. "if search string" tests whether the mod has been completed already, then "line match string" is the line you want to put the inserted lines before/after.
You can insert both lines at once with a \n between as well.
osm0sis said:
Nah, per the OP, the "patch" directory is only for replacing files whole, which you shouldn't do to keep it universal.
You just need a working sed command in the script directory.
You can borrow this script function from my AnyKernel2 to make the insertion easier for you as a sed beginner:
Code:
# insert_line <file> <if search string> <before|after> <line match string> <inserted line>
insert_line() {
if [ -z "$(grep "$2" $1)" ]; then
case $3 in
before) offset=0;;
after) offset=1;;
esac;
line=$((`grep -n "$4" $1 | head -n1 | cut -d: -f1` + offset));
sed -i "${line}s;^;${5}\n;" $1;
fi;
}
Then follow that with an insert_line command using the syntax provided. "if search string" tests whether the mod has been completed already, then "line match string" is the line you want to put the inserted lines before/after.
You can insert both lines at once with a \n between as well.
Click to expand...
Click to collapse
im such a noob, even this basic code is hard for me to understand :crying:
do i have to keep grep commands ?, because i dont need to replace or find something in the file, only inserting after line #24
$1 is the file here right ?, and your trying to find $2 and add someting, and in case of 3$ there is another case, and edits will probably be before/after $4 ?
AL_IRAQI said:
im such a noob, even this basic code is hard for me to understand :crying:
do i have to keep grep commands ?, because i dont need to replace or find something in the file, only inserting after line #24
$1 is the file here right ?, and your trying to find $2 and add someting, and in case of 3$ there is another case, and edits will probably be before/after $4 ?
Click to expand...
Click to collapse
What is on line 24? Wouldn't it be more universal to match against some part of the contents of the current line 24 to ensure its always in the right place, even if the number changes?
The $1 $2 $3 $4 $5 correspond to each argument supplied as per the <syntax> in the #commented line.
osm0sis said:
What is on line 24? Wouldn't it be more universal to match against some part of the contents of the current line 24 to ensure its always in the right place, even if the number changes?
The $1 $2 $3 $4 $5 correspond to each argument supplied as per the <syntax> in the #commented line.
Click to expand...
Click to collapse
i modded systemui for N5X,N6/6P,N9 and android one device, all of them has the same layout
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
or i can search for "<Preference android:title="@string/other"" and add it before it
oh, now its easier to understand ;P
Okay, so.. Probably:
Code:
insert_line res/xml/tuner_prefs.xml "com.android.systemui.tuner.NavBarTuner" before "com.android.systemui.tuner.OtherPrefs" "<Preference android:title=\"@string/nav_bar\" android:key=\"nav_bar\" android:fragment=\"com.android.systemui.tuner.NavBarTuner\" />\n<Preference android:title=\"@string/color_and_appearance\" android:key=\"color_transform\" android:fragment=\"com.android.systemui.tuner.ColorAndAppearanceFragment\" />";
osm0sis said:
Okay, so.. Probably:
Code:
insert_line res/xml/tuner_prefs.xml "com.android.systemui.tuner.NavBarTuner" before "com.android.systemui.tuner.OtherPrefs" "<Preference android:title=\"@string/nav_bar\" android:key=\"nav_bar\" android:fragment=\"com.android.systemui.tuner.NavBarTuner\" />\n<Preference android:title=\"@string/color_and_appearance\" android:key=\"color_transform\" android:fragment=\"com.android.systemui.tuner.ColorAndAppearanceFragment\" />";
Click to expand...
Click to collapse
i removed /patch and added the script to SystemUI-res.sh, but it doesn't seems to work
screenshot:
AL_IRAQI said:
i removed /patch and added the script to SystemUI-res.sh, but it doesn't seems to work
screenshot:
Click to expand...
Click to collapse
You shouldn't see that message for what you're doing unless you've got things set up weird. Can you just attach your WIP zip and I'll take a look?
osm0sis said:
You shouldn't see that message for what you're doing unless you've got things set up weird. Can you just attach your WIP zip and I'll take a look?
Click to expand...
Click to collapse
here you are, link
Thanks
osm0sis said:
You shouldn't see that message for what you're doing unless you've got things set up weird. Can you just attach your WIP zip and I'll take a look?
Click to expand...
Click to collapse
AL_IRAQI said:
here you are, link
Thanks
Click to expand...
Click to collapse
Okay, try this. I've got you about 90% of the way there; fixed all the weirdness with the unnecessary patch folder, extra files and extracmd. The sed insertion is still untested, so you'll have to check that out, and from what you and others have told me you'll need to update the included tools for Nougat support.
Download the newer bak/smali and apktool, then that's done with
Code:
dx --dex --output=classes.dex <file>.jar
from within the latest SDK. Add the classes.dex to the jar (it's a zip). Name them like the current ones (version numbers, etc.) and put them in tools and remove the old ones. :good:
[ Attachment removed. ]
osm0sis said:
Okay, try this. I've got you about 90% of the way there; fixed all the weirdness with the unnecessary patch folder, extra files and extracmd. The sed insertion is still untested, so you'll have to check that out, and from what you've told me you'll need to update the included tools for Nougat support.
Download the newer bak/smali and apktool, then that's done with
Code:
dx --dex --output=classes.dex <file>.jar
from within the latest NDK. Add the classes.dex to the jar (it's a zip). Name them like the current ones (version numbers, etc.) and put them in tools and remove the old ones. :good:
Click to expand...
Click to collapse
Awesome, ill try it today :fingers-crossed:
Edit:
i already downloaded newer tools, and now i have to run this code ?
Code:
dx --dex --output=classes.dex apktool_2.0.3.jar
dx --dex --output=classes.dex baksmali-2.1.1.jar
dx --dex --output=classes.dex smali-2.1.1.jar
AL_IRAQI said:
Awesome, ill try it today :fingers-crossed:
Edit:
i already downloaded newer tools, and now i have to run this code ?
Code:
dx --dex --output=classes.dex apktool_2.0.3.jar
dx --dex --output=classes.dex baksmali-2.1.1.jar
dx --dex --output=classes.dex smali-2.1.1.jar
Click to expand...
Click to collapse
@AL_IRAQI, sorry, just saw your edit. The latest bak/smali is 2.1.3, and latest apktool is 2.2.0, but otherwise yes, you run that command on each jar separately from the build-tools directory of the latest Android SDK, and put the resulting classes.dex inside the jar (which is a zip file). Then it's able to be run on Android as in my zip.
New commit!
tools: binary updates:
https://github.com/osm0sis/APK-Patcher/commit/e4f6e3e90e13c862652183460a86e49cb0514fd9
Since I felt bad @AL_IRAQI didn't appear to be able to figure it out for his mods. :angel:
From an incorrect thread:
aj20010319 said:
Can I combine APK Patcher with aroma? Like use the standard updater-binary and adding all these functions from updater-binary into a different shell script and executing it later?
Click to expand...
Click to collapse
aj20010319 said:
Okay so I somehow managed to get aroma and apk patcher together, made separate zips and then made one zip flash the other.
I am trying to decompile framework-res.apk, but it's failing, logs show this error,
Code:
framework-res.apk
I: Using Apktool 2.2.0 on framework-res.apk
I: Loading resource table...
I: Decoding AndroidManifest.xml with resources...
I: Regular manifest package...
I: Decoding file-resources...
java.lang.NoClassDefFoundError: Failed resolution of: Ljavax/imageio/ImageIO;
at brut.androlib.res.decoder.Res9patchStreamDecoder.decode(Res9patchStreamDecoder.java:39)
at brut.androlib.res.decoder.ResStreamDecoderContainer.decode(ResStreamDecoderContainer.java:33)
at brut.androlib.res.decoder.ResFileDecoder.decode(ResFileDecoder.java:120)
at brut.androlib.res.decoder.ResFileDecoder.decode(ResFileDecoder.java:87)
at brut.androlib.res.AndrolibResources.decode(AndrolibResources.java:262)
at brut.androlib.Androlib.decodeResourcesFull(Androlib.java:131)
at brut.androlib.ApkDecoder.decode(ApkDecoder.java:108)
at brut.apktool.Main.cmdDecode(Main.java:163)
at brut.apktool.Main.main(Main.java:81)
Caused by: java.lang.ClassNotFoundException: Didn't find class "javax.imageio.ImageIO" on path: DexPathList[[zip file "/tmp/apkpatcher/tools/apktool_2.2.0-dexed.jar"],nativeLibraryDirectories=[/system/lib]]
at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java)
at java.lang.ClassLoader.loadClass(ClassLoader.java)
at java.lang.ClassLoader.loadClass(ClassLoader.java)
... 9 more
Suppressed: java.lang.ClassNotFoundException: javax.imageio.ImageIO
at java.lang.Class.classForName(Native Method)
at java.lang.BootClassLoader.findClass(ClassLoader.java)
at java.lang.BootClassLoader.loadClass(ClassLoader.java)
... 11 more
Caused by: java.lang.NoClassDefFoundError: Class not found using the boot class loader; no stack trace available
Decoding APK resources failed. Aborting...
Click to expand...
Click to collapse
Seems like you're in the wrong thread? I'm not sure. You can try reverting to the previous builds of tools, but beyond that I'm not sure at the moment. It's on my to-do to test everything out again in the near future and resolve any issues I can come up with. Please reply in the APK-Patcher thread.
Now tested up to downloading AOSP and make toolbox you should be all set
Please give thanks to this thread: https://forum.xda-developers.com/newreply.php?do=newreply&p=43622764
Warning: I hacked my way through this stuff a few weeks ago I am not an expert!
How to compile Android Open Source Code modules
I don't compile C code on Windows machines I have no idea about that.
Notice
This guide is a quick and dirty how to make a module. It will not cover finalizing setting up the source codes for your device. It is only my goal to enable you to compile binaries such as grep, toolbox, dumpstate, dalvikvm, jack and etc.
===>] Setup Ubuntu 64bit [<===Unplug that Windows drive, plug in a fresh hard drive and install Ubuntu latest/greatest. Ignore the recommendation to downgrade gnu make!, for now.
Open a terminal and issue these commands (warning ppa repository for OpenJDK 7 is said to have a security issue?, isn't being updated?.. whatevs it works)
Code:
sudo apt-get update
sudo apt-get upgrade
sudo add-apt-repository ppa:openjdk-r/ppa
sudo apt-get install openjdk-7-jdk
sudo apt-get install openjdk-8-jdk
sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386 lib32z1 libbz2-1.0:i386
sudo apt-get install git ccache automake lzop bison gperf build-essential zip curl zlib1g-dev zlib1g-dev:i386 g++-multilib python-networkx libxml2-utils bzip2 libbz2-dev libbz2-1.0 libghc-bzlib-dev squashfs-tools pngcrush schedtool dpkg-dev liblz4-tool make optipng
(choose Java 1.7 in the following way)
Code:
sudo update-alternatives --config java
(let me know if I missed anything please)
"Tried the Android SDK only it is missing too many things we need as a developer"
===>] Setup Android Studio SDK & NDK [<===Installation Paths:
*** I install to /home/username/Android and /home/username/Android/Sdk and /home/username/Android/Sdk/ndk-bundle ***
NOTE: from here forward username will == droidvoider
Note: Android Studio IDE isn't necessary only the SDK & NDK are needed to compile AOSP.
Install Android Studio Proper: (don't worry about setting up paths we will cover that, just install it)
https://developer.android.com/studio/install.html
or
SDK Only:
Typically we install these things manually by creating the directory then just unzipping the files there.
https://developer.android.com/studio/index.html#linux-bundle (scroll down for sdk only)
Code:
mkdir /home/droidvoider/Android
mkdir /home/droidvoider/Android/Sdk
(then unzip the sdk zip to that directory. I recommend the file explorer copy/paste right click uncompress and done.)
https://dl.google.com/android/repository/tools_r25.2.3-linux.zip
Install NDK through the SDK Manger:
(from terminal 'studio.sh' and then configure, and then sdk manger --- if this is hard to figure out tell me I will elaborate)
or
Manually Install Native Development Kit -- 'c programming support'
Download the Native Development Kit from Google: https://developer.android.com/ndk/downloads/index.html
Code:
mkdir /home/droidvoider/Android/Sdk/ndk-bundle
Then just unzip the ndk files into the directory we created above.
===>] Setup your toolchain [<===** This example is arm64-v8a aarch64 **
1. Navigate to /home/droidvoider/Android/Sdk/ndk-bundle/build/tools and then open a terminal "right click open area"
2. mkdir /home/droidvoider/toolchains
3. ./make_standalone_toolchain.py --arch arm64 --api 23 --stl=libc++ --install-dir /home/mm/toolchains/aarch64-linux-android-4.9
4. cd /home/droidvoider
5. gedit .bashrc and morph this in at the bottom.. (AND edit or replace the existing PATH variable)
DON'T just PASTE IN *my* $PATH export!! I included my entire path statements to show you.
Code:
export PATH=$PATH:/usr/local/android-studio/bin:/home/droidvoider/Android/Sdk/platform-tools:/home/droidvoider/Android/Sdk/ndk-bundle:/home/droidvoider/Android/Sdk/tools
I feel this is human readable, for example change Android_Build_Out to be on your desktop instead if you want.
Code:
export PATH=$PATH:/home/droidvoider/toolchains/aarch64-linux-android-4.9
export NDK=/home/droidvoider/Android/Sdk/ndk-bundle
export SYSROOT=$NDK/platforms/android-23/arch-arm64
export TARGET=aarch64-linux-android
export HOST=$TARGET
export BUILD=x86_64-linux
export ANDROID_NDK_BIN=/home/droidvoider/toolchains/aarch64-linux-android-4.9/bin
export CC=$ANDROID_NDK_BIN/aarch64-linux-android-gcc-4.9
export CPP=$ANDROID_NDK_BIN/aarch64-linux-android-g++
export AR=$ANDROID_NDK_BIN/aarch64-linux-android-ar
export OUT_DIR_COMMON_BASE=/home/droidvoider/Android_Build_Out
Note: You might want to setup an alternate toolchain also but this is all of the puzzle pieces.
===>] Google's version of this How To -- Just for reference [<===https://source.android.com/source/requirements.html
https://source.android.com/source/initializing.html
===>] Install the repo tool [<===https://source.android.com/source/downloading.html
(don't type repo init or repo sync --- I will be taking back over from there on the next page)
Added Repair Notes -- Not part of the install!
Have you accidentally installed or removed something you shouldn't have? (welcome to development, here try this before reinstall)
sudo apt-get clean
sudo apt-get update
sudo apt-get install -f
sudo dpkg -a --configure
sudo apt-get dist-upgrade
sudo apt-get install -f
sudo dpkg -a --configure
Selecting the correct AOSP branch and downloading it.
Tested up to downloading AOSP and make toolbox -- you should be all set
===>] Match your build number to it's AOSP sources [<===preface: You can get this from your device if you're on the same build id as your the available source code from your vendor for your device. Otherwise you need to open the AP file from the firmware that matches those available sources to extract the system.img, to extract build.prop. I explain how to open a system.img file below under retrieving your hardware drivers. build.prop is in the main directory of system.img
(Many times the build number is the same. For me I believe all of MM builds are using this number.)
Assumes sources match current device, worked out true in my case
1. Plug in your device and get it connected. (DEVELOPER OPTIONS|USB DEBUGGING) and select allow on device
2. Retrieve the build number that matches the available sources for your device.
From your ubuntu terminal retrieve the build id using this command:
Code:
adb shell getprop | grep 'ro.build.id'
Yields something similar to this: [ro.build.id]: [MMB29K]
3. Match it up to the Nexus build numbers (This info is for AT&T Note 5 Marshmallow MMB29K, get your specific build number!)
https://source.android.com/source/build-numbers.html#source-code-tags-and-builds
MMB29K android-6.0.1_r1 Marshmallow Nexus 5, Nexus 5X, Nexus 6, Nexus 7 (flo/deb), Nexus 9 (volantis/volantisg)
===>] Bring down a specific AOSP source branch [<===
4. Make a directory for the source code.
Code:
mkdir /home/droidvoider/Desktop/AOSP_Android_6.01_r1
5.
Code:
cd /home/droidvoider/Desktop/AOSP_Android_6.01_r1
6. Bring down the sources, this one is approximately 13 gigabytes
Code:
repo init --depth=1 -u https://android.googlesource.com/platform/manifest -b android-6.0.1_r1
repo sync
===>] I'm not sure the rest of this is needed [<===For compiling toolbox the remainder wasn't needed.. But I have a large list of things to do so I can't test each item. If you can't compile a specific module continue reading.
===>] Merge Vendor sources & AOSP sources [<===
7. Download the available sources for your device. In this example I downloaded PE6 Marshmallow sources for AT&T Note 5:
http://opensource.samsung.com/reception/receptionSub.do?method=sub&sub=F&searchValue=SM-N920A
8. Read the readme file from the sources platform zip to understand how to merge them into the AOSP sources. For the 2 Samsungs I've worked with the idea is to replace any directory that already exists. But if there is just one file such as core.mk only replace the one file. Then edit the .mk files as described in your readme. (and/or take info from cyanogen/lineagos) -- <I can help more, ask>
note: you probably don't need to take the configs from LineageOS and put them into your .mk files. However, if you do need to get more configs then you should get a big fat message when you type make 'modulename'. At first only edit .mk files as described by vendor device source readme file.
===>] Merge in Hardware drivers and etc [<===possibly unnecessary depends what you're doing
9. Obtain a copy of the firmware for your device that matches the version of the source code you are able to download from your vendor.
for me that was Build Number: MMB29K.N920AUCU2BPE6 but your mileage will almost certainly vary!
10. Download https://github.com/anestisb/android-simg2img
11. Unzip it right in your download folder, open the folder and then 'open in terminal'
12. Make it and then move it a directory in your path. Warning: My command puts in in the Ubuntu default /bin folder.
Code:
make
sudo mv append2simg img2simg simg2img simg2simg simg_dump.py /bin
13. Uncompress the AP file from the matching firmware and extract the system.img into it's own directory
then select that folder, right click, open in terminal
Code:
simg2img system.img sys.raw
mkdir sys
sudo mount -t ext4 -o loop sys.raw sys/
14. A drive mounted, look on your task bar it should've wiggled too. Copy the etc and vendor folders into the main folder of the sources we are merging
===>] Listing and building modules [<===Navigate to the folder where you download the sources "/home/droidvoider/Desktop/AOSP_Android_6.01_r1" and open in terminal.
Code:
make modules -- list the available modules
make <module name> -- builds a specific module
example: make dumpstate
description: Will build everything needed for dumpstate and place it in the folder we specified in our export (above step). The final build line will read install and detail the final output folder
Example successful output:
[CODE]
Install: /home/droidvoider/Android_Build_Out/Android_6.01_r1/target/product/generic/system/bin/dumpstate
===>] Android Build System, basic intro [<===Notice: I built this how to to answer the same question from 3 people regarding working with toolbox and the dirtycow exploit. So I decided to give a direct example of using toolbox.c from farm-root
#ifdef
Our makefile is Android.mk and that's where we link things together. If you look at the Android.mk file for farm-root you will notice bridge.c is used 3 different times called different 'module' names. bridge_pull, bridge_push, bridge_pull_boot. Each of these will be binaries of those names.
Inside bridge.c you will see #ifdef FARM_PULL and then you will see #else and further you will see #endif which you may have noticed matches inside the Android.mk file for the bridge modules -DFARM_PULL -DFARM_BOOT <== Notice the double define on bridge_pull_boot
toolbox.c
toolbox.c is going to be the same way. You will need to copy shared.h and shared.c into the directory where toolbox.c resides. Then edit the Android.mk, in our example:
1. Navigate to this directory and open: system/core/toolbox/Android.mk
2. CTRL + F and search for "LOCAL_MODULE := toolbox"
3. Add: LOCAL_CFLAGS += -DFARM_PULL -DFARM_BOOT (in this example add one, both or even new ones you created)
4. Navigate to the main directory of the sources, you should see a Makefile and a build_64bit.sh
5. from terminal: make toolbox
Note: I think from here you can Google it out in a few minutes if that is not the case please let me know.
Working with C cross platformUbuntu is Linux based just like Android and this makes testing blocks of code extremely easy. You of course can't use Android headers and in some rare cases you can't test the code on Ubuntu at all but in most cases you can. When I want to design something for Android I open gedit and save it as a .c file. Then I compile it using gcc -o mycode mycode.c There's plenty of examples on using gcc with linux but just understand you can do it all. Then before too much work test it on Android. (helpful commands at end of post)
My advice really is to build out your small blocks of code on your linux box but then paste them into your Android program folder, edit your Android.mk, add it to your Makefile including your 'push' section so that you can simply type make push to test it.
I am in fact trying to encourage you to learn C and not so much trying to encourage you to hack things. But I know that interest/passion is what teaches, not my words and not someone else's curriculum. So in that spirit I will do my best to give examples to help you with 'whatever' it is you are passionate about. Let me know what's missing.
Don't forget to compile for Android first
Before you can test your code you will have compiled it using the cross compiler for Android. ndk-build, or the correct gcc cross compiler. (Personally I put the .c file into a directory with Android.mk and a Makefile then just type make to build it to Android)
see examples section I will add a couple examples.
Android Developer Bridge -- a developers tool
adb is included with the Android SDK along with some other tools. Some of those tools are fastboot for unlocking bootloaders and another way of flashing. There is monitor which is a cool tool for remotely viewing processes, logcat, memory dumps and etc.
But pointedly what we will use the most is simply adb.
Using adb to test your code on locked down Android systems
Shell has fairly high privileges, you may not be aware but you can execute binaries and bash scripts. We use /data/local/tmp/ for these things. You can create a directory, add or remove files, execute your binaries and even execute shell scripts using sh script.sh
ndk-build places the binary in libs/(arch type) .. For a quick test you can just open a terminal in that directory then:
Code:
adb push mybinary /data/local/tmp/
adb shell
cd data/local/tmp
chmod 777 mybinary
./mybinary
Added:
Examples of basic make files for Android.
happy coding
If you get an errorPlease reissue the command but pipe the output to a file.
make toolbox > /home/droidvoider/Desktop/build_toolbox-output.txt
zip that up with your source code, including your customized header files and attach it to this thread.
puzzles are fun but I like all the pieces
droidvoider said:
Tested up to downloading AOSP and make toolbox -- you should be all set
===>] Match your build number to it's AOSP sources [<===preface: You can get this from your device if you're on the same build id as your the available source code from your vendor for your device. Otherwise you need to open the AP file from the firmware that matches those available sources to extract the system.img, to extract build.prop. I explain how to open a system.img file below under retrieving your hardware drivers. build.prop is in the main directory of system.img
(Many times the build number is the same. For me I believe all of MM builds are using this number.)
Assumes sources match current device, worked out true in my case
1. Plug in your device and get it connected. (DEVELOPER OPTIONS|USB DEBUGGING) and select allow on device
2. Retrieve the build number that matches the available sources for your device.
From your ubuntu terminal retrieve the build id using this command:
Code:
adb shell getprop | grep 'ro.build.id'
Yields something similar to this: [ro.build.id]: [MMB29K]
3. Match it up to the Nexus build numbers (This info is for AT&T Note 5 Marshmallow MMB29K, get your specific build number!)
https://source.android.com/source/build-numbers.html#source-code-tags-and-builds
MMB29K android-6.0.1_r1 Marshmallow Nexus 5, Nexus 5X, Nexus 6, Nexus 7 (flo/deb), Nexus 9 (volantis/volantisg)
===>] Bring down a specific AOSP source branch [<===
4. Make a directory for the source code.
Code:
mkdir /home/droidvoider/Desktop/AOSP_Android_6.01_r1
5.
Code:
cd /home/droidvoider/Desktop/AOSP_Android_6.01_r1
6. Bring down the sources, this one is approximately 13 gigabytes
Code:
repo init --depth=1 -u https://android.googlesource.com/platform/manifest -b android-6.0.1_r1
repo sync
===>] I'm not sure the rest of this is needed [<===For compiling toolbox the remainder wasn't needed.. But I have a large list of things to do so I can't test each item. If you can't compile a specific module continue reading.
===>] Merge Vendor sources & AOSP sources [<===
7. Download the available sources for your device. In this example I downloaded PE6 Marshmallow sources for AT&T Note 5:
http://opensource.samsung.com/reception/receptionSub.do?method=sub&sub=F&searchValue=SM-N920A
8. Read the readme file from the sources platform zip to understand how to merge them into the AOSP sources. For the 2 Samsungs I've worked with the idea is to replace any directory that already exists. But if there is just one file such as core.mk only replace the one file. Then edit the .mk files as described in your readme. (and/or take info from cyanogen/lineagos) -- <I can help more, ask>
note: you probably don't need to take the configs from LineageOS and put them into your .mk files. However, if you do need to get more configs then you should get a big fat message when you type make 'modulename'. At first only edit .mk files as described by vendor device source readme file.
===>] Merge in Hardware drivers and etc [<===possibly unnecessary depends what you're doing
9. Obtain a copy of the firmware for your device that matches the version of the source code you are able to download from your vendor.
for me that was Build Number: MMB29K.N920AUCU2BPE6 but your mileage will almost certainly vary!
10. Download https://github.com/anestisb/android-simg2img
11. Unzip it right in your download folder, open the folder and then 'open in terminal'
12. Make it and then move it a directory in your path. Warning: My command puts in in the Ubuntu default /bin folder.
Code:
make
sudo mv append2simg img2simg simg2img simg2simg simg_dump.py /bin
13. Uncompress the AP file from the matching firmware and extract the system.img into it's own directory
then select that folder, right click, open in terminal
Code:
simg2img system.img sys.raw
mkdir sys
sudo mount -t ext4 -o loop sys.raw sys/
14. A drive mounted, look on your task bar it should've wiggled too. Copy the etc and vendor folders into the main folder of the sources we are merging
Click to expand...
Click to collapse
And where is exactly the main folder? Sorry, Im just confused
DigitalDoraemon said:
And where is exactly the main folder? Sorry, Im just confused
Click to expand...
Click to collapse
it's no problem this stuff isn't easy to just figure out on your own. remember to substitute droidvoider for your ubuntu user name
In this example my sources are on my desktop in a folder named Android_6.01_r1
Sources for toolbox for example:
/home/droidvoider/Desktop/Android_6.01_r1/system/core/toolbox/<sources will be here including Android.mk>
Script for modules, including toolbox
/home/droidvoider/Desktop/Android_6.01_r1/Makefile <--- this is our modules script, if you will
<open a terminal in the above folder then use that Makefile like so>
make toolbox <---- this will compile only what is needed to compile the module 'toolbox' (this takes a minute)
Out export folder we decided in ./home/droidvoider/bashrc
/home/droidvoider/Android_Build_Out/Android_6.01_r1/target/product/generic/system/bin
Anybody, please compile grep utility for arm and x86... Minimum Platform Version Android 4.0.3, API Level - 15
Thanks
Great & useful .