Database Info

[SCRIPTS] ROM, kernel, anykernel updater and Environment-setup scripts

Hey guys
I've been working on some scripts to automate my work for quite some time now and thought, why not share it?
So that's what I'm gonna do now
For now, we have three scripts: one to build a ROM, one to compile a kernel, to make an anykernel updater zip and to set up and entire build environment on Arch and Ubuntu-based distros (2nd post). The kernel and anykernel scripts, however, are linked, which means the kernel script automatically executes the anykernel script so as soon as you compile a kernel, you get a recovery flashable zip.
A small note: These scripts were made for the LG Optimus 4x HD, but can easily be modified for any other device.
So, you may ask yourself now 'nice, but what do these script do exactly?' well, that's what I'm gonna tell you now
ROM
The ROM build script can be found HERE
This script is made for CM 11, but can be easily modded to be used with other ROMs. I will also add some more scripts to support more ROMs in the future.
In order to make it compatible with your machine and your device, we have several configurable parameters in the beginning of the script:
Code:
# Configurable parameters
ccache_dir=/home/laufersteppenwolf/ccache/CM11
ccache_log=/home/laufersteppenwolf/ccache/CM11/ccache.log
jobs_sync=30
jobs_build=20
rom=cm
rom_version=11
device_codename=p880
If you want to use a special ccache dir, you can define it using the first parameter. Same for ccache's log.
The "jobs_sync" variable defines how many jobs will be running at the same time while running "repo sync" (---> the -j value)
The "jobs_build" variable defines how many jobs will be used for the make command
"rom" defines the ROM you are trying to compile (used in the lunch command, but also for the zip upload command)
"rom_version" defines the version of the ROM. You can change it to 10.2 or any other version depending on the CM version you are compiling.
Last but not least, the devices codename. In my case it's "p880", but you can just change it to your device (mako, hammerhead, maguro,...) and the lunch as well as the upload command will be changed to your device.
About the upload, if you have a goo.im account and also configured it in your ssh_config, you can just leave it the way it is. However, if you use a different file hoster, please edit the last command to your needs
Furthermore, we have some more features. These, however, don't need to be defined inside the script, but added to the execution command.
You can execute the script with the command:
Code:
. build.sh [-h -n -r -c -ncc -d -j #]
But what do these flags do?
Code:
-h | --help
shows a help message and exits the script
Code:
-n | --nosync
skips the "repo sync" command
Code:
-r | --release
uploads the build when it's done
Code:
-c | --clean
executes "make clean" instead of "make installclean"
Code:
-ncc | --no_ccache
disables ccache
Code:
-d | --debug
shows some debug stuff (not really needed for the "normal" user )
Code:
-j #
sets a custom number of jobs for the "make" command
Example:
Code:
. build.sh -c -r
This command syncs the repos, executes "make clean" instead of "make installclean" and uploads the build to the defined file hoster as soon as the build is done.
Kernel
The kernel build script can be found HERE
This script is made for the p880 (aka x3), but can be easily modded to work with your device.
We have 2 configurable parameters in the beginning of the script:
Code:
defconfig=cyanogenmod_x3_defconfig
jobs=32
"defconfig" defines the name of your defconfig to be used
And "jobs" defines the -j value to be used for make
All modules are being copied to ./out/modules/ and the zImage is being copied to ./out/zImage. Which means no more searching for modules inside the various folders, but having everything in one place
If you take a look into the script, the created out dir is after the above command copied to "~/smb/kernel/out". This is where my network drive is mounted. Please change it to a path of your choice.
This folder now needs to have the files from the anykernel folder inside, because the anykernel build script is being executed next. What this script exactly does will be explained later.
When the anykernel updater zip is done, the script cd's back to the place it was initially executed and will then be done.
If you don't want to make an anykernel zip, just remove the commands after
Code:
find -name '*.ko' | xargs -I {} cp {} ./out/modules/
(well, maybe except for the commands that tell you the script it done )
Anykernel updater zip
Wanna know what an anykernel updater zip is?
The anykernel updater zip only replaces the actual kernel (zImage) and not the whole boot.img. This means the ramdisk stays the same, which makes the kernel compatible with every ROM and Android version.
The script and it's needed tools can be found HERE
This script creates an anykernel updater zip from a given zImage and modules. The zImage has to be in the same folder as the script itself. The modules have to be in a folder called "modules", which also should be located in the same folder as the script. In the beginning of the script, you can define the name of the zip:
Code:
zipname="WWJB_v009_anykernel.zip"
IMPORTANT:
Please make sure to edit the updater-script so it is compatible with your device! Otherwise you can (hard) brick your device!
I am not responsible for possibly bricked devices!
Another feature of the anykernel updater is, you can mod the ramdisk while flashing (repacking the boot.img). This can be done inside THIS file. By default it creates an unsecured boot image. But you can edit way, way more using this method. For an example what can be done, you might wanna take a look at THIS file
The signapk.jar and the sign keys were taken from the latest CM11 sources.
You have a question, suggestion, bug report? Please feel free to leave a reply in this thread
Many thanks go to @GermainZ and @doixanh for helping me fighting with bash
Thanks also go to Koush, for "inventing" the anykernel updater

Environment setup script
Who is bored of setting up an entire build environment for Android? I certainly am, which is why I wrote this "tiny" script
This script sets up a complete build environment, so you can directly start to repo sync a ROM after the script is done. And the best thing is, it is extremely customizable without even having to edit the script itself. All done by appending the appropriate flags to the executing command.
A further big advantage of this script is, that it is compatible with the two most common Linux distros Arch Linux and Ubuntu (also including all distros that are based on these 2, like Xubuntu, Lubuntu,....). This compatibility is achieved by detecting the distro in the beginning of the script and then using the appropriate commands (apt-get or pacman).
The script is located HERE and called "setup.sh"
To run the script, cd to the folder the script is located and enter ". setup.sh" and append all desired flags. If you need help, just append the "-h" or "--help" flag and some help will be shown.
So, you may ask yourself now "That's nice, but what the hell does this script install/set up?" and the answer is quite simple: All you need, and (if desired) even more
The "base package" includes:
Ccache
Java
GIT
All needed build tools
Python 2
The Android SDK
Furthermore, you can also choose the "extended package" by appending the flag "-e" or "--extended", which includes:
SSH
iostat
bmon
htop
geany
The dev-host commandline tool by GermainZ
udev-rules (on Arch only)
If this still isn't enough for you, you can also add your own packages. This can be done by either editing the script and entering the packages inside the quotes of the following line:
Code:
extra="" # Add here some extra packages to install
OR
by appending the flag "-s" or "--special" and then listing the packages within quotes, seperated by spaces.
Example:
Code:
. setup.sh -s "package1 package2 package3"
This script is made to perform all actions on its own, however, it will not do so by default. To have it automated, you need to append either "-a" or "--automated". The script will then install the "base package" with all default packages (such as Oracle's JDK 6).
But you can still also append the other flags, like "-e", "-s", "-j",...
Now, let's talk about Java. This script can install 3 Java versions: Oracle's JDK 6, openJDK 6 and openJDK 7. The default version is Oracle's JDK 6. You can change the version by either appending the "-j" or "--java" flag followed by the desired version (1, 2 or 3), or if the -a flag isn't triggered, the script will ask you about it.
What? You don't want to install/change java at all? No problem! Just append "--no-java" and Java will get completely skipped.
The same thing also happens to the SDK when appending "--no-sdk".
The script has been tested on (X)ubuntu 13.10 and 14.04 by @nilse and me, and on Arch Linux by @SMillerNL and me. However, I have only tested Oracle's JDK 6, no other java versions but according to the wiki pages, it should work just fine
You have suggestions, feedback, improvements? Shoot! Just let me know and I'll do my best to include it
A big thanks goes to my testers @nilse and @SMillerNL who also made some nice suggestions. But also to @GermainZ for his dev-host commandline tool
Furthermore to @eagleeyetom for giving me the idea

Thanks awesome work it will be helpfull i was looking for such thing yesterday XD you're best @laufersteppenwolf

The anykernel script doesn't work on newer devices with zImage-dtb right?
Doesn't look like it based on the script, but I've had trouble with the M8 DTB so figured I'd ask.

u saved my life thanks buddy i will try it !!!

Hi
Fits ics?

xboxfanj said:
The anykernel script doesn't work on newer devices with zImage-dtb right?
Doesn't look like it based on the script, but I've had trouble with the M8 DTB so figured I'd ask.
Click to expand...
Click to collapse
To be honest, I have no idea
I have only tested it on the 4x HD myself, but if you use all the same commands as the the ROM build process does (maybe also grab the needed files) it should work. I mean, it does exactly the same thing as when the ROM build packs the boot image, it puts the zImage and ramdisk into one file. Only anykernel does this on the device itself, rather than on your PC.
If you want, we can take a look into it together
Marília de Oliveira said:
Fits ics?
Click to expand...
Click to collapse
Sure, should work with all ROMs, just edit some stuff (like rom_version=11 to rom_version=9 when compiling CM) and it should work just fine

laufersteppenwolf said:
To be honest, I have no idea
I have only tested it on the 4x HD myself, but if you use all the same commands as the the ROM build process does (maybe also grab the needed files) it should work. I mean, it does exactly the same thing as when the ROM build packs the boot image, it puts the zImage and ramdisk into one file. Only anykernel does this on the device itself, rather than on your PC.
If you want, we can take a look into it together
Sure, should work with all ROMs, just edit some stuff (like rom_version=11 to rom_version=9 when compiling CM) and it should work just fine
Click to expand...
Click to collapse
How do I edit? Can you explain me better .. Thanks

Thanks a lot @lauferstppenwolf
Really takes the stress out of executing commands during my kernel compiles.

Marília de Oliveira said:
How do I edit? Can you explain me better .. Thanks
Click to expand...
Click to collapse
open in notepad or gedit or whatever and edit what u want
BTW does -j number of jobs should be for example "-j6" or "-j 6"?

Marília de Oliveira said:
How do I edit? Can you explain me better .. Thanks
Click to expand...
Click to collapse
It's quite simple, open the script with an editor (I personally like Geany) and edit the following lines:
Code:
ccache_dir=/home/laufersteppenwolf/ccache/CM11
ccache_log=/home/laufersteppenwolf/ccache/CM11/ccache.log
jobs_sync=30
jobs_build=20
rom=cm
rom_version=11
device_codename=p880
example:
You want to compile CM9 for the Nexus 5 (codename hammerhead) on a dual core CPU and your username (on your PC) is username:
Code:
ccache_dir=/home/[COLOR="Red"]username[/COLOR]/ccache/CM11
ccache_log=/home/[COLOR="red"]username[/COLOR]/ccache/CM11/ccache.log
jobs_sync=[COLOR="red"]5[/COLOR]
jobs_build=[COLOR="red"]3[/COLOR]
rom=cm
rom_version=[COLOR="red"]9[/COLOR]
device_codename=[COLOR="red"]hammerhead[/COLOR]
and you're already done Your script can now compile CM9 for the Nexus 5

laufersteppenwolf said:
It's quite simple, open the script with an editor (I personally like Geany) and edit the following lines:
Code:
ccache_dir=/home/laufersteppenwolf/ccache/CM11
ccache_log=/home/laufersteppenwolf/ccache/CM11/ccache.log
jobs_sync=30
jobs_build=20
rom=cm
rom_version=11
device_codename=p880
example:
You want to compile CM9 for the Nexus 5 (codename hammerhead) on a dual core CPU and your username (on your PC) is username:
Code:
ccache_dir=/home/[COLOR="Red"]username[/COLOR]/ccache/CM11
ccache_log=/home/[COLOR="red"]username[/COLOR]/ccache/CM11/ccache.log
jobs_sync=[COLOR="red"]5[/COLOR]
jobs_build=[COLOR="red"]3[/COLOR]
rom=cm
rom_version=[COLOR="red"]9[/COLOR]
device_codename=[COLOR="red"]hammerhead[/COLOR]
and you're already done Your script can now compile CM9 for the Nexus 5
Click to expand...
Click to collapse
I have done everything ... where else do I put the script?

Marília de Oliveira said:
I have done everything ... where else do I put the script?
Click to expand...
Click to collapse
just put it inside the root of your sources (the place where you have initiated the repo) and then run it using ". build.sh [flags]"

laufersteppenwolf said:
just put it inside the root of your sources (the place where you have initiated the repo) and then run it using ". build.sh [flags]"
Click to expand...
Click to collapse
Thanks ... Worked in xperia mini pro with ics :good:

gerciolisz said:
open in notepad or gedit or whatever and edit what u want
BTW does -j number of jobs should be for example "-j6" or "-j 6"?
Click to expand...
Click to collapse
Sry for the late reply, haven't seen it at all
it's "-j x", as it only listens to the things behind the actual flag

laufersteppenwolf said:
Sry for the late reply, haven't seen it at all
it's "-j x", as it only listens to the things behind the actual flag
Click to expand...
Click to collapse
thx after reading that script i know now i thought it is same like brunch/lunch -jx

Alright guys, a big update just found its way to github: a script to set up a complete build environment on Ubuntu and Arch-based distros.
If you want me to add support for another distro, please contact me either via PM or via IRC (#TeamFun or #p880-dev) and give me details on what must be changed (install command, packages, ...)
Also the kernel script got some updates, to abort the script if the kernel didn't compile and to copy the anykernel zip directly to dropbox to share it with your testers
Cheers

laufersteppenwolf said:
You want to compile CM9 for the Nexus 5 (codename hammerhead) on a dual core CPU and your username (on your PC) is username:
Code:
ccache_dir=/home/[COLOR="Red"]username[/COLOR]/ccache/CM11
ccache_log=/home/[COLOR="red"]username[/COLOR]/ccache/CM11/ccache.log
jobs_sync=[COLOR="red"]5[/COLOR]
jobs_build=[COLOR="red"]3[/COLOR]
rom=cm
rom_version=[COLOR="red"]9[/COLOR]
device_codename=[COLOR="red"]hammerhead[/COLOR]
Click to expand...
Click to collapse
couldnt you make the script universal by using $HOME?

Wow! Great job and well written... I'll be trying this on my old archserver box that I thought I would have to convert to Ubuntu because I couldn't get all the required packages configured correctly. Looking over your scripts you've touched it all!

Thanks a bunch for the scripts, especially the env-setup one. I'll test it one of these days in Manjaro.
However, env-setup/setup.sh has a small bug: after running the script in LM16 (build environment already set up) it wants to reinstall Java no matter which flag is given.
Also, it doesn't detect installed SDK for some reason.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}

[TOOL][LOG] Easy-to-share log and report for developers and users

Hello XDA comunity!
I want to share you a very useful tool that I have created, its name is: "kielyd".
What's keilyd?
Keilyd is a BASH daemon that constantly dumps the system's logs, warnings and other useful info in a easy-to-share "ZIP" with the main purpose to create a very informative and useful bug report.
keilyd is expansible and configurable. It's very easy to add new features because BASH interacts directly with the Linux/BSD commands in the Android System.
How does it works?
Keilyd is launched at device boot, with the "init.d support" and as root (see requeriments below). After launched, keilyd relaunches itself in a "daemon mode", that runs in the background, then, it dumps several information in the /sdcard and compress it in a single zip, so, the user can send that zip (in fact, are 2 zip's or more if you use the "snapshot" feature) to the developer of their ROM, to a forum or send it to a 3rd party developer and report a bug with all the system logs included.
Keilyd stores 2 zips: the first one is the current log (the "very last file") and the second one is the last log before keilyd were launched again.
If you won't enable de daemon, you can create a single zip when you use the "snapshot" feature, if you take two or more snapshots in the same minute, the ZIP will be the same, otherwise, one script per different minute will be created. The Snapshot feature can create as much zips as needed.
What info is collected?
That's a good question!
The info that is collected is (this applies ONLY for a vanilla copy of keilyd):
Kernel log (aka "dmesg") and other kernel info
Logcat (aka "logcat")
List of process running and memory statics
A copy of all tweaks in init.d (including keilyd itself)
A copy of build.prop and sysctl.conf
A copy of /proc/config.gz (if exists)
Some cpuinfo
Mounted filesystems and usage
How does this "daemon" can help me?
Developer:
Have you had the need to get some system log from a user?
Probably the answer is "yes", but, the user may have no idea how to do that, so, you have to give them a "step-by-step" guide of "how to get a system log in the terminal".
This simplifies the process, so, the user can report a bug with logs with no terminal (if the daemon is enabled) or with a single command that looks like
Code:
keilyd --snap
User:
Well, the complement of the developer... You can share "golden" information just by sending 2 zips files to the dev. Minimal Android knowledge is the only prerequisite if the daemon is working as spected.
This daemon is not a demon, but it can be very bad if not implemented as spected. Please, ensure that ALL THE COMMANDS ARE KNOWN and that THERE IS NO PORPRIETARY interference.
Why BASH?
A good idea is to create a daemon in C/C++ and share it as a "binary", BUT, as we are recolecting information from the device, the user must have to be concient of what are the script recollecting, if the user can't read bash, it can ask to a 3rd developer. You can feel safer because it's under the GPLv3 licence!
In addition, BASH run in (almost) any UNIX based OS, including but not limited to Apple OS X, GNU/Linux, Android, iPhone OS, etc...
This prevent a "blind trust" from the user, and give them facility of enable-disable the script, add features and so on.
Requeriments
- A rooted device
- Busybox (or run-parts)
- init.d support
- BASH 4.3.x ONLY SUPPORTED by the moment.
- Terminal emulator/HIDE]
How can I include it in my ROM?
Take a look of my repo in GitHub, it contains the last version of keilyd and the implementation during compilation in the Makefile.
If you are a ROM modder, download the latest version from above, include init.d and bash support in you ROM.
Or you can create a CWM/TWRP/Philz... flashable zip (it would be great!)
Users: how to use
When something goes weird, open the terminal emulator and write:
Code:
keilyd --snap
and send to the developer all zips in
Code:
/sdcard/keilyd/out
type
Code:
keilyd --help
and a helpful message will be printed.
If something goes wrong and you need to reboot the device (or the device reboots itself), just send both zips to the developer or the forum where you are asking help (This feature only works if the daemon is enabled).
Developers: how to use
You can add/remove things in order to fit your needs, the only thing that we ask you is that you write the whole script in BASH, without "blobs" that may private the user from know what is exacly logged and stored.
If some user sends you the zip, you will notice that the files are plain-text, and you can view it in any text editor.
TODO:
List all apps installed (system and user)
Secure "user-modificable" settings (Allmost DONE)
A graphical interface (an app) to manage the daemon
A flashable ZIP
Aports and sugestions are WELLCOME, please, coment in the box below
Install redistributable package:
If the avobe requeriments are meet...
1. Open this GItHub URL
Click in the "Download ZIP" icon in the right.
2. Put the zip in your phone
3. Unzip it.
4. If you want to use the script... A; otherwise, B.
A.
a. Open the terminal emulator.
b. cd to the folder that contains
Code:
install.sh
c. As root, type "
Code:
sh install.sh
" in the terminal.
B.
a. Open ES File Explorer
b. Enable root explorer and mount /system as rw
c. Copy
Code:
daemon/18keily
to
Code:
/etc/init.d
d. Tap and hold the
Code:
18keily
file and open the poperties
e. Change te permisions to
Code:
rwx r-x r-x
f. Copy the
Code:
redistrib/bash
file to
Code:
/system/xbin/bash
g. Change the permisions as above.
h. Symlink /etc/init.d/18keily to /system/xbin/keilyd
i. Reboot
NOTE:
Keilyd can be used in 2 ways:
1. As a daemon that does it job without human interaction, that constatantely and automaticaly creates 2 zips in the output directory.
2. As a Script that only needs to be called with the '--snap' parameter, but it requieres human interaction and does not constantely repeat the process.
Use the first one if you are a beta tester, or if you know that you ROM can hang sudently. Recomended for Unstable ROM'S.
Use the Last One if your ROM is stable or official, but you want to do a bug report.
You can allways use the second option if you use the first one.
By default, the daemon is disabled. You can enable if you change the "ENABLED" variable to "Y". I personally recommend values for "MINS" between 5 and 15.
BUGS REPORTS in the GITHUB REPOSITORY

Reserved

Hi there,
This looks like what I need, did you close the repo? Can you share a link to the script? I have a few devices, and I want to have some forensic information for troubleshooting (I.E. log all the warnings, errors and critical logs, + a snapshot every n minutes about the system usage). Right now I have to go where the device is, connect it with ADB and query the logs (most of the time, the logs were overwrited)... Thanks in advance.

[GUIDE] Build CyanogenMod (or AOSP) from source

Start building!​Requirements:
A good CPU (2 cores more prefered)
4 GB RAM or more
Knowledge of using Linux
Time and patience
Step 0: (getting ready)
Make sure you have at least 50GB space free and all the required packages installed.
You can get a guide to installing packages on Ubuntu here.
Step 1:
Get the repo tool installed.
A guide for this over here.
Step 2:
Create a folder in your home directory named 'android'. This can also be any other name in any other place just make sure you remember the path.
Step 3:
Run
Code:
cd ~/android/
inside a Terminal (or the path you created the directory in)
Step 4: (getting the source)
Inside your android folder run
Code:
repo init -u git://github.com/S6310-Development/android.git -b cm-11.0
You can edit the 'cm-11.0' parameter to the version of CyanogenMod you want to build. Currently supported versions are 'cm-11.0' and 'cm-12.0'.
After you ran the command run
Code:
repo sync
The syncing can take from half an hour up to a few days. This depends on your Internet connection. On a 30Mbits/s connection it takes about two to three hours to sync.
Step 5: (preparing the build)
Make sure your PATH variable points to Python 2.7 if you have multiple instances of Python installed.
Step 6:
Open up a Terminal window in the Android source tree. (for people who followed step 2 exactly this is '~/android/')
Step 7: (building)
We have provided a prebuilt build script so no new one has to be made. You can run this script with
Code:
./build-script.sh
Select your device once the selection comes up and select the build type depending on the type of build you want to create.
We use 'userdebug' for releases and 'eng' for testing and debugging. 'user' in general is never touched by us.
Happy building!​

must try if i have

[GUIDE] Extracting proprietary files from OTA updates easily

Hi all
I thought I'd try and write an easy and simple guide on how I extract blobs from vendor updates, cause I couldn't find a guide online myself.
A few things first:
- I will be writing this guide on Windows. You will need to find the Linux/MacOS tools yourself cause this is what I use.
- I use Windows Subsystem for Linux in this guide. It's not needed, but will make your life a lot easier. Follow this guide if you haven't got it installed already.
- This will assume you know how to build a ROM, know what device/vendor trees are, have some slight knowledge of the command line and know how to use basic Git.
First, we need to grab an OTA. I own a MI 8 Lite, so I usually use xiaomi.eu to update my blobs as it's easy to capture the zips. Depends on your device.
Now you've captured your OTA, you need to get the blobs extracted out. I use UR-Tool on Windows 10 but it's completely your choice. I will be proceeding this tutorial with UR-Tool.
Extract your zip into the UR-Tool build environment. Run the tool and choose your zip, extract it, like here: https://i.imgur.com/Y50Y3IZ.png
Okay, so now we have two directories, system and vendor (I'm fairly sure if you're non-treble you'll have one folder which means you can skip this next step.)
Go into system, and remove the symlink named vendor. You can then copy in the vendor folder into system.
In my device tree, I also had /sbin/charge needed in my proprietary-files.txt. I needed to extract my boot.img and put this folder into the system folder. Just check your device tree's needed files.
Copy the entire folder to somewhere easy to find and preferably a path which doesn't contain any spaces. We will be using it later.
Okay, so now we've done all this, we are going to move to the vendor and Git side of things. I will be using WSL from here!
So, now we have our fully ready to go blobs... but what now?
Well we need to prepare our vendor of course!
Like I said above I'll be using WSL from here as I needed "patchelf" for mlipay blobs. You will need something that can execute .sh files and using WSL is probably the easier option. If you're on Linux you don't need to do anything really.
We need 3 things to clone in. I've taken the liberty of stripping down Lineage's vendor to just the critical stuff for extracting blobs. You can find my repository here.
Lets make a new directory. We will clone in our trees like this:
Code:
git clone (device tree link) device/(vendor)/(codename)
git clone (vendor tree link) vendor/(vendor)/(codename)
git clone https://github.com/nysadev/extract_blobs_vendor vendor/lineage
This will create a directory that looks like a ROM, but severely stripped down. The reason we do the folders like this is because the scripts in device tree will point to a structure like this. This will help us avoid script location errors later. Make sure in your device tree that extract-files.sh points to vendor/lineage (should be in the first few lines).
Your root directory now should look like this: https://i.imgur.com/YiqMaDJ.png
Now go to your device tree. This is where all the good stuff happens. You need to know the path here of your extracted blobs from earlier.
Run these two commands:
Code:
chmod +x setup-makefiles.sh
./extract-files.sh (path/to/your/extracted/system)
Hopefully if all goes well you should see this with no "file missing in source errors": https://i.imgur.com/jErpyCv.png
Just to be sure, at the end of when it extracts, run "./extract-files.sh" again, just to prepare your vendor.
Now you're done! You'll hopefully see a few good modified libs, changes to Android.mk and your device-vendor.mk.
You're probably going to want to use git here to upload your changes. Just run:
Code:
git add .
git commit -m "(codename): update to (ota-version)"
git push -f
or something of the sort.
Well done! You're now done! Feel free to reply with any errors you get.
My Telegram is @nysadev; you're maybe better off catching me there.
I hope my guide helped you. Drop a like or something, idek.

It will be very useful

You rock! Love ya

how do u get proprietary-files.txt if u r building from scratch?

wheres UR-tool?

[Guide] Building post 2017 android kernel from source

About this guide
I have been building kernels for many machine types and have been fiddling around with custom android kernels for my own phone but to my great frustration the rules changed quite a lot for my new phone. It seems every kernel after 2017 is A LOT harder to build and needs quite some fiddling around te get it to boot. The documentation is terrible the least on the steps and all howtos are all outdated by now. This guide shares with you the secrets to build yourself a modern post 2017 kernel from source. I only own a op7 at this moment but I am pretty sure the rules apply to all devices.
So why do we need to build our own kernel? Well that's simple. I have scrolled through many custom kernels missing a simple module that I need. In my case binfmt. Perhaps you need a specific device driver compiled into your kernel or you want to hack in/patch in some special functions that are only needed by you. Or you love a certain kernel but need that one little adjustment in the configs that this kernel doesn’t have and so on.
Lets get through it step by step but I do assume you know something about Linux, you know how to use the terminal and that you have some knowledge on building binaries from source. Ready? Ok lets get started.
These steps are required to be followed to be successful:
1. Set up a building environment
2. Obtain the kernel source
3. Download the proper tool-chain
4. Adjust configuration files to match your needs
5. Build the actual kernel image
6. Backup your current boot.img!!!
7. Trow the image into a anykernel zip and flash it.
8. Close your eyes prey and run the kernel.
Prerequisites
You will need:
A fairly fast computer with lets say 4 gigs of RAM and a decent CPU.
A workable linux distribution that you can boot into (I prefer ubuntu bionic at this moment).
The kernel sources for your device.
ROOT!!
Lots of spare time.
A attitude that makes you unstoppable.
1. Setting up the building environment
Ok lets first open a terminal. You will need the terminal a lot. As a matter of fact it will be your friend through this guide so better get used to it. Assuming you are on a ubuntu like distribution lets fetch the important packages first
Code:
$sudo apt install build-essential bizon flex git
This command will install some of the prerequisites you need to do anything. More packages will need to be installed on the way but at least you will be able to type $make
Now lets create a working directory. Go into your home folder and create a working directory which will later on contain your kernel sources and tool chain.
Code:
$cd
$mkdir [COLOR="MediumTurquoise"]name-of-your-working-folder[/COLOR]
2. Obtain the kernel source
This one is tricky as my experience have learned me the kernel sources supplied by my own vendor are usually badly documented and contain a few bugs here and there resulting in either build errors or a unbootable kernel. So usually I grab a custom kernel that has minimal changes to the stock kernel and boots by only using a single kernel file without extra kernel modules attached. The custom kernels usually have the nasty building bugs flawed out and thus your chances for success are higher. I used the exkernel in my case and out of respect for the hard work of flang2 I bought the app that came along.
To get the sources google around and find the sources on for example git and click the download or clone button there and copy the link of the depository.
Then simply clone the depository to your working directory so after changing to your working directory type
Code:
$git clone link-you-just-copied
Git will now automatically clone the entire kernel source into the source sub-directory. You can change the name of this directory to make you life easier.
3. Download the proper tool-chain
This is where I lost many days and weeks of my life. Where do we find the proper tool-chain for our kernel?
You basically need 3 compilers
clang
GCC
device-tree-compiler
There are many versions of clang and GCC and you will need to find the one that matches your kernel and builds for your cpu architecture. I will assume in this guide that your CPU is aarch64 or arm64 (both names are used at the same time).
To obtain clang head over to the following address:
"https://android.googlesource.com/platform/prebuilts/clang/host/linux-x86/"
now in the tags and branches you can find the version of android you are looking for. If your kernel is for android 9 then use the android 9 tag. Click on it and then click tgz which will provide you with a huge tarball.
Create a folder in your working directory named toolchain en extract the contents of the tarball into the sub-folder clang.
You can try any subversion of clang there is but I recommend to simply peek into the build.config files located in the root of your kernel source directory which usually specify the exact subversion of clang used.
Time to get GCC which is easier since there is only one version right now.
Head over to
"https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/"
and use the correct version here as well. In my case I picked the latest build. Then do the same as for clang but name the sub-folder GCC or whatever you please to do.
And then finally get yourself the device-tree-compiler
Head over to
"https://github.com/dgibson/dtc/tree/v1.4.4"
and clone this depository into ~/your-working-directory/toolchain/
and navigate to the subdir you cloned it into
then build by typing
Code:
$make
any missing package errors will need you to install them using
Code:
$sudo apt install missing-package-name
and do make again until the build completes.
If all went well your tool-chain and working directory will look like this:
Code:
/[COLOR="mediumturquoise"]current-working-dir[/COLOR]/[COLOR="mediumturquoise"]kernel-source-dir[/COLOR]/
|
/toolchain/
|
/clang/
/gcc/
/dtc/
4. Adjust configuration files to match your needs
Lets first grab the correct defconfig file and setup the build directory. This defconfig file contains the default kernel configuration parameters which you propably want to adjust.
Head over to ~/working-directory/kernel-source-dir/arch/arm64/configs/
see what you can find there. I found the elementalX_defconfig file which is the file I needed to build the Exkernel from scratch. If you are trying a stock kernel dive a little bit deeper into the vendor folder and see if you can find a good config file there.
Alternatively sometimes the config is hidden on your device itself. Have a look at
/proc/config.gz
the file inside the archive contains a valid defconfig for your device but beware your may need to open it with menuconfig first and save it again to make it workable for the compiler script.
Place the defconfig file you like to use into the configs folder.
(Skip the following step if your are going to build your kernel for the first time because we first want to see it work.)
Now head over to your kernel source dir and type the following command
Code:
$PATH="~/[COLOR="mediumturquoise"]working-dir[/COLOR]/toolchain/clang/[COLOR="mediumturquoise"]clang-r353983c[/COLOR]/bin:~/[COLOR="mediumturquoise"]working-dir[/COLOR]/toolchain/gcc/a[COLOR="mediumturquoise"]arch64-linux-android-4.9[/COLOR]/bin:${PATH}" \
make menuconfig O=out \
ARCH=arm64 \
CC=clang \
CLANG_TRIPLE=aarch64-linux-gnu- \
CROSS_COMPILE=aarch64-linux-android-
now remember to use the correct paths to your tool-chain. In this command I use clang-r353983c but you may use a different version. The same applies for linux-android-4.9.
I would like to thank Nathan Chance for supplying documentation on the clang tool-chain on his github.
If all well you will be presented with a menu in which you can adjust things. First open the defconfig file you wish to use and then save the file after you are done. Now REMEMBER! With this command the menuconfig will save your new file into the /kernel-source-dir/out folder.
(end of the skipped step)
Now it is time to prepare our kernel build using the following command (do this from your kernel source dir) and remember that this command will look in the /kernel-source-dir/arch/arm64/configs/ directory
$make O=out ARCH=arm64 name-of-your-defconfig
If all goes well you will see something like configuration written to .config
Everything is up and ready to go. Your hart will start pounding from this moment on
5. Build the actual kernel image
This is where things get complicated. I hope you have many years to live because this part consumes time, a lot of it.
type
Code:
$PATH="~/[COLOR="mediumturquoise"]working-dir[/COLOR]/toolchain/clang/[COLOR="mediumturquoise"]clang-r353983c[/COLOR]/bin:~/[COLOR="mediumturquoise"]working-dir[/COLOR]/toolchain/gcc/[COLOR="mediumturquoise"]aarch64-linux-android-4.9[/COLOR]/bin:${PATH}" \
make -j$(nproc --all) O=out \
ARCH=arm64 \
CC=clang \
DTC=~/[COLOR="mediumturquoise"]working-directory[/COLOR]/toolchain/dtc/dtc \
CLANG_TRIPLE=aarch64-linux-gnu- \
CROSS_COMPILE=aarch64-linux-android-
ready set GO!
This will depending on your machine take between forever and eternally. You will witness many warnings on the DTC build seems normal these days and a few warnings on the CC part. Most important is that no errors are thrown at you.
If all goes well you will see a normal exit status and you will have a “working” kernel image.
“Error?: Well that happens. Try a different build of clang check your command line. And if all that fails try to find out what is wrong in the source and that means digging through thousands of forum pages until you find out whats wrong. But using the google tools usually goes well.”
Almost there go and check
kernel-source-dir/out/arch/arm64/boot
and there should be a image-dtb or image.gz-dtb file depending on you settings.
That is your kernel image right there. The difference in size between image and image-dtb should not be huge. 10Megs in difference usually means your dtb is not good but trying is the only way to find out if it works.
6. Backup your current boot.img!!!
You know what to do here right? Do not skip this step unless you like bricked devices or want to reflash and lose your data and all that kind of stuff. Not sure what you are doing stop here or backup your entire device including system vendor etc.
7. Trow the image into a anykernel zip and flash it.
Ok something changed in the last few years. Unpacking repacking and booting using fastboot somehow gives me problems. Dm-verity errors and all kind of red screens. Signing the boot image results into new errors. Well this is how I did it.
Get yourself a anykernel zip file. I used the Exkernel.zip because it only contains a kernel image which I like. Open the zip in a good zip tool (I used ark) and replace the image-dtb file with the one you created. Place this new zip you created on a memory stick and then….
Flash it using twrp or any tool of choice.
8. Close your eyes prey and run the kernel.
Two things can happen.
1. Blank screen nothing happens. Only god can help you, repeat all the steps.
2. Your android starts booting. Start crying of joy
Check in your android if this is indeed the kernel you build. If so time to make some adjustments to your configs or happily enjoy your boosted phone.
Now please remember. If you plan to distribute your kernel that you do the correct steps of accrediting the original programmer and trow the source online. If you use a already custom kernel please respect the hard work of the maker and communicate your plans with him/her.