Docker TWRP CICD

Docker TWRP CICD - Android Software/Hacking General [Developers Only]

TWRP CICD Project
This is my personal effort given to you, the community, which spends a lot of hours to craft a fully-functional and perfectly tuned ROM for our beloved Smartphones!
What is it?
This project aims to provide a Build system for Android Developers which will be able to build a TWRP recovery image for a different set of Codenames given, automatically for you, ( by default ) every night. Not only built stock sources that are already available on TWRP Github account, but feel free to build even your custom code thanks to the support of local_manifests/*.xml fully supported by this Docker.
Where can I find it
CICD
- Github: https://github.com/julianxhokaxhiu/docker-twrp-cicd
- Docker Hub: https://hub.docker.com/r/julianxhokaxhiu/docker-twrp-cicd
Why?
At the current status I find very time consuming the task to always setup the environment the right way to build a sporadic recovery for any kind of device, depending on the tree.
Therefore this projects aims on providing an easy-to-use build system which may help you on providing a Ready-To-Flash IMG at the end of each Build round. The same you would download from twrp.me website.
How does it work?
This is a pre-packaged Docker system based on Debian, with all the dependencies in place to correctly build ( even in an optimized way ) any TWRP codename image. A cronjob will take care to start the build script on the configured time ( by default 10:00 UTC ~= 02:00 PDT ), which then will take care to build every codename given with an environment variable to the Docker.
All you need from now on is just the Docker Engine installed on your favourite Linux distribution.
I want to use it right now!
I am pretty sure you want! The Docker has now been tested for a nearly a Month and I'm successfully installing my own builds for a couple of weeks on different devices ( really a great satisfaction! ).
If you want to run it as well, I suggest you to take a look at this Bash script that will run for you the Docker. Since the script has been studied to work on top of the "VPS Powered By Docker" project, I suggest you to take a look at it. Although nothing is preventing you to use it in your favourite way.
Requirements
See the detailed list on the project README.
What about License
All my projects are always developed with MIT license, which means feel free to do what you want with it. I don't really care if you do business with it, it was a great challenge for me reaching this autonomous entity to run, therefore I don't mind of possible outcomes of it. Although Issues are always welcome for improvements, if any found, of suggestion for a possible feature enhancements.
Enjoy!
XDA:DevDB Information
TWRP CICD, Tool/Utility for all devices (see above for details)
Contributors
JulianXhokaxhiu
Source Code: https://github.com/julianxhokaxhiu/docker-twrp-cicd
Version Information
Status: Testing
Created 2018-01-01
Last Updated 2018-01-01

Related

[Q] Snapshot of Android source

Hi!
I planning to create packages from Android SDK for some popular Linux distributions. However, most distro have a policy against repackaging third party binary packages (such as you can download from developer.android.com).
I would like use snapshot of Android source tree, because
- Android SDK not changes too fast
- repo utility fetches a very, really very lot of unneeded information (a whole history of all repository in android tree)
- It can increase a package build time a _lot_. As some autobuild solutions have a hard timeout for package building, I would like avoid using repo anyway.
So, the question is clear: is there somewhere a snapshot from the source tree?

hron84 said:
Hi!
I planning to create packages from Android SDK for some popular Linux distributions. However, most distro have a policy against repackaging third party binary packages (such as you can download from developer.android.com).
I would like use snapshot of Android source tree, because
- Android SDK not changes too fast
- repo utility fetches a very, really very lot of unneeded information (a whole history of all repository in android tree)
- It can increase a package build time a _lot_. As some autobuild solutions have a hard timeout for package building, I would like avoid using repo anyway.
So, the question is clear: is there somewhere a snapshot from the source tree?
Click to expand...
Click to collapse
I would like to know how to do this also.

HotSwapper: build all open source roms easily.

I have been meaning to release this for awhile. I maintained a project that required me to build various AOSP forks and often switch between Android versions and various roms.
The use case for the HotSwapper is simple. You want to build your own ROM, but don't have an environment. Or, you already have built one ROM for your device, but want to try another supported ROM. This allows you to switch between ROM branches and forks with just one command.
The idea is that you can install VirtualBox on your machine, boot into a Debian (Use Ubuntu for best results, only thing it is tested on) and have a rom built in two commands. Future builds only take
Code:
./hotswapper make
command.
This should be helpful to anyone who wants to build roms easily, or is tired of looking up branch names or github addresses.
Anyway, it is a little thing, but I used a version of it to make my life easier, and thought I would share.
Steps (skip any of these you like, if you have already done something similar):
Install VirtualBox
Boot up an Ubuntu image
Code:
sudo apt-get install git
Code:
git clone https://www.github.com/mateor/HotSwapper
Code:
cd HotSwapper
EDIT the BUILD file (see below)
Code:
sudo ./bootstrap
Code:
./hotswapper make cm[code]
[/LIST]
The above [code]./hotswapper make cm
will make CyanogenMod for Android 4.4. If you wanted to build SlimRom for 4.3, you would use
Code:
./hotswapper make slim 4.3
. Easy enough, right?
This is a command line app, but should be pretty easy to play with. I made it as a POC for a proposal I submitted to a company. It is pretty rough, honestly, but I have tested it to work. I used many of the main mechanisms when I was maintaining Auto-patcher and OpenPDroid.
XDA:DevDB Information
HotSwapper, Tool/Utility for all devices (see above for details)
Contributors
mateorod, mateorod
Version Information
Status: Beta
Current Beta Version: 1.0
Beta Release Date: 2014-05-15
Created 2014-05-15
Last Updated 2014-05-15

[ZIP] LazyFlasher - the swiss army knife of flashing custom kernels!

LazyFlasher & no-verity-opt-encrypt
INTRODUCTION
Hello Users and Developers of XDA!
LazyFlasher is a custom kernel flashing tool designed to make it easy to dynamically modify ramdisks and inject kernel binaries into the current boot image.
It's the swiss army knife of kernel flashing for use in Team Win Recovery Project.
The intent behind it was to allow a 1 custom kernel fits all approach, where your users can flash single zip on any ROM for a particular device,
allowing your kernel to be compatible with the vast majority of custom ROMs already out there. It takes away the pain of building custom boot.img
for each and every variant a user requests and puts it into 1 low maintenance intelligent universal flashable zip.
You might already know of @osm0sis's AnyKernel2 project. This approach is similar to that. Back in late 2015 I decided to design a more compatible, more friendly, and more feature filled version. Since then, AnyKernel2 has apparently improved a lot so if LazyFlasher doesn't accomplish what you need, AnyKernel2 probably will.
LazyFlasher does not currently support ELF boot images.
For users of no-verity-opt-encrypt: This thread can also be used to discuss the no-verity-opt-encrypt project which is just a minimal version of the LazyFlasher framework.
Disqus thread: https://www.xda-developers.com/xda-...-is-an-alternative-to-the-anykernel2-project/
THE GITHUB REPOSITORY
You can find LazyFlasher's development and download it here: https://github.com/jcadduono/lazyflasher
LazyFlasher source code is distributed under the BSD 2-clause license. You can do anything you want with it, however, some of the binaries used by it are under GPLv2 or GPLv3 licenses.
FEATURES
ChromeOS support (ChromeOS test-key signing and recognition)
MediaTek device support (MTK headers)
SELinux policy injection support via sepolicy-inject
Example scripts to disable dm-verity or forced encryption during the install process (010-no-force-encrypt, 015-no-dm-verity)
A process that executes a sorted list of scripts for making the desired modifications (separate from the framework)
Handily unpacks, decompresses, applies changes, compresses, and repacks boot images quickly and safely
Supports Gzip, LZ4, Bzip2, and LZO ramdisks. Support for LZMA and XZ is a work in progress
Supports arm (armv7), arm64 (aarch64), x86 (i386), x86_64 (amd64), mips, and mips64 architectures
Supports dtb.img replacement (place it in the root folder named "dtb.img")
Scans fstab and partition locations for the boot partition, optionally allows a preset location
Intelligently installs kernel modules by copying the previous layout of /system/lib/modules and creating symlinks
Creates modprobe supported /lib/modules aliases if kernel modules are included in the installer (030-kernel-modules)
Installs new files to the ramdisk and sets their permissions automatically based on file type from ramdisk-patch (020-patch-ramdisk)
Includes an optional bbe tool for applying binary patches
Unnecessary architectures and tools can be removed to save space
Many useful functions and variables included in the patch.d environment to simplify modification/patching scripts (patch.d-env)
Simple "make" build system
SETTING UP LAZYFLASHER
LazyFlasher is only designed for building on Unix based systems such as Linux, Mac OS X, and FreeBSD.
To download it (feel free to fork it so you can have a copy on your GitHub to modify instead!):
Code:
cd ~/build
git clone -b kernel-flasher https://github.com/jcadduono/lazyflasher.git
cd lazyflasher
To use LazyFlasher, you'll probably want to take a look at the Makefile, config.sh, and META-INF/com/google/android/update-binary (a shell script).
There's a few things you can change there to personalize it to your needs.
(make another git commit to save your setup!)
You should also check out the README! There is a lot of useful information there.
Once the installer is set up to your liking, all you have to do to build it is copy the resulting kernel binary from your kernel build output into the lazyflasher folder.
If you have built with kernel modules (make modules_install), copy build/lib/modules -> lazyflasher/modules.
Now simply run:
Code:
make
A TWRP flashable zip and sha1sum is created!
You should consider signing the zip with AOSP test-keys so that users can verify its integrity before flashing it.
LOOKING TO TRIM DOWN THE INSTALLER?
I have forked the kernel-flasher branch to a branch called kernel-flasher-arm64-minimal. This provides all the features of kernel-flasher except sepolicy injection and bbe, and only supports arm64 devices. It reduces the minimum zip size from 1860 KB to 200 KB.
You can use this branch instead if you like. If you're using another architecture, just look at the trimming commit as an example and apply it for your arch.
BUT I AM ON LE WINDOWS!
How did you build your kernel binary?!
Anyways, there is an alternative for building it on Windows.
You can download the LazyFlasher kernel-flasher branch as a zip file from GitHub and extract it somewhere on your PC.
Make your modifications using Notepad++.
You can then use a tool such as 7-zip to create a zip file by selecting everything in the folder, right clicking, and going to 7-Zip -> Add to "lazyflasher.zip".
LIMITATIONS AND KNOWN ISSUES
It will not run on TWRP built in Android 4.3 or earlier (usually builds older than 2.8.0.0)
Requires Busybox to exist in the TWRP build. All official builds should have this.
There may occasionally be some devices that are unsupported due to extreme modifications made to the boot image format by the manufacturer. If you have one of these devices, feel free to contact me and I will try to add support for it if it is worth the effort.
If you have an issue, please gather a recovery.log from TWRP after flashing and I will try to look into it. I can't do anything to diagnose your problem without a recovery log.
Code:
adb pull /tmp/recovery.log
JUST WANT TO DISABLE VERITY/ENCRYPTION?
You can build lazyflasher by itself, empty, without a kernel image or modules and flash it!
It's already set up to automatically disable verity and make encryption optional.
Alternatively, there's a branch already set up called no-verity-opt-encrypt. You can find prebuilt official zips at: https://build.nethunter.com/android-tools/no-verity-opt-encrypt/
WHO ELSE IS USING LAZYFLASHER?
I'll keep a list here of cool projects that are using it. Feel free to ask for yours to be added.
The Kali Linux NetHunter project (GitHub, Website)
no-verity-opt-encrypt, no-verity-force-encrypt, twrp-data-fstype-swap (Website/Download)
WHAT IS LAZYFLASHER USING?
LazyFlasher makes use of a few open-source projects. You can find their source code here:
bootimg / libbootimg for boot image unpacking/repacking - https://github.com/jcadduono/android_external_libbootimg
bbe for binary patching - https://github.com/jcadduono/android_external_bbe
bzip2 for ramdisks - https://github.com/jcadduono/android_external_bzip2
lz4 for ramdisks - https://github.com/jcadduono/android_external_lz4
futility for ChromeOS boot image signing - https://github.com/jcadduono/platform_external_vboot_reference
sepolicy-inject for sepolicy policy injection - https://github.com/jcadduono/android_external_sepolicy-inject
XDA:DevDB Information
LazyFlasher, Tool/Utility for the Android General
Contributors
jcadduono
Source Code: https://github.com/jcadduono/lazyflasher
Version Information
Status: Stable
Current Stable Version: 5.1
Stable Release Date: 2017-02-01
Created 2017-02-02
Last Updated 2017-02-07

We should give this man award.
The Job that he has done with this and Nethunter is just amazing.
Thank you and keep up the good work

Nice tool. Keep up a good work

Great Work! Hope i can include it into my Projects in the Future...
Thanks a lot for that!

You are a goddamn god.

Honestly Annoying said:
You are a goddamn god.
Click to expand...
Click to collapse
thx dude, usually that phrase is reserved for Chainfire accomplishments

y u nu support mah Indian AF MTK fone @jcadduono
Kidding, awesome work on this though!

Good one , this man develops for the developers !

Is it possible to have this on i9100 ? AnyKernel2 doesn't work with i9100

Skyline said:
Is it possible to have this on i9100 ? AnyKernel2 doesn't work with i9100
Click to expand...
Click to collapse
nope, no plans to support that device, surprised they are still out there, i would expect most to be dead emmc by now. :|
you can probably modify boot-patcher.sh to copy partitions to split-img folder then flash them back instead of using bootimg
i don't even know if lazyflasher's binaries will run on any of the i9100 twrp builds. might be too old.
if i can get an example layout of i9100's partitions i can fork it to a new branch, called kernel-flasher-sgs2 and make it compatible for you guys.

jcadduono said:
nope, no plans to support that device, surprised they are still out there, i would expect most to be dead emmc by now. :|
you can probably modify boot-patcher.sh to copy partitions to split-img folder then flash them back instead of using bootimg
i don't even know if lazyflasher's binaries will run on any of the i9100 twrp builds. might be too old.
if i can get an example layout of i9100's partitions i can fork it to a new branch, called kernel-flasher-sgs2 and make it compatible for you guys.
Click to expand...
Click to collapse
They are too old but still supported by lineage 14.1 and official twrp 3.0.2-1 without any problems
osmOsis dev of anykernel2 said that i9100 and older devices are having different boot img header format when i tried to run anykernel2 script it says Android magic is not found something like that

interesting tnx

Wow...
I'm just recognizing now, how powerful lazyflasher is ...
Setting default.prop values and settings. Disable Encryptions and so on. Wish I could contribute something but I'm still learning how it works. For now I've just included lazyflasher into my PATCH to disable DM Verity and forced Encryption and to make some edits on the default.prop. That's really useful since the build.prop doesn't allows such deep changes.
@jcadduono what would be in the Theory possible with the Lazyflasher? Could we add things like Gouverneurs or default Kernel Clockings? Init.d Support? Sorry if I sound noobish :angel:

So, uh, is there a TL;DR for lazy people? :silly: :laugh:

@jcadduono this is absolutely awesome. thank you for your hard work.

Hope that someone can dev nethunter to Asus zenfone 5 t00f :fingers-crossed:

can you please make a tutorial vedio of it because i don't get it and i'm sorry for my stupidity

Will it work for Android Oreo / LOS 15?

Did anyone succeeded in removing dm-verity? I got this error

good

Database Info

welcome