Database Info

inside android: reference books

reference books on Linux kernel
1. Linux Kernel Development.
2. Understanding the Linux Kernel.
3. Linux Device Drivers.
reference books on Android application developing
1. Professional Android Application Development.
about "inside android"
Hi, all.
As experienced android developers or just beginners, sometimes you might have questions about what's going on inside android. Such as how activity starts, how service starts, etc. If we look into the code, we might have the answer, but it takes time and much effort. In fact, there's millions of lines of source code in android. Fortunately, Shengyang Luo wrote a book in chinese to clarify all these questions, with all the related knowledge he got in these years as an experienced engineer of linux/android. Now, he and I decide this book belongs to the android family, so to make more people get benefit from it, we will rewrite it in English.
Also, we want this book to be more adapted to your need, we might share the outline and part of the content of the book here, so that we all can discuss it while rewriting it. We might change the book according to your advice. For example, there might be more details for the parts you are most interested. To do this, we will have pilot PPTs and passages soon. The pilot topics would be as below:
Android design philosophy--everything is component
Initializing a Build/developing/debugging Environment for Android
Android architecture
Hardware Abstraction Layer(HAL)
drivers specific to android
process management
messaging handling
input handling
UI architecture
resource management framework
Dalvik virtual machine
Which is most interested to you? Please share with us, and we will make it available ASAP.
We haven't talked with any press for publishing yet. So if any press is interested in publishing this book, please contact me. As you all know, writing is a tough job, especially when the book is regarding such a complicated android os system, so cheers, encouragement and funding are welcomed badly. Funding would help us not worry about making a living and dedicate to the writing or even hire some people to accelerate the progress. For funding, please visit indiegogo.com and search "inside android".
The book would have 3 parts: say hello to android, drivers specific to android and understanding android in scenes. The first 2 parts are foundation knowledge for the third part, which is the major part of the book.
Part I say hello to android includes 3 chapters, and introduces how to download the source code, build it, run it, smart pointer and Hardware Abstraction Layer(HAL). Since smart pointer is widely used in the native code of application framework, it would be very helpful if we get to know it before we dive into the scenes. Knowing HAL would help us understand the architecture of android, and then make the third part easier to learn.
Part II drivers specific to android also includes 3 chapters, introducing 3 drivers specific to android: Logger Driver, Binder IPC Driver and Ashmem Shared Memory Driver. No doubt, these 3 drivers, especially the Binder IPC Driver, are the foundation of android. knowing about them would definitely help understand android.
Part III understanding android in scenes includes more than 10 chapters, and introduces android in different scenes, such as the start of something(activity, service, zygote, service manager, etc.), process management, messaging handling etc.
The book is based on android 2.3. If needed, we can rewrite it on newer version.
Below is the outline of the book:
Part I: say hello to android
chapter 1　prerequisite
1.1　reference books on Linux kernel
1.2　reference books on Android application developing
1.3　download, build and run Android source code
1.3.1　download Android source code
1.3.2　build Android source code
1.3.3　run Android emulator
1.4　download, build and run Android kernel source code
1.4.1　download Android kernel source code
1.4.2　build Android kernel source code
1.4.3　run Android emulator
1.5　develop one Android application(example)
1.6　build and pack Android module independently
1.6.1　mmm: Builds all of the modules in the supplied directories
1.6.2　build one Android module independently
1.6.3　repack Android images
chapter 2 Hardware Abstraction Layer(HAL)
2.1　develop Android hardware driver
2.1.1　implementation of one kernel driver
2.1.2　modify Kconfig of the kernel
2.1.3　modify Makefile of the kernel
2.1.4　build the driver
2.1.5　test the driver
2.2　test the driver with C executable
2.3　develop HAL module
2.3.1　specification for HAL module
2.3.2　interface for HAL module
2.3.3　load the HAL module
2.3.4　access permission for the hardware
2.4　develop service to the Android hardware
2.4.1　defite the service interface to hardware
2.4.2　implementation of the service
2.4.3　JNI interface for the service
2.4.4　start of the service
2.5　Android application on the service to hardware
chapter 3 smart pointer
3.1　light weight pointer
3.1.1　implementation
3.1.2　example
3.2　strong pointer and weak pointer
3.2.1　strong pointer
3.2.2 weak pointer
3.2.3 example
Part II: drivers specific to android
chapter 4 Logger driver
4.1　Logger format
4.2　Logger driver
4.2.1　data structure
4.2.2　initialization
4.2.3　open the Logger driver
4.2.4　read the log
4.2.5　write the log
4.3　runtime library for the Logger driver
4.4　log interface for C/C++
4.5　log interface for Java
4.6　Logcat tool
4.6.1　data structure
4.6.2　initialization
4.6.3　read the log
4.6.4　output the log
chapter 5 Binder IPC Driver
5.1　Binder driver
5.1.1　data structure
5.1.2　initialization
5.1.3　open
5.1.4　mapping to the memory
5.1.5　kernel cache management
5.2　runtime library for Binder IPC Driver
5.3　example for Binder IPC Driver application
5.4　counting for Binder object reference
5.4.1　lifecycle of Binder local object
5.4.2　lifecycle of Binder instance
5.4.3　lifecycle of Binder reference
5.4.4　lifecycle of Binder proxy
5.5　Binder object death notification
5.5.1　register death notification
5.5.2　send death notification
5.5.3　unregister death notification
5.6　start of Service Manager
5.6.1　open and mapping on Binder
5.6.2　register as Binder context manager
5.6.3　wait for the Client request in loop
5.7　getting the Service Manager proxy
5.8　start of Service
5.8.1　register Service
5.8.2　start of Binder thread pool
5.9　how to get Service proxy
5.10　Java interface for Binder IPC Driver
5.10.1　getting the Java proxy of Service Manager
5.10.2　define the Java service
5.10.3　start of Java service
5.10.4　getting Java service proxy
5.10.5　calling Java service
chapter 6 Ashmem Shared Memory Driver
6.1　Ashmem driver
6.1.1　data structure
6.1.2　initialization
6.1.3　open
6.1.4　mapping to the memory
6.1.5　lock and unlock
6.1.6　recycle
6.2　Ashmem interface in runtime library cutils
6.3　C++ interface for Ashmem
6.3.1　MemoryHeapBase
6.3.2　MemoryBase
6.3.3　example
6.4　Java interface for Ashmem
6.4.1　MemoryFile
6.4.2　example
6.5　how Ashmem shares memory
Part III: understanding android in scenes
chapter 7 start of Activity
7.1　Activity example
7.2　start of root Activity
7.3　start of child Activity in the same process
7.4　start of child Activity in the new process
chapter 8 start of Service
8.1　Service example
8.2 start of Service in the same process
8.3　start of Service in the new process
chapter 9 Broadcast
9.1　example
9.2　register
9.3　send
chapter 10 Content Provider
10.1　Content Provider example
10.1.1　ArticlesProvider
10.1.2　Article
10.2　start of Content Provider
10.3　data sharing of Content Provider
10.3.1　data sharing model
10.3.2　data transmission
10.4　Content Provider's notification for data Change
10.4.1　register ContentObserver
10.4.2　send notification for data Change
chapter 11 start of Zygote and System process
11.1　start script of Zygote process
11.2　start of Zygote
11.3　start of System
chapter 12 start of Android application
12.1　creation of application process
12.2　start of Binder thread pool
12.3　creation of message loop
chapter 13 message handling
13.1　creation of message queue
13.2　message cycle
13.3　message sending
13.4　message handling
chapter 14 keyboard message handling
14.1　keyboard message handling model
14.2　start of InputManager
14.2.1　creation of InputManager
14.2.2　start of InputManager
14.2.3　start of InputDispatcher
14.2.4　start of InputReader
14.3　register InputChannel
14.3.1　creation of InputChannel
14.3.2　register InputChannel of server side
14.3.3　register the window of currently activated application
14.3.4　register InputChannel of client side
14.4　Dispatch of the keyboard message
14.4.1　InputReader gets keyboard event
14.4.2　InputDispatcher dispatch keyboard event
14.4.3　the window of currently activated application gets keyboard message
14.4.4　InputDispatcher gets notification when the keyboard event handling is done
14.5　unregister InputChannel
14.5.1　destroy application window
14.5.2　unregister InputChannel of client side
14.5.3　unregister InputChannel of server side
chapter 15 message loop model of Android application thread
15.1　message loop model of application main thread
15.2　message loop model of nongraphic application child thread
15.3　message loop model of graphic application child thread
chapter 16 installation and showup of Android application
16.1　installation of application
16.2　showup of application
chapter 17 UI system
17.1 UI architecture
17.2 application UI framework
17.3 WindowManagerService
17.4 SurfaceFlinger
17.5 Multiple Display support
chapter 18 resource management framework
18.1 resource management framework
18.2 compilation of resource
18.3 search for resource
chapter 19 Dalvik virtual machine(DVM)
19.1 introduction
19.2 start of DVM
19.3 execution of DVM
19.4 register JNI methods
19.5 DVM process
19.6 DVM thread

send me a pm about my question
llrraa2010 said:
The book is based on android 2.3. If needed, we can rewrite it on newer version.
Click to expand...
Click to collapse
too much things are different between android 2.3 and android 4(maybe 4.4 kitkat) i hope you'll write a modern book :')
please write/publish also a PDF
when will you complete it?
what about the price$$$?
will you send me a free copy, Right?(i am the first to reply! yeah)
jokes aside, send me a private message about info

inside android: build one module of Android alone

1. run envsetup.sh
[email protected]:~/Android$ . ./build/envsetup.sh
after this, there would be some extra commands available:
- croot: Changes directory to the top of the tree.
- m: Makes from the top of the tree.
- mm: Builds all of the modules in the current directory.
- mmm: Builds all of the modules in the supplied directories.
- cgrep: Greps on all local C/C++ files.
- jgrep: Greps on all local Java files.
- resgrep: Greps on all local res/*.xml files.
- godir: Go to the directory containing a file.
2. use mmm to build the module, such as Email app:
[email protected]:~/Android$ mmm packages/apps/Email/
After building, there would be one Email.apk in out/target/product/generic/system/app, where all app files are located here. Also, out/target/product/generic/system/bin for exexcutable, out/target/product/generic/system/lib for library, and out/target/product/generic/system/lib/hw for HAL interface files.
3. repack system.img
[email protected]:~/Android$ make snod
4. run emulator
[email protected]:~/Android$ emulator
about "inside android"
Hi, all.
As experienced android developers or just beginners, sometimes you might have questions about what's going on inside android. Such as how activity starts, how service starts, etc. If we look into the code, we might have the answer, but it takes time and much effort. In fact, there's millions of lines of source code in android. Fortunately, Shengyang Luo wrote a book in chinese to clarify all these questions, with all the related knowledge he got in these years as an experienced engineer of linux/android. Now, he and I decide this book belongs to the android family, so to make more people get benefit from it, we will rewrite it in English.
Also, we want this book to be more adapted to your need, we might share the outline and part of the content of the book here, so that we all can discuss it while rewriting it. We might change the book according to your advice. For example, there might be more details for the parts you are most interested. To do this, we will have pilot PPTs and passages soon. The pilot topics would be as below:
Android design philosophy--everything is component
Initializing a Build/developing/debugging Environment for Android
Android architecture
Hardware Abstraction Layer(HAL)
drivers specific to android
process management
messaging handling
input handling
UI architecture
resource management framework
Dalvik virtual machine
Which is most interested to you? Please share with us, and we will make it available ASAP.
We haven't talked with any press for publishing yet. So if any press is interested in publishing this book, please contact me. As you all know, writing is a tough job, especially when the book is regarding such a complicated android os system, so cheers, encouragement and funding are welcomed badly. Funding would help us not worry about making a living and dedicate to the writing or even hire some people to accelerate the progress. For funding, please visit indiegogo.com and search "inside android".
The book would have 3 parts: say hello to android, drivers specific to android and understanding android in scenes. The first 2 parts are foundation knowledge for the third part, which is the major part of the book.
Part I say hello to android includes 3 chapters, and introduces how to download the source code, build it, run it, smart pointer and Hardware Abstraction Layer(HAL). Since smart pointer is widely used in the native code of application framework, it would be very helpful if we get to know it before we dive into the scenes. Knowing HAL would help us understand the architecture of android, and then make the third part easier to learn.
Part II drivers specific to android also includes 3 chapters, introducing 3 drivers specific to android: Logger Driver, Binder IPC Driver and Ashmem Shared Memory Driver. No doubt, these 3 drivers, especially the Binder IPC Driver, are the foundation of android. knowing about them would definitely help understand android.
Part III understanding android in scenes includes more than 10 chapters, and introduces android in different scenes, such as the start of something(activity, service, zygote, service manager, etc.), process management, messaging handling etc.
The book is based on android 2.3. If needed, we can rewrite it on newer version.
Below is the outline of the book:
Part I: say hello to android
chapter 1　prerequisite
1.1　reference books on Linux kernel
1.2　reference books on Android application developing
1.3　download, build and run Android source code
1.3.1　download Android source code
1.3.2　build Android source code
1.3.3　run Android emulator
1.4　download, build and run Android kernel source code
1.4.1　download Android kernel source code
1.4.2　build Android kernel source code
1.4.3　run Android emulator
1.5　develop one Android application(example)
1.6　build and pack Android module independently
1.6.1　mmm: Builds all of the modules in the supplied directories
1.6.2　build one Android module independently
1.6.3　repack Android images
chapter 2 Hardware Abstraction Layer(HAL)
2.1　develop Android hardware driver
2.1.1　implementation of one kernel driver
2.1.2　modify Kconfig of the kernel
2.1.3　modify Makefile of the kernel
2.1.4　build the driver
2.1.5　test the driver
2.2　test the driver with C executable
2.3　develop HAL module
2.3.1　specification for HAL module
2.3.2　interface for HAL module
2.3.3　load the HAL module
2.3.4　access permission for the hardware
2.4　develop service to the Android hardware
2.4.1　defite the service interface to hardware
2.4.2　implementation of the service
2.4.3　JNI interface for the service
2.4.4　start of the service
2.5　Android application on the service to hardware
chapter 3 smart pointer
3.1　light weight pointer
3.1.1　implementation
3.1.2　example
3.2　strong pointer and weak pointer
3.2.1　strong pointer
3.2.2 weak pointer
3.2.3 example
Part II: drivers specific to android
chapter 4 Logger driver
4.1　Logger format
4.2　Logger driver
4.2.1　data structure
4.2.2　initialization
4.2.3　open the Logger driver
4.2.4　read the log
4.2.5　write the log
4.3　runtime library for the Logger driver
4.4　log interface for C/C++
4.5　log interface for Java
4.6　Logcat tool
4.6.1　data structure
4.6.2　initialization
4.6.3　read the log
4.6.4　output the log
chapter 5 Binder IPC Driver
5.1　Binder driver
5.1.1　data structure
5.1.2　initialization
5.1.3　open
5.1.4　mapping to the memory
5.1.5　kernel cache management
5.2　runtime library for Binder IPC Driver
5.3　example for Binder IPC Driver application
5.4　counting for Binder object reference
5.4.1　lifecycle of Binder local object
5.4.2　lifecycle of Binder instance
5.4.3　lifecycle of Binder reference
5.4.4　lifecycle of Binder proxy
5.5　Binder object death notification
5.5.1　register death notification
5.5.2　send death notification
5.5.3　unregister death notification
5.6　start of Service Manager
5.6.1　open and mapping on Binder
5.6.2　register as Binder context manager
5.6.3　wait for the Client request in loop
5.7　getting the Service Manager proxy
5.8　start of Service
5.8.1　register Service
5.8.2　start of Binder thread pool
5.9　how to get Service proxy
5.10　Java interface for Binder IPC Driver
5.10.1　getting the Java proxy of Service Manager
5.10.2　define the Java service
5.10.3　start of Java service
5.10.4　getting Java service proxy
5.10.5　calling Java service
chapter 6 Ashmem Shared Memory Driver
6.1　Ashmem driver
6.1.1　data structure
6.1.2　initialization
6.1.3　open
6.1.4　mapping to the memory
6.1.5　lock and unlock
6.1.6　recycle
6.2　Ashmem interface in runtime library cutils
6.3　C++ interface for Ashmem
6.3.1　MemoryHeapBase
6.3.2　MemoryBase
6.3.3　example
6.4　Java interface for Ashmem
6.4.1　MemoryFile
6.4.2　example
6.5　how Ashmem shares memory
Part III: understanding android in scenes
chapter 7 start of Activity
7.1　Activity example
7.2　start of root Activity
7.3　start of child Activity in the same process
7.4　start of child Activity in the new process
chapter 8 start of Service
8.1　Service example
8.2 start of Service in the same process
8.3　start of Service in the new process
chapter 9 Broadcast
9.1　example
9.2　register
9.3　send
chapter 10 Content Provider
10.1　Content Provider example
10.1.1　ArticlesProvider
10.1.2　Article
10.2　start of Content Provider
10.3　data sharing of Content Provider
10.3.1　data sharing model
10.3.2　data transmission
10.4　Content Provider's notification for data Change
10.4.1　register ContentObserver
10.4.2　send notification for data Change
chapter 11 start of Zygote and System process
11.1　start script of Zygote process
11.2　start of Zygote
11.3　start of System
chapter 12 start of Android application
12.1　creation of application process
12.2　start of Binder thread pool
12.3　creation of message loop
chapter 13 message handling
13.1　creation of message queue
13.2　message cycle
13.3　message sending
13.4　message handling
chapter 14 keyboard message handling
14.1　keyboard message handling model
14.2　start of InputManager
14.2.1　creation of InputManager
14.2.2　start of InputManager
14.2.3　start of InputDispatcher
14.2.4　start of InputReader
14.3　register InputChannel
14.3.1　creation of InputChannel
14.3.2　register InputChannel of server side
14.3.3　register the window of currently activated application
14.3.4　register InputChannel of client side
14.4　Dispatch of the keyboard message
14.4.1　InputReader gets keyboard event
14.4.2　InputDispatcher dispatch keyboard event
14.4.3　the window of currently activated application gets keyboard message
14.4.4　InputDispatcher gets notification when the keyboard event handling is done
14.5　unregister InputChannel
14.5.1　destroy application window
14.5.2　unregister InputChannel of client side
14.5.3　unregister InputChannel of server side
chapter 15 message loop model of Android application thread
15.1　message loop model of application main thread
15.2　message loop model of nongraphic application child thread
15.3　message loop model of graphic application child thread
chapter 16 installation and showup of Android application
16.1　installation of application
16.2　showup of application
chapter 17 UI system
17.1 UI architecture
17.2 application UI framework
17.3 WindowManagerService
17.4 SurfaceFlinger
17.5 Multiple Display support
chapter 18 resource management framework
18.1 resource management framework
18.2 compilation of resource
18.3 search for resource
chapter 19 Dalvik virtual machine(DVM)
19.1 introduction
19.2 start of DVM
19.3 execution of DVM
19.4 register JNI methods
19.5 DVM process
19.6 DVM thread

new open source library ogles_gpgpu: GPGPU for Android and iOS using OpenGL ES 2.0

(please note: I would like to post this to the developer forum, but since I just registered, I'm not allowed to do so. furthermore, I'm not allowed to post direct links (see below))
I would like to announce a new library for GPU-based processing on mobile systems – ogles_gpgpu.
As many of you know, it is often beneficial in terms of performance and energy efficiency to execute certain processing tasks on the GPU instead of the CPU. This is especially the case for image processing tasks. ogles_gpgpu enables fast and portable, GPU-powered processing by using OpenGL ES 2.0 shaders.
Since transferring data to and from the GPU is often a bottleneck for GPU processing, platform-specific fast texture access is also implemented. The library is written in well documented, clear C++ code. An interface for Android systems via JNI is provided. Example applications show how to use this library. All code is LGPL licensed.
There are several scenarios on how to use this library: You can for example pass image data (or arbitrary byte data) to ogles_gpgpu, which creates an OpenGL texture from it. You can then process it on the GPU by applying a series of filters (OpenGL shaders) on it. This kind of rendering happens off-screen. Afterwards, you can lock the result data and obtain a pointer to it. You can then copy this data for future processing or directly analyze or modify it. Another possible scenario is to directly pass an OpenGL texture ID as input for ogles_gpgpu. This is for example beneficial if you can obtain camera frames as OpenGL texture from the camera API of your target platform (both Android and iOS allow this and example projects or provided for this). Now ogles_gpgpu can directly run the filters on this camera frame texture. This can happen off-screen or optionally on-screen, which means that the result image is also displayed to a render surface. After processing on the GPU side, you can access the result data again as described in the first scenario. By this, you can do further CPU-based processing of the result data. This is for example necessary, if certain algorithms can not (efficiently) be implemented as OpenGL shaders.
At current development stage, there are not so many image processing filters implemented, yet, but this is about to change in the future. The most important thing is that a portable architecture for GPU-based processing is available, which allows fast texture access by using platform-specific optimizations.
You can check out the project on github: github.com/internaut/ogles_gpgpu
More information can be found on my personal website: mkonrad.net/projects/ogles_gpgpu.html
Thank you.

Android Webview and webvr / webxr

Spend several days trying to make Android Webview go webvr / webxr
​
I have several projects to develop in VR and I used Unuty and Unreal for this. But with webvr/webxr coming it is a chance to get unified platform to distribute VR experience. Only problem I need to distribute it as stand-along app for google play and gearvr store (due to heavy video 360 4k/6k content - beyond Wi-Fi b/g/n capabilities).
​
Test device has android 7.1 - so webview is the chrome itself and in dev options set as chrome. Also tried to set it as chrome dev, chrome canary - no luck
​
my discovery for now
for webvr - this technology named deprecated and you need to set chrome://flags on normal chrome and it works well. Webview ignores that settings (might be if chrome://flags accessed from Webview it helps - have`t tried yet). There is an option to use chrome start-up parameters via adb but no good for production
​
for webxr you need to acquire Origin Trial Token from Goolge. And again stand alone chrome if accessing site with Origin Trial Token goes VR at no probs. But Webview simply ignores Token .
​
For now I have only one proper option - compile chromium and embed it in app. On windows I can distribute app using WinJS (.net and win32 also good) + Microsoft EDGE Webview. Works surprisingly well with Windows Mixed Reality. For SteamVR again I can compile Gecko or use supermedium (Gecko compiled by other guys).
​
But on android I still in search for way to show Webview with local html with my webxr content (based on threejs mostly).

Hilo

ARTist: The Android Runtime Instrumentation and Security Toolkit

NOT MINE!
video: ARTist - An Instrumentation Framework for Reversing and Analyzing Android Apps and the Middleware
site: https://artist.cispa.saarland/
github: https://github.com/Project-ARTist
Overview
ARTist is a flexible open source instrumentation and hybrid analysis framework for Android apps and Android’s Java middleware. It is based on the Android Runtime’s (ART) compiler and modifies code during on-device compilation.
ARTist blends particularly well into the Android app install mechanism because it does not change the app’s package (APK) file but just replaces the compiled native version, hence it preserves the package signature so that modified apps still receive updates.
Use cases
Depending on whether you just want to use ARTist as an applications to leverage existing instrumentation and analysis modules or if you want to actively develop using ARTist, there is documentation available for the different use cases:
ARTist for Users explains how to use ready-made ARTist versions that are packages as Android apps (APK files). With these instructions, you can download & install ARTist and start trying it out immediately.
ARTist for developers provides much more insight into the overall ARTist Ecosystem and explains how to setup your machine to get started with developing with and for ARTist.
Applications & Deployment
ARTist can be used to instrument arbitrary third party apps and Androids Java middleware layer (e.g., system server). When applied to apps, the ARTist compiler is shipped to a device via the ArtistGui app that provides the user with full control over the analysis and instrumentation of installed apps. Users can pick applications for recompilation so that the current modules can take effect. The app also allows to keep apps instrumented so that the recompilation step is repeated immediately after the target app is updated.
ARTist can also be used in custom ROMs to instrument apps when they are installed or even the middleware itself. Instead of deploying it to the application later using ArtistGui app, in this case ARTist replaces the system compiler so that it is automatically triggered as a part of the default toolchain to install applications. The middleware, however, is mostly compiled at first boot and after major system updates.
For more information about the different deployment options, have a look at the developer’s page.
Modularization
The true power of ARTist is leveraged by using its module framework. While ARTist only provides the framework, the actual business logic is implemented in different modules that can be chosen and combined. A module encapsules the implementation of a certain use case with all its dependencies. For more information on what exactly modules are and how to create them, please refer to the module page.
Typical use cases for ARTist modules include:
Method hooking: Inline Reference Monitoring, method call tracing, …
Debugging and analysis of third-party applications
Customization of applications and the system
You can find ready-made module packages in our binary section. You can just download them to your device, import the module via ArtistGui and apply them to your apps.
Problem: stop the development (stable in android 7.1). Anyone know something similar like this? (they compare it with xposed and Frida).