Related
This all works on stock/nonroot phones
I got Mono running on Android.
http://www.koushikdutta.com/2008/12/mono-on-android-success-at-last.html
Started working on Java/C# interop and found out that DLR works on Mono:
http://www.koushikdutta.com/2009/01/microsoft-dlr-and-mono-bring-python-and.html
As a result, you can write applications in Python and Ruby on Android too now.
Anyhow, if anyone else is interested in working on this project with me, please let me know! I've already gotten all the relevent source hosted at Google Code: http://code.google.com/p/androidmono. Basically the next bit of work involves implemeting a Java interop using the DLR.
Nifty.
As for Dalvik & JIT, I think dexopt already replaces some heavy usage calls with inline native code. Hopefully dalvik vm will get full JIT in the future?
This makes me very happy!!!
Thank you for your work!!
jashsu said:
Nifty.
As for Dalvik & JIT, I think dexopt already replaces some heavy usage calls with inline native code. Hopefully dalvik vm will get full JIT in the future?
Click to expand...
Click to collapse
It doesn't do any inline native replacements: dexopt optimizes the dex file. Which includes inline dex byte code replacements. There is no JIT at all, but Google said it would be something definitely on the horizon. Personally I think DEX is a pretty stupid move on Google's part; they could have just gone with CIL-- and write a Java compiler for that instead. The Mono JIT compiler works on many platforms; so V1 of Android could have been running JIT compiled native code with that route... which is an order of magnitude better in performance.
I'm going to be doing some performance comparisons of Mono vs Dalvik; Mono will obviously win, but it will be interesting to see the margin. I'll also experiment with binding Mono to the Android runtime to create Android applications in C#.
Optimization
Virtual machine interpreters typically perform certain optimizations the first time a piece of code is used. Constant pool references are replaced with pointers to internal data structures, operations that always succeed or always work a certain way are replaced with simpler forms. Some of these require information only available at runtime, others can be inferred statically when certain assumptions are made.
The Dalvik optimizer does the following:
For virtual method calls, replace the method index with a vtable index.
For instance field get/put, replace the field index with a byte offset. Also, merge the boolean / byte / char / short variants into a single 32-bit form (less code in the interpreter means more room in the CPU I-cache).
Replace a handful of high-volume calls, like String.length(), with "inline" replacements. This skips the usual method call overhead, directly switching from the interpreter to a native implementation.
Prune empty methods. The simplest example is Object.<init>, which does nothing, but must be called whenever any object is allocated. The instruction is replaced with a new version that acts as a no-op unless a debugger is attached.
Append pre-computed data. For example, the VM wants to have a hash table for lookups on class name. Instead of computing this when the DEX file is loaded, we can compute it now, saving heap space and computation time in every VM where the DEX is loaded.
All of the instruction modifications involve replacing the opcode with one not defined by the Dalvik specification. This allows us to freely mix optimized and unoptimized instructions. The set of optimized instructions, and their exact representation, is tied closely to the VM version.
Most of the optimizations are obvious "wins". The use of raw indices and offsets not only allows us to execute more quickly, we can also skip the initial symbolic resolution. Pre-computation eats up disk space, and so must be done in moderation.
There are a couple of potential sources of trouble with these optimizations. First, vtable indices and byte offsets are subject to change if the VM is updated. Second, if a superclass is in a different DEX, and that other DEX is updated, we need to ensure that our optimized indices and offsets are updated as well. A similar but more subtle problem emerges when user-defined class loaders are employed: the class we actually call may not be the one we expected to call.
These problems are addressed with dependency lists and some limitations on what can be optimized.
Click to expand...
Click to collapse
Koush said:
Personally I think DEX is a pretty stupid move on Google's part; they could have just gone with CIL-- and write a Java compiler for that instead. The Mono JIT compiler works on many platforms; so V1 of Android could have been running JIT compiled native code with that route... which is an order of magnitude better in performance.
I'm going to be doing some performance comparisons of Mono vs Dalvik; Mono will obviously win, but it will be interesting to see the margin. I'll also experiment with binding Mono to the Android runtime to create Android applications in C#.
Click to expand...
Click to collapse
Google had different objectives, they didn't go after maximum performance. Remember, that handsets have different constrains than desktops and laptops. So they went after minimizing RAM usage (byte code interpreter => maximum possible sharing of read-only memory pages among processes) and battery life. Performance had to be acceptable, not priority.
You would not be able to fit everything into RAM if you used Mono and you would get the patent problems with Net/Mono/etc as a bonus.
lu_tze said:
Google had different objectives, they didn't go after maximum performance. Remember, that handsets have different constrains than desktops and laptops. So they went after minimizing RAM usage (byte code interpreter => maximum possible sharing of read-only memory pages among processes) and battery life. Performance had to be acceptable, not priority.
You would not be able to fit everything into RAM if you used Mono and you would get the patent problems with Net/Mono/etc as a bonus.
Click to expand...
Click to collapse
.NET, C#, IL, et al are all ECMA standards. Mono is LGPL/GPL. There are no patent or licensing issues with it that is unfamiliar to the OHA. They reuse plenty of open source projects.
An interpreter is not power efficient OR performant, simply due to the fact is it doing the 10 times as much work to do the same thing as native code. In addition, Mono features an Ahead Of Time compiler (AOT) that would let you compile everything to native code before it even hits the phone (or just once, and cache it). Most of Android's power and memory optimizations currently comes from Google's application life cycle (activities can be killed and resumed at the system's whim) -- that has nothing to do with Dalvik. I'm not criticizing the API or the implementation, just the runtime.
They could have spent their time making the Mono runtime play nicely with the shared memory subsystem.
I'm rebuilding mono with a minimal configuration to check out the disk and memory footprint.
Koush said:
Not that most of you will care, but I got Mono running on Android.
Click to expand...
Click to collapse
I'm a noob.... how can I install this? Very good Job Kush!
pic.micro23 said:
I'm a noob.... how can I install this? Very good Job Kush!
Click to expand...
Click to collapse
I haven't released anything yet. I'm trying to figure out how to statically link all it's dependencies, minimize the size, bind to the Android runtime, convert DEX to CIL and then CIL to ARM, and all sorts of other goodness. Basically a lot of experimenting to do before anything is "released". It's just in proof of concept phase right now.
Dalvik sucks
http://www.koushikdutta.com/2009/01/dalvik-vs-mono.html
Koush said:
Dalvik sucks
http://www.koushikdutta.com/2009/01/dalvik-vs-mono.html
Click to expand...
Click to collapse
Lol. nice article Koush. It's surprising mono is that much faster
Koush said:
I haven't released anything yet. I'm trying to figure out how to statically link all it's dependencies, minimize the size, bind to the Android runtime, convert DEX to CIL and then CIL to ARM, and all sorts of other goodness. Basically a lot of experimenting to do before anything is "released". It's just in proof of concept phase right now.
Click to expand...
Click to collapse
Just read the dalvik vs mono article. It's certainly interesting work. I agree that by ignoring JIT they're certainly not going after the most beneficial optimizations. That said, I don't think it's something they've completely excluded from future implementation.
I think you should consider trying to get Mono added as an external project. If nothing, having unofficial support for a vm which supports C#/CIL could bring in a significant amount of developer interest from the WinMo dev community. The coretech team would be the folks to set up a new project.
I've now gotten mono working on all G1s. You don't need Debian OR root. Still a couple kinks to work out, but I have it on the market for anyone interested in playing with it. More information at the link below.
http://www.koushikdutta.com/2009/01/mono-for-android-now-available-on.html
I thought this was only a platform for development but its made my g1 much faster and reduced memory from steel and stock browsers as well. My market is still 12 mb and mono is about 11 mb. Is this normal?
Also everytime, I run mono, does it do the same thing it does the first time it was installed and opened?
Sorry, i know nothing about mono but i can tell its definitely optimizing the performance on the g1 though.
great work koushe,
hbguy
This awesome, I have been waiting to see how this all turned out after reading your first post about it a few days ago.
hbguy said:
I thought this was only a platform for development but its made my g1 much faster and reduced memory from steel and stock browsers as well. My market is still 12 mb and mono is about 11 mb. Is this normal?
Also everytime, I run mono, does it do the same thing it does the first time it was installed and opened?
Sorry, i know nothing about mono but i can tell its definitely optimizing the performance on the g1 though.
great work koushe,
hbguy
Click to expand...
Click to collapse
Koush would know more than I would, but installing mono shouldn't affect everything else on Android. It's not like everything is suddenly using mono instead of dalvik. I suspect you have a strong case of the placebo effect
hbguy said:
I thought this was only a platform for development but its made my g1 much faster and reduced memory from steel and stock browsers as well. My market is still 12 mb and mono is about 11 mb. Is this normal?
Also everytime, I run mono, does it do the same thing it does the first time it was installed and opened?
Sorry, i know nothing about mono but i can tell its definitely optimizing the performance on the g1 though.
great work koushe,
hbguy
Click to expand...
Click to collapse
Yeah, you're just imagining things I haven't even attempted like DEX->CIL yet.
The first APK of Mono is quite large though. I've updated it with a number of bug fixes and also am making it use eglib now. This trimmed the size by a few MB. Getting Mono to work with Bionic might not be possible... (that would trim off another 2MB).
Once again, the APK is just a developers release... something to play with and test.
I have been messing with mono on my G1.
Is it safe to say it will only work with command line apps?
I got my own hello world and a few other things running, but if I try and run any sort of gui I get errors.
Yes, only command line stuff will work. WinForms will not work on Android.
However, you should be able to get OpenGL ES working via PInvoke. I haven't tried it, but it should work just fine.
Koush said:
Yes, only command line stuff will work. WinForms will not work on Android.
Click to expand...
Click to collapse
That is what I thought, I think PInvoke is just a bit out of my skill set.
Thanks for the work none the less.
I got mono building in the Android build environment, and the Mono team accepted a patch to make it work on Android. There's also some changes external to Mono which can be found at the androidmono google code repository:
http://code.google.com/p/androidmono/
I haven't found a good source of discussion on this topic but has anyone experimented with gcc optimization flags when compiling the kernel or a ROM? I dug around and found sparse information on the subject and rarely is Android specifically mentioned. Just curious if anyone has 2 cents on the subject.
I've been trying to figure out how to implement using these flags in kernel compilation but I'm confused if you're supposed to put these in your makefile (both main one and the arch\arm\... makefile?)
is ffast-math preferred over some of the Linaro blog post flags ("We’ve switched from a base of -O2 -fno-strict-aliasing to -O3 -fmodulo-sched -fmodulo-sched-allow-regmoves.")
I've tried sifting through Linaro git repos to see anyone actually using any flags they mention but have yet to find anyone using them yet they say they do somewhere.
GCC Optimization Flags Reference http://gcc.gnu.org/onlinedocs/gcc-4.0.4/gcc/Optimize-Options.html
Some background information on the Linaro blogs about -o3 make flag
Compiler flags used to speed up Linaro Android 2011.10, and future optimizations
People trying out the latest Linaro Android builds may notice they’re faster than the older versions. One of the reasons for this is that we’re using a new set of compiler flags for this build.
We’ve switched from a base of -O2 -fno-strict-aliasing to -O3 -fmodulo-sched -fmodulo-sched-allow-regmoves.
To optimize application startup, we’ve also switched the linker hash style to GNU (ld --hash-style=gnu), and patched the Android dynamic linker to deal with those hashes.
Getting rid of -fno-strict-aliasing enables rather efficient additional optimizations – but requires that the code being compiled follows some rules traditionally not enforced by compilers.
Given the traditional lack of enforcement, there’s lots of code out there (including, unfortunately, in the AOSP codebase) that violates those rules.
Fortunately, gcc can help us detect those violations: With -fstrict-aliasing -Werror=strict-aliasing, most aliasing violations can be turned into built time errors instead of random crashes at runtime. This allowed us to detect many aliasing violations in the code, and fix them (where doable with reasonable effort), or work around them by enabling -fno-strict-aliasing just for a particular subdirectory containing code not ready for dropping it.
-O3 enables further optimizations not yet enabled at -O2, including -finline-functions, -funswitch-loops, -fpredictive-commoning, -fgcse-after-reload, -ftree-vectorize and -fipa-cp-clone.
While some of them are probably not ideal (e.g. -finline-functions tends to increase code size, thereby also increasing memory usage and, in a bad case, reducing cache efficiency), overall -O3 has shown to increase performance.
-fmodulo-sched and -fmodulo-sched-allow-regmoves are fairly new optimizations not currently automatically enabled at any -O level. These options improve loop scheduling – more information can be found here.
We optimized further by adding link-time optimizations in some relevant libraries, and by using -ffast-math in selected parts of the code. -ffast-math is a bit dangerous because it can cause math functions to return incorrect values by ignoring parts of the ISO and IEEE rules for math functions (parts that make optimizations harder, or that simply require additional checks that slow down things considerably), so it should usually not be used for an entire build – but enabling it for parts of the code verified to not rely on exact ISO/IEEE rules can produce quite a speedup.
We also added the option to specify extra optimizations in a board specific config – so now our Panda builds are optimized specifically for Cortex-A9 CPUs while the iMX53 builds optimize for Cortex-A8 instead of relying on the least common denominator.
We also experimented with Graphite related optimizations, such as -fgraphite-identity, -floop-block, -floop-interchage, -floop-strip-mine, -ftree-loop-distribution and -ftree-loop-linear – those optimizations can rearrange loops to allow further optimizations. We’ve turned them back off for the release because they introduced some stability problems, and the benefit seemed minimal.
However, chances are they will come back in a future build. They can help make more efficient use of multi-core CPUs (with the addition of -ftree-parallelize-loops) once the compiler knows how to to threading (currently, Android is built using a generic arm-eabi targeted compiler that has no knowledge of the underlying OS).
Other possible future improvements – though probably not as efficient as the ones already made, or the switch to -ftree-parallelize-loops for multi-core boards – include seeing what effect a switch between ARM and Thumb2 instructions has on performance (enabling the right mode in the right modules), identifying places where the increased code size of -O3 actually hurts, and add the likes of -fno-inline-functions there, identifying further performance critical parts that can handle -ffast-math, fixing the aliasing violations we’ve worked around this time, and – of course – switching to gcc 4.7 when it becomes usable.
make -o3 -o -fmodulo-sched -o -fmodulo-sched-allow-regmoves -Wl,--hash-style=gnu -Werror=strict-aliasing ARCH=arm CROSS_COMPILE=~/(toolchain path)/bin/arm-linux-gnueabihf- zImage modules -j 4
is this how you'd do a make according to the instructions on the linaro blog? This is the only way I could do it for it to actually run something but it doesn't tell me that it's passing these flags
hi all,
I've updated DOSBOX's dynrec core to take advantage of the THUMB2 ARMv7 instructions that must be supported on Windows RT. Among other things, this includes being able to move immediates to registers without using a PC relative load, 32-bit instructions that allow access to all registers, etc. I also added a PLI instruction to preload a cacheblock into the instruction cache before it is executed, and PLD instructions to prefetch data for a minor speedup.
Testing on doom timedemo, I get 3721 realtics, which is a modest 5% speedup from the standard dynrec core. Probably not having directX is a significant bottleneck on video drawing, this can get fixed once I get a wrapper around directinput working. Hopefully these changes can also help out mamaich's x86 pe loader out a well.
-- old post below--
I managed to get dosbox compiled with mamaich's new updates (see below), so we now have dosbox with the dynamic recompiling core. I also included network support which seems to work.
The dynamic recompiling core does give better performance for newer apps. 3982 realtics vs 18088 realtics on doom timedemo, so about a 5x gain
The next step would be to get a better graphics core. There is an opengl wrapper for DX11 called gl2dx that I'm looking into.
I'm also looking at putting at optimizing dynrec for thumb2, for example, it seems that CBZ might be useful (except it only lets you branch forward!!!)
Hmm, I have to unfortunately note that DOS4GW doesn't seem to work in this build. I'm working on it!
In the mean time, enjoy commander keen 1 -6?...
no2chem said:
Hmm, I have to unfortunately note that DOS4GW doesn't seem to work in this build. I'm working on it!
In the mean time, enjoy commander keen 1 -6?...
Click to expand...
Click to collapse
So in the end, I think the dynamic recompiler has some issues.
I know that the VirtualAlloc function is correctly setting execute permissions on memory pages, but I don't know if FlushInstructionCache is actually flushing the instruction cache. It's also possible that the dynamic recompiler is emitting bad assembly, but inspecting the code... doesn't look like it!
no2chem said:
So in the end, I think the dynamic recompiler has some issues.
Click to expand...
Click to collapse
The same issue for me (but I'm using a bit obsolete DosBox sources, need to update). May be this is due to EABI changes (for example registers are not preserved), as DosBox dynamic core targets Linux.
Regarding DosBox. Look here - http://vogons.zetafleet.com/viewtopic.php?t=31787
This is the ARMv7 optimized dynamic core for DosBox by M-HT. It is not yet integrated into the official DosBox branch.
If you're concerned about FlushInstructionCache not being implemented correctly, let me know. As part of my work to port Chromium, I've learned how to do cache management in ARM a little closer to the metal (moving the relevant values to the coprocessor registers directly) and have both more control (for example, clearing the data cache lines as well) and more precise control over exactly what is done to the cache lines.
However, bear in mind that I haven't been able to properly test this yet, as too much of the Chromium core still doesn't want to compile for me... *sigh*.
GoodDayToDie said:
If you're concerned about FlushInstructionCache not being implemented correctly, let me know.
Click to expand...
Click to collapse
As far as I see - FlushInstructionCache works as expected (flushes icache). DosBox crashes due to some other reason, maybe due to the ARM-THUMB interworking problems (it is just a guess).
Good to know. It would be annoying if the official API were wrong...
I've found whu DosBox dynamic core crashes at random.
All existing dynamic cores use ARM code, even the thumb files are using the ARM prologue. Though our CPUs can run the ARM code, there is a bug (?) in RT that switches ARM code to be executed as a THUMB at random. For example you call an ARM procedure 100000 times, but at 100001 something happens and the code starts to execute as THUMB. Maybe a task switch occurs, or maybe some interrupt happens, and when OS returns to your code - it sets T bit in CPSR, so the code is interpreted as THUMB.
I was able to modify the prologue generator in risc_armv4le-thumb-iw.h so that it generates only THUMB code, and now everything started to work in my project. Attached. Code is ugly and unfinished and uses some dirty C hacks, I'll fix them later as it is deep night now. But you may look on the changes and implement similar things in a DosBox build.
mamaich said:
I've found whu DosBox dynamic core crashes at random.
All existing dynamic cores use ARM code, even the thumb files are using the ARM prologue. Though our CPUs can run the ARM code, there is a bug (?) in RT that switches ARM code to be executed as a THUMB at random. For example you call an ARM procedure 100000 times, but at 100001 something happens and the code starts to execute as THUMB. Maybe a task switch occurs, or maybe some interrupt happens, and when OS returns to your code - it sets T bit in CPSR, so the code is interpreted as THUMB.
I was able to modify the prologue generator in risc_armv4le-thumb-iw.h so that it generates only THUMB code, and now everything started to work in my project. Attached. Code is ugly and unfinished and uses some dirty C hacks, I'll fix them later as it is deep night now. But you may look on the changes and implement similar things in a DosBox build.
Click to expand...
Click to collapse
I'll take a look at getting that implemented in my build.
Edit: Haven't looked into it much, but DosBox crashes whenever I open any 32-bit stuff.
mamaich said:
I've found whu DosBox dynamic core crashes at random.
All existing dynamic cores use ARM code, even the thumb files are using the ARM prologue. Though our CPUs can run the ARM code, there is a bug (?) in RT that switches ARM code to be executed as a THUMB at random. For example you call an ARM procedure 100000 times, but at 100001 something happens and the code starts to execute as THUMB. Maybe a task switch occurs, or maybe some interrupt happens, and when OS returns to your code - it sets T bit in CPSR, so the code is interpreted as THUMB.
I was able to modify the prologue generator in risc_armv4le-thumb-iw.h so that it generates only THUMB code, and now everything started to work in my project. Attached. Code is ugly and unfinished and uses some dirty C hacks, I'll fix them later as it is deep night now. But you may look on the changes and implement similar things in a DosBox build.
Click to expand...
Click to collapse
thanks for figuring that out! its a bit odd though, don't you think - wouldn't normal ARM apps crash if this was the case? I'm sure most stuff is compiled with interworking, maybe ms interworking convention is different?
Anyway, I was thinking, doesn't homm3 use directx? Is your emulation layer also doing conversion of that code? It seems like it would be useful to have native wrappers for old versions of DX so they can take direct advantage of DX11 native. Alas, that's for another day.... Also, mfc would be nice too... (its there... MS just didn't provide the .lib, ordinals are [noname]'d out, and .pdb from debug server seems to be missing some exports....)
no2chem said:
hi all,
I managed to get dosbox compiled with mamaich's new updates (see below), so we now have dosbox with the dynamic recompiling core. I also included network support which seems to work.
The dynamic recompiling core does give better performance for newer apps.
The next step would be to get a better graphics core. There is an opengl wrapper for DX11 called gl2dx that I'm looking into.
I'm also looking at putting at optimizing dynrec for thumb2, for example, it seems that CBZ might be useful (except it only lets you branch forward!!!)
Click to expand...
Click to collapse
How big performance gains are you talking about?
bartekxyz said:
How big performance gains are you talking about?
Click to expand...
Click to collapse
Eh, 3982 realtics vs 18088 realtics on doom timedemo, so about a 5x gain
Nice! That should (eventually) allow quite a few legacy apps, if they're old enough and/or not to performance-sensitive (i.e. not realtime) to run acceptably.
no2chem said:
doesn't homm3 use directx? Is your emulation layer also doing conversion of that code?
Click to expand...
Click to collapse
No conversion, I just pass DX to the native OS libraries with some minimal wrappers. RT provides even DirectX 1.0 interfaces
The only problem is that RT prohibits non 32-bit color modes, though video driver supports them (at least 16 bit color is supported by tegra driver).
no2chem said:
I managed to get dosbox compiled with mamaich's new updates
Click to expand...
Click to collapse
I'm using ARM/Tumb dynamic code created by M-HT: http://vogons.zetafleet.com/viewtopic.php?t=31787
I've changed just one function.
I'm also looking at putting at optimizing dynrec for thumb2, for example, it seems that CBZ might be useful (except it only lets you branch forward!!!)
Click to expand...
Click to collapse
Please post your code changes somewhere, so that I could use them in my project too
don't you think - wouldn't normal ARM apps crash if this was the case? I'm sure most stuff is compiled with interworking, maybe ms interworking convention is different?
Click to expand...
Click to collapse
NTARM kernel is a THUM2-only kernel. There is no interworking it it. So it is possible that MS code intentionally sets the T bit to THUMB when returning from interrupt or a task-switch. Unfortunately this means that we can't use ARM code in our programs (I have not seen any ARM code in MS progs, only THUMB2). Anyway, THUMB2 code is very efficient now, its speed can be the same as of ARM code, and size is much smaller.
Generally, THUMB2 should outperform ARM, yes. Also, the state of Thumb vs. Arm is actually set in a kind of weird way: rather than manually setting a processor bit, it's controlled by the jump address. Since even THUMB2 instructions are always 16-bit (2-byte) aligned, the least significant bit of an instruction address is never 1. Therefore, that bit is used (on any jump-type instruction) as a flag to indicate whether the processor should use ARM or THUMB2 mode. In theory, this should mean that the OS doesn't care about the mode being used... but I've never written an interrupt handler for ARM (a couple other platforms, but not ARM) and there might be something weird going on with them.
GoodDayToDie said:
rather than manually setting a processor bit, it's controlled by the jump address
Click to expand...
Click to collapse
This is true for normal jumps. And that is why we have to set lowest bit to 1 manually on indirect branches inside autogenerated THUMB code. But for CPU itself the mode is kept in T bit in CPSR. And "something" sets it to 1 "sometimes". I have not done much testing, but I think that this should be an interrupt, as I've seen that bit changed in the middle of a generated ARM code.
Anyway the THUMB2 code is better then ARM for us - it is more compact, so more code may be kept in cache (both in CPU and DosBox dynamic recompilator's).
updated with a thumb2 dynrec.
Touch input for mouse
For anyone that wants to use their touchscreen for mouse input, navigate to their %AppData%\Local\DosBox folder and modify the dosbox-0.74.conf file so that "Autolock=true" is "Autolock=false"
It works pretty well with the touchscreen and an on-screen keyboard for gaming and such
Some budding devs like me and some others have asked this question and got this answer!
Firstly I want to thanks all who supported me!
My Parents for buying me an Android Device and Supporting
-CALIBAN666- for his thread
franciscofranco for his definition on zRam
abhisahara
Sniper Killer
And all those whom I have forgotten to mention!
Originally posted by Wikipedia
Q: What is zRam?
A: zRam is a module of the Linux kernel, previously called "compcache". zRam increases performance by avoiding paging on disk and instead uses a compressed block device in RAM in which paging takes place until it is necessary to use the swap space on the hard disk drive. Since using RAM is faster than using disks, zRam allows Linux to make more use of RAM when swapping/paging is required, especially on older computers with less RAM installed.
Google has also said to enable zRam as default for Chrome OS Devices!
Click to expand...
Click to collapse
Originally posted by franciscofranco
The zram module creates RAM based block devices: /dev/ramX (X = 0, 1, ...).
Pages written to these disks are compressed and stored in memory itself.
These disks allow very fast I/O and compression provides good amounts of
memory savings.
Basically is for storing swapped pages into compressed memory ram.
Click to expand...
Click to collapse
-CALIBAN666- said:
I think its better to Post this here,when its not better,than sorry!!!
-----------------------------------------------------------------
Once a brief statement for those who are not traveling so long in the Android scene:
ZRAM = ramzswap = Compcache
In order to explain more precisely ZRAM first need other terms are more clearly defined:
Swap can be compared with the swap file on Windows. If the memory (RAM) to complete the PC the data that are being used not actively outsource (eg background applications) so as to re-evacuate RAM free. To this data is written to a hard disk. If required, this data is then read back from there easily. Even the fastest SSD is slower than the RAM. On Android, there is no swap!
In ZRAM unnecessary storage resources are compressed and then moved to a reserved area in the fixed RAM (ZRAM). So a kind of swap in memory.
This Ram is more free because the data then only about 1/4 of the former storage requirements have. However, the CPU has to work in more because they compress the data has (or unpack again when they are needed). The advantage clearly lies in the speed. Since the swap partition in RAM is much faster than this is a swap partition on a hard drive.
In itself a great thing. But Android does not have a swap partition, and therefore brings Android ZRAM under no performance gain as would be the case with a normal PC.
In normal PC would look like this:
Swap = swap file (on disk) -> Slow
ZRAM (swap in RAM) -> Faster than swap
RAM -> Quick
With Android, there is no swap partition, and therefore brings ZRAM also no performance boost.
The only thing that brings ZRAM is "more" RAM. Compressed by the "enlarged" so to speak of the available memory. That's on devices with little RAM (<256MB) also pretty useful. The S2 has 1GB but the rich, and more than. There must not be artificially pushed up to 1.5 GB.
After you activate the ZRAM also has 2 disadvantages. The encoding and decoding using CPU time, which in turn has higher power consumption.
Roughly one can say (For devices with more than 512MB RAM):
Without ZRAM: + CPU Performance | + Battery | RAM
With ZRAM: CPU Performance |-Battery | + RAM
For devices with too little RAM so it makes perfect sense. But who shoots the S2 already be fully complete RAM and then still need more?
Check whether you can ZRAM runs in the terminal with
free or cat / proc / meminfo
I hope it helps to understand zRam!!!!
Click to expand...
Click to collapse
So basically zRam module in kernel increases and optimizes performance!
I didn't write all this information, I just compiled it together in one thread for ease :fingers-crossed:
But making it work is an headache.. you have to add LZO(Lempel–Ziv–Oberhumer) compression through menuconfig and then run zRam scripts through init.d and what not..
robowarrior1377 said:
But making it work is an headache.. you have to add LZO(Lempel–Ziv–Oberhumer) compression through menuconfig and then run zRam scripts through init.d and what not..
Click to expand...
Click to collapse
This thread is a information share thread
It is not a thread in how to enable it in your kernel -_-
Thanks for the info
Sent from my GT-N8000 using xda premium
Zram = extravagant battery
so it makes wasteful battery zram?
excuse me if my English is poorly
gj man thanks for info very helpful :good:
Change the first line of the thread.
Hello dear developers.
I'd like to share with you effect of nearly 300 hours spent on trying to optimize Android and push it to the limits.
In general. You should be already experienced in setting up your buildbox, using git, building AOSP/CyanogenMod/OmniROM from source and cherry-picking things from review/gerrit. Solving git conflicts would also be nice. If you don't know how to build your own ROM from source, this is not a something you can apply to your ROM. Also, as you probably noticed, this is not a something you can apply to already prebuilt ROM (stocks), as these optimizations are applied during compilation, so only AOSP roms, self-compiled from source may use this masterpiece.
So, what is it about? As we know, Android contains a bunch of low-level C/C++ code, which is compiled and acts as a backend for our java's frontend and android apps. Unfortunately, Google didn't put their best at focusing on optimization, so as a result we're using the same old flags set back in 2006 for Android Donut or anything which existed back then. As you guess, in 2006 we didn't have as powerful devices as now, we had to sacrifice performance for smaller code size, to fit to our little devices and run well on very low amount of memory. However, this is no longer a case, and by using newest compilers and properly setting flags, we can achieve something great.
You probably may heard of some developers claiming using of "O3 Flags" in their ROMs. Well, while this may be true, they've applied only to low-level ARM code, mostly used during kernel compilation. Additionally it overwrites O2 flag, which is already fast, so as you may guess, this is more likely a placebo effect and disappears right after you change the kernel. Take a look at the most cherry-picked "O3 Flags commit". You see big "-Os" in "TARGET_thumb_CFLAGS"? This is what I'm talking about.
However, the commit I'm about to present you is not a placebo effect, as it applies flags to everything what is compiled, and mostly important - target THUMB, about 90% of an Android.
Now I'll tell you some facts. We have three interesting optimization levels. Os, O2, O3. O2 enables all optimizations that do not involve a space-speed tradeoff. Os is similar to O2, but it disables all flags that increase code size. It also performs further optimizations to reduce code size to the minimum. O3 enables all O2 optimizations and some extra optimizations that like to increase code size and may, or may not, increase performance. If you want to ask if there's something more like O4, there is - Ofast, however it breaks IEEE standard and doesn't work with Android, as i.e. sqlite3 is not compatible with Ofast's -ffast-math flag. So no go for us.
Now here comes the fun part. Android by default is compiled with O2 flag for target ARM (about 10% of Android, mostly low-level parts) and Os flag for target THUMB (about 90% of Android, nearly everything apart from low-level parfts). Some guys think that Os is better than O2 or O3 because smaller code size is faster due to better fitting in cpu cache. Unfortunately, I proven that it is a myth. Os was good back in 2006, as only with this flag Google was able to compile Dalvik and it's virtual machine while keeping good amount of free memory and space on eMMC cards. As or now, we have plenty of space, plenty of ram, plenty of CPU power and still good old Os flag for 90% of Android.
I've made countless tests to find out what is the most efficient in terms of GCC optimization, two selected tests I am about to present you right now.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
As you may noticed, I compiled whetstone.c benchmark using three different optimization flags - Os, O2 and O3. I set CPU to performance, maximum frequency, and I repeated each test additional two times, just to make sure that Android doesn't lie to me. Source code of this test is available here and you may download it, compile for our beloved Android and try yourself. As you can see O3 > O2 >> Os, Os performs about 2.5x times worse than O2, and about 3.0x times worse than O3.
But, of course. Android is not a freaking benchmark, it's operating system. We can't tell if things are getting better or worse according to a simple benchmark. I kept that in mind and provided community with JustArchi's Mysterious Builds for test. I gave both mysterious builds and didn't tell my users what is the mysterious change. Both builds have been compiled with the same toolchain, same version, same commits. The one and only mysterious change was the fact that every component compiled as target thumb (major portion of an android) has been optimized for speed (O3) in build #1 (experimental), and optimized for size (Os) in build #2 (normal behaviour). Check poll yourself, 9 votes on build 1 in terms of performance, and 1 vote on build 2. I decided that this and benchmark is enough to tell that O2/O3 for target thumb is something that we want.
Now it doesn't matter that match if you wish to use O2 or O3, but here is some comparison:
1. Kernel compiled with O2 has 4902 KB, with O3 4944 KB, so O3 is 42 KB bigger.
2. ROM compiled with O3 is 3 MB larger than O2 after zip compression. Fast overview: 97 binaries in /system/bin and 2 binaries in /system/xbin + 283 libraries in /system/lib and other files, about 400 files in total. 3 MB / 400 = 7,5 KB per file size increase.
3. It's unlikely that code working properly with O2 level might break on O3 level, most issues are on the Os <-> O2 part.
4. If it doesn't cause any issues, and speeds up a binary by a little bit, why not use it?
5. The only real reason to not use O3 is potential higher memory usage due to oversized binaries.
In general, I doubt that this extra chunk of code may cause any significant memory usage or slower performance. I suggest to use O3 if it doesn't cause any issues to you compared to O2, but older devices may use O2 purely for saving on code size, similar way Google did it back in 2006 using Os flag.
[SIZE="+1"]Now let's get down to business[/SIZE].
Here is a list of important improvements:
- Optimized for speed yet more all instructions - ARM and THUMB (-O3)
- Optimized for speed also parts which are compiled with Clang (-O3)
- Turned off all debugging code (lack of -g)
- Eliminated redundant loads that come after stores to the same memory location, both partial and full redundancies (-fgcse-las)
- Ran a store motion pass after global common subexpression elimination. This pass attempts to move stores out of loops (-fgcse-sm)
- Enabled the identity transformation for graphite. For every SCoP we generate the polyhedral representation and transform it back to gimple. We can then check the costs or benefits of the GIMPLE -> GRAPHITE -> GIMPLE transformation. Some minimal optimizations are also performed by the code generator ISL, like index splitting and dead code elimination in loops (-fgraphite -fgraphite-identity)
- Performed interprocedural pointer analysis and interprocedural modification and reference analysis (-fipa-pta)
- Performed induction variable optimizations (strength reduction, induction variable merging and induction variable elimination) on trees (-fivopts)
- Didn't keep the frame pointer in a register for functions that don't need one. This avoids the instructions to save, set up and restore frame pointers; it also makes an extra register available in many functions (-fomit-frame-pointer)
- Attempted to avoid false dependencies in scheduled code by making use of registers left over after register allocation. This optimization most benefits processors with lots of registers (-frename-registers)
- Tried to reduce the number of symbolic address calculations by using shared “anchor” symbols to address nearby objects. This transformation can help to reduce the number of GOT entries and GOT accesses on some targets (-fsection-anchors)
- Performed tail duplication to enlarge superblock size. This transformation simplifies the control flow of the function allowing other optimizations to do a better job (-ftracer)
- Performed loop invariant motion on trees. It also moved operands of conditions that are invariant out of the loop, so that we can use just trivial invariantness analysis in loop unswitching. The pass also includes store motion (-ftree-loop-im)
- Created a canonical counter for number of iterations in loops for which determining number of iterations requires complicated analysis. Later optimizations then may determine the number easily (-ftree-loop-ivcanon)
- Assumed that loop indices do not overflow, and that loops with nontrivial exit condition are not infinite. This enables a wider range of loop optimizations even if the loop optimizer itself cannot prove that these assumptions are valid (-funsafe-loop-optimizations)
- Moved branches with loop invariant conditions out of the loop (-funswitch-loops)
- Constructed webs as commonly used for register allocation purposes and assigned each web individual pseudo register. This allows the register allocation pass to operate on pseudos directly, but also strengthens several other optimization passes, such as CSE, loop optimizer and trivial dead code remover (-fweb)
- Sorted the common symbols by alignment in descending order. This is to prevent gaps between symbols due to alignment constraints (-Wl,--sort-common)
Click to expand...
Click to collapse
Sound cool, doesn't it? Head over to my ArchiDroid project and see yourself how people react after switching to my ROM. Take a look at just one small example, or another one . No bullsh*t guys, this is not a placebo.
However, please read my commit carefully before you decide to cherry-pick it. You must understand that Google's flags weren't touched since 7 years and nobody can assure you that they will work properly for your ROM and your device. You may experiment with them a bit to find out if they're not causing conflicts or other issues.
I can assure you that my ArchiDroid based on CM compiles fine with suggested steps written in the commit itself. Just don't forget to clean ccache (rm -rf /home/youruser/.ccache or rm -rf /root/.ccache) and make clean/clobber.
You can use, modify and share my commit anyway you want, just please keep proper credits in changelogs and in the repo itself. If you feel generous, you may also buy me a coke for massive amount of hours put into those experiments.
Now go ahead and show your users how things should be done .
Cherry-picking time!
Android "Lollipop" (5.1.1 & 5.0.2 tested)
JustArchi's ArchiDroid Optimizations V4.1 for CyanogenMod (latest)
A set of commits you may want to pick to fix O3-related issues:
external_bluetooth_bluedroid | hardware_qcom_display | libcore | frameworks_av #1 | frameworks_av #2
Older entries are provided for reference only. I suggest using only latest commit above.
Android "Lollipop" (5.1.1 & 5.0.2 tested)
JustArchi's ArchiDroid Optimizations V4 for CyanogenMod
Android "Kitkat" 4.4.4:
JustArchi's ArchiDroid Optimizations V3 for CyanogenMod
JustArchi's ArchiDroid Optimizations V3 for OmniROM
JustArchi's ArchiDroid Optimizations V2
JustArchi's ArchiDroid Optimizations V1
AFTER applying above commit and AFTER EVERY CHANGE regarding flags, ALWAYS make clean/clobber AND empty ccache (rm -rf ~/.ccache)
Q: How to properly change toolchains used in local manifest?
Open from your source rootdir .repo/local_manifests/roomservice.xml (or create one). Here is a sample manifest that replaces default 4.8 toolchain (both eabi and androideabi) with 4.8 SaberMod and 4.9 ArchiToolchain:
Code:
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<remove-project name="platform/prebuilts/gcc/linux-x86/arm/arm-eabi-4.8" />
<project name="ArchiDroid/Toolchain" path="prebuilts/gcc/linux-x86/arm/arm-eabi-4.8" remote="github" revision="architoolchain-5.2-arm-linux-gnueabihf" />
<remove-project name="platform/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.8" />
<project name="ArchiDroid/Toolchain" path="prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.8" remote="github" revision="uber-4.9-arm-linux-androideabi" />
</manifest>
This is only an example, you should use the toolchains that suit you best. My ArchiDroid/Toolchain github repo is a good start to test various different toolchains and decide which one you like the most, or which one causes the least problems for you. I do not suggest any other magic tricks to include custom toolchains, putting your selected one in proper path is enough, avoid magic android_build modifications.
[size=+1]Troubleshooting[/size]
Q: Compiler errror:
Code:
(...)/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.8/bin/../libexec/gcc/arm-linux-androideabi/4.8.x-sabermod/cc1: error while loading shared libraries: libcloog-isl.so.4: cannot open shared object file: No such file or directory
This error can be fixed by installing missing library. libcloog-isl.so.4 is provided by libcloog-isl4 package, so on debian-like OSes, you should be able to fix it with:
Code:
apt-get install libcloog-isl4
Q: Compiler errror:
Code:
(...)/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.8/bin/../libexec/gcc/arm-linux-androideabi/4.8.x-sabermod/cc1: error while loading shared libraries: libisl.so.13: cannot open shared object file: No such file or directory
This error is very similar to above, but considers other shared library. libisl.so.13 is provided by libisl13 package. Now the problem is that this package is in testing/sid, so we'll need to install it from there.
Add to your /etc/apt/sources.list following entries:
Code:
deb http://ftp.debian.org/debian testing main contrib non-free
deb-src http://ftp.debian.org/debian testing main contrib non-free
Then apt-get update && apt-get install libisl13.
Issues below are for older commits and should be used for reference only
Kitkat THUMB O2+ errors?
These are the most common issues.
* Change -O3 flag from TARGET_thumb_CFLAGS back to -Os, make clean/clobber, empty ccache and try again. This fixes most of the issues.
* RIL problems for for the Exynos 4210 family? Add -fno-tree-vectorize to TARGET_thumb_CFLAGS.
* Broken exFAT -> https://github.com/JustArchi/android_external_fuse/commit/78ebbc4404de260862dca5f0454bffccee650e0d
Errors caused by toolchain?
1. Try Google's GCC 4.8 if you used Linaro 4.8 or SaberMod 4.8
2. Fallback to Google's GCC 4.7 if above didn't help (change TARGET_GCC_VERSION back to 4.7)
Errors caused by GCC 4.8+?
* ART Fix (bootloop) -> https://github.com/JustArchi/android_art/commit/71a0ca3057cc3865bd8e41dcb94443998d028407
* Not booting kernel -> https://github.com/JustArchi/androi...mmit/9262707f4ea471acf40baa43ffe4bfb3cff64de9 and https://github.com/JustArchi/androi...mmit/41a70abcdad746d9415f3ee40f90528feb0c9bdd
Errors caused by GCC 4.9+?
* Graphical glitches in PlayStore -> https://github.com/JustArchi/android_external_webp/commit/36c6201fbb108d6b757f860e2cd57f3982191662
Errors caused by Linaro?
* error: unknown CPU architecture -> https://github.com/JustArchi/androi...mmit/5e5e158a7c147725beae1eeb6785174baacecb03 (Keep in mind that this is a sample fix for smdk4412 kernel, you may need to use similar solution in your own case. Also, this error happens only with Linaro toolchain, doesn't happen with Google's GCC)
Other errors?
* error: undefined reference to 'memmove' -> https://github.com/XperiaSTE/androi...mmit/679a4e571ef77f47892a785e852d8219c1e6807a
[size=+1]Credits[/size]
@IAmTheOneTheyCallNeo - For inspiration and first steps
@metalspring - For some nice commits
@sparksco - For SaberMod, some nice commits and support for the optimization idea
Bravo! I see flags in here that I am yet to try. Thanks for all your work and dedication to this effort.
Going to do a comparison this week against my "neofy" initiative
Thanks!
-Neo
Forum Moderator
This is amazing man!
Enviado desde mi Moto G mediante Tapatalk
Damn... Wish I had an international gs3. I'd be on SlimKat, find your thread and be like:
I'm gonna be 20 years old in about an hour... Should I ask?
Great work :highfive:
I saw you had already a custom 4.10 linaro. If I use it, do I need to change anything on your commit?
Thanks
Sent from my SAMSUNG-SGH-I747 using Tapatalk
_MarcoMarinho_ said:
Great work :highfive:
I saw you had already a custom 4.10 linaro. If I use it, do I need to change anything on your commit?
Click to expand...
Click to collapse
It won't boot, stl port has segfaults when using GCC 4.9+. If you want to use it, sure, change TARGET_GCC_VERSION and include proper toolchain.
JustArchi said:
It won't boot, stl port has segfaults when using GCC 4.9+. If you want to use it, sure, change TARGET_GCC_VERSION and include proper toolchain.
Click to expand...
Click to collapse
Is this the best toolchain [stability/performance] to compile a ROM?
https://github.com/JustArchi/Linaro/tree/4.8-androideabi
_MarcoMarinho_ said:
Is this the best toolchain [stability/performance] to compile a ROM?
https://github.com/JustArchi/Linaro/tree/4.8-androideabi
Click to expand...
Click to collapse
Yes.
JustArchi said:
Yes.
Click to expand...
Click to collapse
Just one last question. Do I need to configure envsetup.sh to the custom toolchain directory?
_MarcoMarinho_ said:
Just one last question. Do I need to configure envsetup.sh to the custom toolchain directory?
Click to expand...
Click to collapse
No, but you need to add Linaro entries to your local_manifest.
https://github.com/JustArchi/ArchiDroid/blob/2.X-EXPERIMENTAL/__dont_include/roomservice.xml#L3
Thank you @JustArchi
i make a build for Slimkat (nexus 5)
Everything is fine but just a noobish question,
I have to enable "TARGET_USE_O3=true" in BoardConfig.mk ?
micr0g said:
Thank you @JustArchi
i make a build for Slimkat (nexus 5)
Everything is fine but just a noobish question,
I have to enable "TARGET_USE_O3=true" in BoardConfig.mk ?
Click to expand...
Click to collapse
No, you shouldn't have any commits apart from mine. It already specifies O3 for everything.
Can I use a 4.7 toolchain with the ArchiDroid optimizations? I've tried compiling my kernel with different 4.8 toolchains before, and it's always resulted in boot loops.
Codename13 said:
Can I use a 4.7 toolchain with the ArchiDroid optimizations? I've tried compiling my kernel with different 4.8 toolchains before, and it's always resulted in boot loops.
Click to expand...
Click to collapse
Actually you can. Just change TARGET_GCC_VERSION back to 4.7.
Well done mate, I'll try contributing from my side to this project
great work archi !! also after porting a fully working 4.4.2 touchwiz to i9300 is it possible to make aosp for i9300 more stable now?
@JustArchi which flags could be used for kernel compiling? And where should I put it in in the Makefile? I don't compile the complete ROM because I have to low machine for that, but I am developing a custom kernel for my Xperia Z. BTW Fajnie trafić na innych Polaków
dragonnn said:
[B @JustArchi[/B] which flags could be used for kernel compiling? And where should I put it in in the Makefile? I don't compile the complete ROM because I have to low machine for that, but I am developing a custom kernel for my Xperia Z. BTW Fajnie trafić na innych Polaków
Click to expand...
Click to collapse
Ktoś tu mówił o Polakach?
Świetna robota @JustArchi :highfive:
dragonnn said:
@JustArchi which flags could be used for kernel compiling? And where should I put it in in the Makefile? I don't compile the complete ROM because I have to low machine for that, but I am developing a custom kernel for my Xperia Z. BTW Fajnie trafić na innych Polaków
Click to expand...
Click to collapse
If you have whole ROM tree then you cherry-pick this commit, lunch your target and make bootimage. This is enough.
If you have standalone kernel, take a look at main Makefile .