C#, VB, Ruby, F#, and Python on Android (via Mono and DLR) - G1 Android Development

This all works on stock/nonroot phones
I got Mono running on Android.
http://www.koushikdutta.com/2008/12/mono-on-android-success-at-last.html
Started working on Java/C# interop and found out that DLR works on Mono:
http://www.koushikdutta.com/2009/01/microsoft-dlr-and-mono-bring-python-and.html
As a result, you can write applications in Python and Ruby on Android too now.
Anyhow, if anyone else is interested in working on this project with me, please let me know! I've already gotten all the relevent source hosted at Google Code: http://code.google.com/p/androidmono. Basically the next bit of work involves implemeting a Java interop using the DLR.

Nifty.
As for Dalvik & JIT, I think dexopt already replaces some heavy usage calls with inline native code. Hopefully dalvik vm will get full JIT in the future?

This makes me very happy!!!
Thank you for your work!!

jashsu said:
Nifty.
As for Dalvik & JIT, I think dexopt already replaces some heavy usage calls with inline native code. Hopefully dalvik vm will get full JIT in the future?
Click to expand...
Click to collapse
It doesn't do any inline native replacements: dexopt optimizes the dex file. Which includes inline dex byte code replacements. There is no JIT at all, but Google said it would be something definitely on the horizon. Personally I think DEX is a pretty stupid move on Google's part; they could have just gone with CIL-- and write a Java compiler for that instead. The Mono JIT compiler works on many platforms; so V1 of Android could have been running JIT compiled native code with that route... which is an order of magnitude better in performance.
I'm going to be doing some performance comparisons of Mono vs Dalvik; Mono will obviously win, but it will be interesting to see the margin. I'll also experiment with binding Mono to the Android runtime to create Android applications in C#.
Optimization
Virtual machine interpreters typically perform certain optimizations the first time a piece of code is used. Constant pool references are replaced with pointers to internal data structures, operations that always succeed or always work a certain way are replaced with simpler forms. Some of these require information only available at runtime, others can be inferred statically when certain assumptions are made.
The Dalvik optimizer does the following:
For virtual method calls, replace the method index with a vtable index.
For instance field get/put, replace the field index with a byte offset. Also, merge the boolean / byte / char / short variants into a single 32-bit form (less code in the interpreter means more room in the CPU I-cache).
Replace a handful of high-volume calls, like String.length(), with "inline" replacements. This skips the usual method call overhead, directly switching from the interpreter to a native implementation.
Prune empty methods. The simplest example is Object.<init>, which does nothing, but must be called whenever any object is allocated. The instruction is replaced with a new version that acts as a no-op unless a debugger is attached.
Append pre-computed data. For example, the VM wants to have a hash table for lookups on class name. Instead of computing this when the DEX file is loaded, we can compute it now, saving heap space and computation time in every VM where the DEX is loaded.
All of the instruction modifications involve replacing the opcode with one not defined by the Dalvik specification. This allows us to freely mix optimized and unoptimized instructions. The set of optimized instructions, and their exact representation, is tied closely to the VM version.
Most of the optimizations are obvious "wins". The use of raw indices and offsets not only allows us to execute more quickly, we can also skip the initial symbolic resolution. Pre-computation eats up disk space, and so must be done in moderation.
There are a couple of potential sources of trouble with these optimizations. First, vtable indices and byte offsets are subject to change if the VM is updated. Second, if a superclass is in a different DEX, and that other DEX is updated, we need to ensure that our optimized indices and offsets are updated as well. A similar but more subtle problem emerges when user-defined class loaders are employed: the class we actually call may not be the one we expected to call.
These problems are addressed with dependency lists and some limitations on what can be optimized.
Click to expand...
Click to collapse

Koush said:
Personally I think DEX is a pretty stupid move on Google's part; they could have just gone with CIL-- and write a Java compiler for that instead. The Mono JIT compiler works on many platforms; so V1 of Android could have been running JIT compiled native code with that route... which is an order of magnitude better in performance.
I'm going to be doing some performance comparisons of Mono vs Dalvik; Mono will obviously win, but it will be interesting to see the margin. I'll also experiment with binding Mono to the Android runtime to create Android applications in C#.
Click to expand...
Click to collapse
Google had different objectives, they didn't go after maximum performance. Remember, that handsets have different constrains than desktops and laptops. So they went after minimizing RAM usage (byte code interpreter => maximum possible sharing of read-only memory pages among processes) and battery life. Performance had to be acceptable, not priority.
You would not be able to fit everything into RAM if you used Mono and you would get the patent problems with Net/Mono/etc as a bonus.

lu_tze said:
Google had different objectives, they didn't go after maximum performance. Remember, that handsets have different constrains than desktops and laptops. So they went after minimizing RAM usage (byte code interpreter => maximum possible sharing of read-only memory pages among processes) and battery life. Performance had to be acceptable, not priority.
You would not be able to fit everything into RAM if you used Mono and you would get the patent problems with Net/Mono/etc as a bonus.
Click to expand...
Click to collapse
.NET, C#, IL, et al are all ECMA standards. Mono is LGPL/GPL. There are no patent or licensing issues with it that is unfamiliar to the OHA. They reuse plenty of open source projects.
An interpreter is not power efficient OR performant, simply due to the fact is it doing the 10 times as much work to do the same thing as native code. In addition, Mono features an Ahead Of Time compiler (AOT) that would let you compile everything to native code before it even hits the phone (or just once, and cache it). Most of Android's power and memory optimizations currently comes from Google's application life cycle (activities can be killed and resumed at the system's whim) -- that has nothing to do with Dalvik. I'm not criticizing the API or the implementation, just the runtime.
They could have spent their time making the Mono runtime play nicely with the shared memory subsystem.
I'm rebuilding mono with a minimal configuration to check out the disk and memory footprint.

Koush said:
Not that most of you will care, but I got Mono running on Android.
Click to expand...
Click to collapse
I'm a noob.... how can I install this? Very good Job Kush!

pic.micro23 said:
I'm a noob.... how can I install this? Very good Job Kush!
Click to expand...
Click to collapse
I haven't released anything yet. I'm trying to figure out how to statically link all it's dependencies, minimize the size, bind to the Android runtime, convert DEX to CIL and then CIL to ARM, and all sorts of other goodness. Basically a lot of experimenting to do before anything is "released". It's just in proof of concept phase right now.

Dalvik sucks
http://www.koushikdutta.com/2009/01/dalvik-vs-mono.html

Koush said:
Dalvik sucks
http://www.koushikdutta.com/2009/01/dalvik-vs-mono.html
Click to expand...
Click to collapse
Lol. nice article Koush. It's surprising mono is that much faster

Koush said:
I haven't released anything yet. I'm trying to figure out how to statically link all it's dependencies, minimize the size, bind to the Android runtime, convert DEX to CIL and then CIL to ARM, and all sorts of other goodness. Basically a lot of experimenting to do before anything is "released". It's just in proof of concept phase right now.
Click to expand...
Click to collapse
Just read the dalvik vs mono article. It's certainly interesting work. I agree that by ignoring JIT they're certainly not going after the most beneficial optimizations. That said, I don't think it's something they've completely excluded from future implementation.
I think you should consider trying to get Mono added as an external project. If nothing, having unofficial support for a vm which supports C#/CIL could bring in a significant amount of developer interest from the WinMo dev community. The coretech team would be the folks to set up a new project.

I've now gotten mono working on all G1s. You don't need Debian OR root. Still a couple kinks to work out, but I have it on the market for anyone interested in playing with it. More information at the link below.
http://www.koushikdutta.com/2009/01/mono-for-android-now-available-on.html

I thought this was only a platform for development but its made my g1 much faster and reduced memory from steel and stock browsers as well. My market is still 12 mb and mono is about 11 mb. Is this normal?
Also everytime, I run mono, does it do the same thing it does the first time it was installed and opened?
Sorry, i know nothing about mono but i can tell its definitely optimizing the performance on the g1 though.
great work koushe,
hbguy

This awesome, I have been waiting to see how this all turned out after reading your first post about it a few days ago.

hbguy said:
I thought this was only a platform for development but its made my g1 much faster and reduced memory from steel and stock browsers as well. My market is still 12 mb and mono is about 11 mb. Is this normal?
Also everytime, I run mono, does it do the same thing it does the first time it was installed and opened?
Sorry, i know nothing about mono but i can tell its definitely optimizing the performance on the g1 though.
great work koushe,
hbguy
Click to expand...
Click to collapse
Koush would know more than I would, but installing mono shouldn't affect everything else on Android. It's not like everything is suddenly using mono instead of dalvik. I suspect you have a strong case of the placebo effect

hbguy said:
I thought this was only a platform for development but its made my g1 much faster and reduced memory from steel and stock browsers as well. My market is still 12 mb and mono is about 11 mb. Is this normal?
Also everytime, I run mono, does it do the same thing it does the first time it was installed and opened?
Sorry, i know nothing about mono but i can tell its definitely optimizing the performance on the g1 though.
great work koushe,
hbguy
Click to expand...
Click to collapse
Yeah, you're just imagining things I haven't even attempted like DEX->CIL yet.
The first APK of Mono is quite large though. I've updated it with a number of bug fixes and also am making it use eglib now. This trimmed the size by a few MB. Getting Mono to work with Bionic might not be possible... (that would trim off another 2MB).
Once again, the APK is just a developers release... something to play with and test.

I have been messing with mono on my G1.
Is it safe to say it will only work with command line apps?
I got my own hello world and a few other things running, but if I try and run any sort of gui I get errors.

Yes, only command line stuff will work. WinForms will not work on Android.
However, you should be able to get OpenGL ES working via PInvoke. I haven't tried it, but it should work just fine.

Koush said:
Yes, only command line stuff will work. WinForms will not work on Android.
Click to expand...
Click to collapse
That is what I thought, I think PInvoke is just a bit out of my skill set.
Thanks for the work none the less.

I got mono building in the Android build environment, and the Mono team accepted a patch to make it work on Android. There's also some changes external to Mono which can be found at the androidmono google code repository:
http://code.google.com/p/androidmono/

Related

KSM, does it really improves performance ?

Well sadly i don't have an answer for that question yet...
I'm trying to think of a way to put KSM to the test on my android device.
As far as i understand it is possible that the kernel actually causes high CPU usage trying to map and unmap memory pages over and over again.
This issue is known for linux and other virtual machines so it is possible that the Same effect will be on the android vm
Testings that i found are not relevant to android.
For example:
The result is a dramatic decrease in memory usage in virtualization environments. In a virtualization server, Red Hat found that thanks to KSM, KVM can run as many as 52 Windows XP VMs with 1 GB of RAM each on a server with just 16 GB of RAM. Because KSM works transparently to userspace apps, it can be adopted very easily, and provides huge memory savings for free to current production systems. It was originally developed for use with KVM, but it can be also used with any other virtualization system - or even in non virtualization workloads, for example applications that for some reason have several processes using lots of memory that could be shared.
Click to expand...
Click to collapse
http://kernelnewbies.org/Linux_2_6_32
What i would really want to know is what would happen if each of these VMs Would run a different application/game/audio/graphics software at the same time ? or what if the same vm will run many different apps ? and also to compare cpu usage with and without KSM
Guess i'll need a tool for that. something like 'iostat' but for memory diagnostic and another tool to see a per process CPU usage but 'top' is not good enough for that.
Any way, the best test should present clear results with precised data.
I'll keep looking for legit way to put it to the test.
If you can think of a way to test KSM with android, please let me know.
This is a technique that relates mostly to processes like virtualisation. For example, when you load 5 windows XP VMs, you'll have a good 10 - 20 services that are practically the same in memory in each VM. Instead of each service using 10mb (ie, 10mb x 5 = 50mb), you only need say 15 or 20mb using KSM. If you use different applications, it is very unlikely that anything would be saved FOR THAT APPLICATION. However, the main elements of a Windows XP System would still be there (drivers, explorer, firewall, logon, search and so on). Means little in one setup, but when you have several VMs it is shown to be a huge advantage. As we know a simple XP install can use 500mb of RAM actively, and this is fairly uniform across instals.
With android, i don't know if there are specific RAM savings to be had. Don't know enough about the inner workings and the sandbox android puts its apps in or how apps interact with system services. Sadly, i can't think of a good way to test it out either, but i'll be keeping an eye on this topic for someone (much) more knowledgeable to come along.
Harbb said:
Sadly, i can't think of a good way to test it out either, but i'll be keeping an eye on this topic for someone (much) more knowledgeable to come along.
Click to expand...
Click to collapse
Enter bedalus, stands there with a vacant expression on his face. Harbb looks disappointed.
kernels ; battery ; ROM ; gov/sched
That entire paragraph was dedicated to you bedalus, we both know that.
Lol
I hope someone can answer this though.
kernels ; battery ; ROM ; gov/sched
Wait for someone............
Sent from my Nexus S using xda premium
KSM does not improve performance on Android just like that - all enabling KSM does, is enable SUPPORT for the Feature but Applications would have to make use of the feature, which they don't.
You can easily verify this like that :
echo 1 > /sys/kernel/mm/ksm/run
<wait and/or run the Applications of your choice>
cat /sys/kernel/mm/ksm/pages_sharing
IF the above shows a value > 0 then you are making use of KSM else it's just available, without anyone using the feature.
Here's an interesting Article that gives a little more insight :
http://www.linux-kvm.com/content/using-ksm-kernel-samepage-merging-kvm
By the way, the same is true for ZCACHE. If you really want to make better use of your Memory (RAM) then using ZRAM as a Swapdevice does work (and may often make sense, too).
That all said : There appear to be efforts to make use of KSM http://forum.xda-developers.com/showthread.php?t=1464758 - so things may well change ...
any update on this...?

Understanding Android platform in a nutshell (in layman's terms)

System.out.println("Hello peoples");
==>The purpose of this guide is to help people who don't know anything about programming,aren't modders,guys with knowledge about technology.
==>Initially I loved computers and their capabilities and have a little knowledge on the C and Java languages and just how computers (think).
You have to understand that computing tries to emulate human behaviors on how to solve problems.This is where programming kicks in.
So what is Android?
===================
1)Android in general is an operating system that was meant to run on mobile devices e.g. cell phones
and has expanded to tablets and notebooks.Android is divided into three language groups:
a)the system's framework and apps are written in java.
b)the Android's core [kernel] is pure C -language.
c)the Android's libraries are written in C++.
App libraries are called by apps that need more functionality that java can't provide.These are usually plugins like
decoders e.g. ffmpeg libraries that are used in video decoding,flash-players e.t.c. Here,java native methods are used and the android NDK platform
is used to enable the java apps call these libraries during execution.(Please read further on Android SDK,NDK and Jni).
So what happens when you have just flashed that new Rom.
====================================================================
First you have to understand that cell phones have their own embedded firmware not including the recovery and you will see why.
a)the recovery partition can be flashed to install aftermarket recovery roms.So even if you mess your recovery,you can still install again.(This varies with different phones).
b)their is that system chip which you cannot touch and there is a reason for it.Think of this partition like a PC BIOS.If you mess with your bios
your system is toast aka Bricking the system.Since phones are classed as embedded systems,manufactures don't want people messing with it as it would result into
cryptic errors and system vulnerability.
when the on button is pressed something simple yet complicated happens:
1==>The kernel which is compressed to save space usually in a (zImage) format is deflated or expanded.Since your NAND chips are partitioned,the kernel is given a very
special chunk in which is protected from user data.
2==>The kernel finds which base address it needs to start executing e.g.(0x00200000) and mostly when you put a wrong kernel base
address your phone enters into a boot-loop because arguments are being passed to invalid locations.It is important to know where your kernel base address
starts.You can try looking it up in your kernel sources(try searching for the mach msm folder and into the makefile) or just goggle it up or use Xda-Kitchen.
3==>with the correct kernel base address,execution starts.Usually the (init.rc) file in the (ramdisk folder) gives symlinks and creates structures i.e. folders that will house
modules and sets the correct paths to android files and framework.
4==>After arguments have successfully been passed,the handles are now passed to Android.Basically Android checks for (init.d) scripts that are available
this is true to GingerBread and Cyanogenmod 7.After that audio checks are done followed by camera services and then arguments are passed to (core.jar),a
critical framework file which is huge around 50Mb in size in CyanogenMod 7.
5==>Here the DalvikVM (Dalvik virtual machine) is called and the process of optimizing your system files starts.The framework get optimized first as this contains critical
code needed to run your device.Then your flingers and renderers and called.This are engines used by android e.g. (pixelflnger) which is used in touchscreen.
Your system's sensors are usually started around this point (your compass,light sensors e.t.c)your phone apps e.g. contacts,calendar get optimized around this point
and depending on the number of apps your manufacturer installed,it will take some time.
6==>your network get's activated around this point and is probably the time the capacitive buttons and lights on your gadget light up.This is usually a good indication
that your system has loaded.When your bootanimation ends the handles are passed to activate your home launcher.There is usually still a lot of activity going
on to fully ready your system and this is why if you try to use it immediately especially on low-end phones,your system lags or get (not responding) errors.
common misconceptions
=====================
1==>There is are reason why goggle gave minimum specifications needed to run Android because this is a full operating system(OS) unlike the past relic
phones that ran on 50MHz processors.
2==>please don't complain that some of your Ram is not the same as specified on the phones catalog(e.g. you have 256Mb of Ram but in the task manager it indicates you have 179Mb).
There is a reason to it.There are core processes that eat a chunk of your Ram and are hidden so that you don't try silly stuff like trying to kill them in thinking that your are trying to get more memory.
Think of it like this,it's like trying to kill (services.exe or svchost.exe) processes in Windows.You will just be trying to get system hangs,bluescreens
or just a system crash.
Will Add more info later.Please feel free to correct anything i might have not addressed properly or share your views.
Happy modding.
Kernel topic
============
In simple terms,the kernel is the core of any operating system e.g. windows, Mac,or any Linux distro like Ubuntu and Android.Android kernels come mostly
in the form of a compressed kernel (zImage).The kernel is written in pure C-language,which gives it direct access to memory and registers unlike java
which has to pass through the java VM(virtual machine).This makes code written in C-language to be very fast and robust but also dangerous.
==>Many Androids in the eclair regime ran on kernel 2.6.29. This was not a complete kernel and as by my experience there was alot of code missing from it.
2.6.29
======
==>a lot of androids did not have adb functionality due to the framework being embedded to allow USB mounting to PC.This was a very rigid method
of doing it(also a very old method).
==>In the case of other devices, when viewing the internal task manager,many processes were viewed as (0.00) byte files.In essence you could not determine
the amount of RAM your app was taking.This is true in the case of huawei u8120.
==>In the case of shutting down the phone,even in some cases under load,it did so very fast.It killed threads and handles mercilessly. Many people misunderstood
this concept and thought their phones ran faster as compared to 2.6.32 kernel.
2.6.32.9
========
Here there was a ton of improvements as developers and modders became more aware of support and tweaks.
==>Fixed issues like the internal task managers.It was now possible to accurately know how much RAM your apps were taking.
==>Resolved how the android system shutdown.Instead of merciless killing of handles and threads still running,it killed them appropriately.This is why
when shutting down your system takes a while.You can use adb to see these events.
==>Usb mounting to pc was also made somewhat generic and flexible across many devices.
==>It did change some methods on how the camera is being accessed mostly in eclair,donout and earlier versions of android phones.This issue made
cameras not function.
==>wifi methods were also changed and developers and modders had to re-write their code to allow compatibility.
==>This was also the year of many froyo phones and overclocking was a common thing.
extra notes
===========
During eclair era,many developers were in such a hurry to produce new android phones every few months that they never even thought of long-term support
to newer versions of android that would come much later.This is where i have to praise Iphones for standardisation of its OS across all of its devices.
==>If you try to copy paste code from 2.6.32.9 and paste it into 2.6.29 and expect it to work then expect tons of errors when compiling.The (g_android)
module was properly coded into 2.6.32.9 so if you try enabling it an older kernel,expect errors.
==>If you are a modder and looking to buy a new phone,please see if it has a fan base of coders or support.Avoid buying phones where you will not get
any support from the manufacturers or other devs e.g Huawei Technologies.This company sucks alot.After they produce a phone they forget about you the customer
so you will have to handle the upgrades all by yourself.
Do not use this command (make -i)when compiling kernels.It will skip errors and you may smile after it's finished but the end is just a tragedy.your kernel is bound
not to function properly or even function at all.
Happy modding.

Porting Chromium to Windows RT

So, I've been at this for about 48 hours now (not continuously, but closer than you might think) and I figured I should take a break from modifying project files and puzzling over alignment issues to discuss the project, share some of the problems I've been having and ask if anybody can help, and so on.
The general idea is "Chromium build for Windows (on x86/x64) and build on ARM (for Linux), so there must be a way to build it for Windows on ARM". For the most part, that even looks like it's true. Probably at least 80% of 654 Visual Studio projects (no, that's not a joke) either build just fine with only minor amounts of work, or are things that we don't actually need (I'll try building the test suites... once everything else builds!!)
Areas that have given me problems (caution: some chance of brief rants ahead):
v8. Less than you might think, though. Setting the flags for Arm seems to have been enough.
Sandbox. There's a fair bit of thunking coded in assembly going on in the sandbox for x86. Not sure what's up with it (I don't know exactly how the Chromium sandbox works) but it'll have to come out or be replaced. The Linux (including ARM) sandbox seems to be SELinux-based, which doesn't help at all.
Native Client (NaCl). I think all the assembly is in test code, though, so I may just boldly #ifdef if all away.
libjpg-turbo (libjpg). Piles of carefully optimized assembly... for x86 and x64. There is a set of ARM assembly (for Linux) that Visual Studio won't compile, but something else might... or I may tweak until it works. Of course, I could also just accept the speed hit and use the version of libjpg implemented in nice, portable C.
Anything where the developers tried to use some SSE to speed things up. I may be able to replace it with NEON code, or I may just remove it and hope **** doesn't break. We'll see.
Inline assembly in general. Even when it's ARM assembly, Visual Studio / CL.exe don't want anything to do with it (__asm is apparently now an invalid keyword). I suspect I'll have to just pull the assembly out into stand-alone functions in their own files, then compile them to object files and link them back in later. If I can figure out the best way to do this (for example, I'll want to inline the asm functions) then it shouldn't impact performance. Seriously though, I kind of hate inline assembly. I can read assembly just fine, but I'm usually staring at it in a debugger or disassembly tool, not in the middle of source code I'm trying to build...
Everywhere that the current state of the CPU is cared about (exception and crash handlers, in particular) because the CONTEXT structure is, of course, CPU-specific. They're pretty easy to get past, though.
Low-level functions, like MemoryBarrier. Fortunately, it's implemented in ntdll.h... but as a macro, which breaks at least half the places it's referenced. Solution: where it breaks things, undefine the macro and just have it be an inline function that does what the macro did.
Running out of memory. Not even joking... well, OK, a little bit. I've got 32GB; I won't actually run out. Both Visual Studio and cl.exe do at times, though!. Task Manager says VS is currently using 1,928 MB, and before I restarted it, it broke 2.5GB private working set. Pretty good for a program that for some reason is still 32-bit...
Goddamn compiler flags. Seriously, every single project (I mentioned there are over 600, right?) has its LIBPATHs hardcoded to point at x86. Several projects have /D:_X86_ or similar (that's supposed to be set by the build tools, not the user, you idiots...) which plays merry hell with the #ifdef guards. Everything has /SAFESEH specified, not in the actual property table where the IDE could have removed it (unneeded and invlaid on ARM) but in the "extra stuff we'll pass on the build command line" field, which means every single .EXE/.DLL project must be modified or the linker will fail.
My current biggest goal is the JPG library; nobody wants to use a browser without it. After that, I'll tackle the sandbox, leaving NaCl for last... well, last before whatever else crops up.
Anyhow, thoughts/comments/advice are welcome... in the mean time, I'm going to go eat something (for the first time in ~22 hours) and then get some sleep.
Kudos for having the patience to look though this monster.
It's my understanding that NaCl is still a pretty niche thing at the moment. Is it possible to easily either disable it or completely hack it out, or do other more critical parts of Chromium now depend on it?
I don't think anything truly depends on it. I'll look in the VS dependency hierarchy and see how many things list it, and how awful it would be to remove them.. after I get the other stuff working. I may pass on the sandbox as well, if possible; it makes the security guy in me cringe something awful, but as they say, shipping is a feature..
great
Please make that happen !
Working on it! I've gotten over half of the projects to build and link, but some other stuff is adamantly refusing to work. I'm beginning to suspect I'll need to work from the other direction - rather than starting at the bottom and building all the dependencies, then combining them into browser components, and then eventually combining all the components into a complete piece of software, I may have to work from the top, removing components until the whole thing builds (at which point it will likely be useless, or all-but) and then seeing what I can add back in. I thought it would be faster to just assume everything can be made to work and only exclude something if it proved intractable, but at this point I've got a ton of very small components and almost no ability to combine them.
It would also help if VS was better at managing such truly immense tasks. For example, I have no simple graph of what all is and is not building, so I'm being forced to manually map that onto the VS dependency tree and see what is blocking a given component from building successfully, and how much is dependent upon it, one erroring project at a time (and there are a *lot* of erroring projects - my last attempt to build any substantial part of the system saw 50 of 400 projects fail).
GoodDayToDie said:
Working on it! I've gotten over half of the projects to build and link, but some other stuff is adamantly refusing to work. I'm beginning to suspect I'll need to work from the other direction - rather than starting at the bottom and building all the dependencies, then combining them into browser components, and then eventually combining all the components into a complete piece of software, I may have to work from the top, removing components until the whole thing builds (at which point it will likely be useless, or all-but) and then seeing what I can add back in. I thought it would be faster to just assume everything can be made to work and only exclude something if it proved intractable, but at this point I've got a ton of very small components and almost no ability to combine them.
It would also help if VS was better at managing such truly immense tasks. For example, I have no simple graph of what all is and is not building, so I'm being forced to manually map that onto the VS dependency tree and see what is blocking a given component from building successfully, and how much is dependent upon it, one erroring project at a time (and there are a *lot* of erroring projects - my last attempt to build any substantial part of the system saw 50 of 400 projects fail).
Click to expand...
Click to collapse
I thinkt tht is a mutch better taktic and mutch less frustrading.
I would love to see just a minimal version of it. After that all the small componens can follow.
50 of 400 is pretty good i think. Better then i expected
Bear in mind that the entire thing is 650 projects. If 50 fail at that level, many of the higher-level ones (dependent upon the lower-level) will fail too. I'll see what I can do. I may or may not be able to get v8 actually working (without it, the JS speed will be very bad, think IE8 at best) and I may have to fall back to the legacy libjpeg (which will cut JPEG render speeds by at least a factor of 2). Skia (2D drawing library used by Chrome) has a bunch of assembly optimizations that I need to get it to use the Arm version of instead. There's a couple of total hacks with the library files I've had to pull, which may or may not result in a working final build. We'll see.
GoodDayToDie said:
Bear in mind that the entire thing is 650 projects. If 50 fail at that level, many of the higher-level ones (dependent upon the lower-level) will fail too. I'll see what I can do. I may or may not be able to get v8 actually working (without it, the JS speed will be very bad, think IE8 at best) and I may have to fall back to the legacy libjpeg (which will cut JPEG render speeds by at least a factor of 2). Skia (2D drawing library used by Chrome) has a bunch of assembly optimizations that I need to get it to use the Arm version of instead. There's a couple of total hacks with the library files I've had to pull, which may or may not result in a working final build. We'll see.
Click to expand...
Click to collapse
the v8 engine ( used in nodejs ) has been ported to ARM :
I still can't link : htt p://ww w.it-wars.com/article305/compiler-node-js-pour-arm-v5
perhaps it will help you
Edit : oups, I just see that another great user of this forum made the port of nodejs to RT
Yep... but they did it without v8. That's not an encouraging result, but I feel like I'm so close...
Is there a GitHub repo so we can help or track the progress of the project ?
Sorry, not at present. There probably should be. The sheer size of the codebase is incredible (about 2.4GB) and having some way to share it practically would be good.
Also, I suspect this would go a lot faster if I don't have to repeat the work of others. I know that there's a working Webkit DLL out there, for example (though with several features, including the V8 JS engine, missing) and if I could get my hands on that it would drastically reduce the number of additional components I need to build. Currently I'm working on the sandbux, but expect that I will need to rip the whole thing out and basically have the browser run as though it was always passed the --no-sandbox parameter, at least for the first build. Too damn much assembly.
http://www.engadget.com/2013/01/22/google-chrome-native-client-arm-support/
This wouldn't have any impact on this project, would it?
Sent from my SCH-I535 using xda-developers app, complete with annoying signatures.
It probably means that NaCl on Windows RT will be possible in the future. At present, I'm cutting it out of the build - too much x86-specific stuff there to port it over myself, and it owuldn't be able to run x86-compiled NaCl code anyhow.
You might have bit off more than you could chew. It'd better if you put your current progress under version control on some public site so that other people may be able to help you.
It's a big and complex project. You are taking a lot of time, and understandably so. But just open up to other people and you could get this done faster.
Yeah, this is probably true. My life also got unexpectedly *busy* in the last week; a couple weeks ago I had many times as much free time as I do now, and so porting has slowed down.
My upload speed would take ages (literally probably at least a day of solid activity; it's embarassingly slow) to push the full source anywhere, but I may make the effort anyhow. I'll have to post it somewhere for GPL compliance in any case...
You may upload only the diff files, they'll probably be smaller then the whole distribution.
Not to pour cold water on you however, IE10 is already faster than the latest Chrome build in Windows Phone, Windows 8.
I don't see the point of this.
I have personally jumped from IE8 > FF > Chrome and finally back to IE10 over the years depends on its usability, smoothness, speed, etc
Speed isn't the only reason to use a browser. I actually prefer IE myself, but there are some things that other browsers do better than it (in the case of Chrome, parts of HTML5, the syncing across Google services, etc.) Also, Chrome gets updated far more often than IE; IE9 was equal with Chrome on speed at its release, and was far behind by the time IE10 came out.
The reason for this project, though, is a mixture of interest in what it takes, and a desire to benefit the community. Microsoft has deeped that only software which they have blessed may run on the Windows RT desktop. I disagree, and have chosen (among several other things) to port a web browser because I feel that it's important for users to have choice.
LastBattle said:
Not to pour cold water on you however, IE10 is already faster than the latest Chrome build in Windows Phone, Windows 8.
I don't see the point of this.
Click to expand...
Click to collapse
Some websites do not get along with the trident rendering engine. Some webdevs are so "Oh f*** IE I don't care" and block access to features just because it is IE. I have experienced this first hand on IE10 on my surface where it tells me to come back when I have a decent browser, only to not have the choice to do that.
This really isn't the webdevs fault either, for years IE was the scum of the internet, only recently has IE caught up to the rest of the browsers (and in my opinion exceeded some) but the years of IE being bad have left a lot of disjointed webdevs who won't even consider giving the latest IE a chance.

[app port] [1/23] dosbox with thumb2 dynrec

hi all,
I've updated DOSBOX's dynrec core to take advantage of the THUMB2 ARMv7 instructions that must be supported on Windows RT. Among other things, this includes being able to move immediates to registers without using a PC relative load, 32-bit instructions that allow access to all registers, etc. I also added a PLI instruction to preload a cacheblock into the instruction cache before it is executed, and PLD instructions to prefetch data for a minor speedup.
Testing on doom timedemo, I get 3721 realtics, which is a modest 5% speedup from the standard dynrec core. Probably not having directX is a significant bottleneck on video drawing, this can get fixed once I get a wrapper around directinput working. Hopefully these changes can also help out mamaich's x86 pe loader out a well.
-- old post below--
I managed to get dosbox compiled with mamaich's new updates (see below), so we now have dosbox with the dynamic recompiling core. I also included network support which seems to work.
The dynamic recompiling core does give better performance for newer apps. 3982 realtics vs 18088 realtics on doom timedemo, so about a 5x gain
The next step would be to get a better graphics core. There is an opengl wrapper for DX11 called gl2dx that I'm looking into.
I'm also looking at putting at optimizing dynrec for thumb2, for example, it seems that CBZ might be useful (except it only lets you branch forward!!!)
Hmm, I have to unfortunately note that DOS4GW doesn't seem to work in this build. I'm working on it!
In the mean time, enjoy commander keen 1 -6?...
no2chem said:
Hmm, I have to unfortunately note that DOS4GW doesn't seem to work in this build. I'm working on it!
In the mean time, enjoy commander keen 1 -6?...
Click to expand...
Click to collapse
So in the end, I think the dynamic recompiler has some issues.
I know that the VirtualAlloc function is correctly setting execute permissions on memory pages, but I don't know if FlushInstructionCache is actually flushing the instruction cache. It's also possible that the dynamic recompiler is emitting bad assembly, but inspecting the code... doesn't look like it!
no2chem said:
So in the end, I think the dynamic recompiler has some issues.
Click to expand...
Click to collapse
The same issue for me (but I'm using a bit obsolete DosBox sources, need to update). May be this is due to EABI changes (for example registers are not preserved), as DosBox dynamic core targets Linux.
Regarding DosBox. Look here - http://vogons.zetafleet.com/viewtopic.php?t=31787
This is the ARMv7 optimized dynamic core for DosBox by M-HT. It is not yet integrated into the official DosBox branch.
If you're concerned about FlushInstructionCache not being implemented correctly, let me know. As part of my work to port Chromium, I've learned how to do cache management in ARM a little closer to the metal (moving the relevant values to the coprocessor registers directly) and have both more control (for example, clearing the data cache lines as well) and more precise control over exactly what is done to the cache lines.
However, bear in mind that I haven't been able to properly test this yet, as too much of the Chromium core still doesn't want to compile for me... *sigh*.
GoodDayToDie said:
If you're concerned about FlushInstructionCache not being implemented correctly, let me know.
Click to expand...
Click to collapse
As far as I see - FlushInstructionCache works as expected (flushes icache). DosBox crashes due to some other reason, maybe due to the ARM-THUMB interworking problems (it is just a guess).
Good to know. It would be annoying if the official API were wrong...
I've found whu DosBox dynamic core crashes at random.
All existing dynamic cores use ARM code, even the thumb files are using the ARM prologue. Though our CPUs can run the ARM code, there is a bug (?) in RT that switches ARM code to be executed as a THUMB at random. For example you call an ARM procedure 100000 times, but at 100001 something happens and the code starts to execute as THUMB. Maybe a task switch occurs, or maybe some interrupt happens, and when OS returns to your code - it sets T bit in CPSR, so the code is interpreted as THUMB.
I was able to modify the prologue generator in risc_armv4le-thumb-iw.h so that it generates only THUMB code, and now everything started to work in my project. Attached. Code is ugly and unfinished and uses some dirty C hacks, I'll fix them later as it is deep night now. But you may look on the changes and implement similar things in a DosBox build.
mamaich said:
I've found whu DosBox dynamic core crashes at random.
All existing dynamic cores use ARM code, even the thumb files are using the ARM prologue. Though our CPUs can run the ARM code, there is a bug (?) in RT that switches ARM code to be executed as a THUMB at random. For example you call an ARM procedure 100000 times, but at 100001 something happens and the code starts to execute as THUMB. Maybe a task switch occurs, or maybe some interrupt happens, and when OS returns to your code - it sets T bit in CPSR, so the code is interpreted as THUMB.
I was able to modify the prologue generator in risc_armv4le-thumb-iw.h so that it generates only THUMB code, and now everything started to work in my project. Attached. Code is ugly and unfinished and uses some dirty C hacks, I'll fix them later as it is deep night now. But you may look on the changes and implement similar things in a DosBox build.
Click to expand...
Click to collapse
I'll take a look at getting that implemented in my build.
Edit: Haven't looked into it much, but DosBox crashes whenever I open any 32-bit stuff.
mamaich said:
I've found whu DosBox dynamic core crashes at random.
All existing dynamic cores use ARM code, even the thumb files are using the ARM prologue. Though our CPUs can run the ARM code, there is a bug (?) in RT that switches ARM code to be executed as a THUMB at random. For example you call an ARM procedure 100000 times, but at 100001 something happens and the code starts to execute as THUMB. Maybe a task switch occurs, or maybe some interrupt happens, and when OS returns to your code - it sets T bit in CPSR, so the code is interpreted as THUMB.
I was able to modify the prologue generator in risc_armv4le-thumb-iw.h so that it generates only THUMB code, and now everything started to work in my project. Attached. Code is ugly and unfinished and uses some dirty C hacks, I'll fix them later as it is deep night now. But you may look on the changes and implement similar things in a DosBox build.
Click to expand...
Click to collapse
thanks for figuring that out! its a bit odd though, don't you think - wouldn't normal ARM apps crash if this was the case? I'm sure most stuff is compiled with interworking, maybe ms interworking convention is different?
Anyway, I was thinking, doesn't homm3 use directx? Is your emulation layer also doing conversion of that code? It seems like it would be useful to have native wrappers for old versions of DX so they can take direct advantage of DX11 native. Alas, that's for another day.... Also, mfc would be nice too... (its there... MS just didn't provide the .lib, ordinals are [noname]'d out, and .pdb from debug server seems to be missing some exports....)
no2chem said:
hi all,
I managed to get dosbox compiled with mamaich's new updates (see below), so we now have dosbox with the dynamic recompiling core. I also included network support which seems to work.
The dynamic recompiling core does give better performance for newer apps.
The next step would be to get a better graphics core. There is an opengl wrapper for DX11 called gl2dx that I'm looking into.
I'm also looking at putting at optimizing dynrec for thumb2, for example, it seems that CBZ might be useful (except it only lets you branch forward!!!)
Click to expand...
Click to collapse
How big performance gains are you talking about?
bartekxyz said:
How big performance gains are you talking about?
Click to expand...
Click to collapse
Eh, 3982 realtics vs 18088 realtics on doom timedemo, so about a 5x gain
Nice! That should (eventually) allow quite a few legacy apps, if they're old enough and/or not to performance-sensitive (i.e. not realtime) to run acceptably.
no2chem said:
doesn't homm3 use directx? Is your emulation layer also doing conversion of that code?
Click to expand...
Click to collapse
No conversion, I just pass DX to the native OS libraries with some minimal wrappers. RT provides even DirectX 1.0 interfaces
The only problem is that RT prohibits non 32-bit color modes, though video driver supports them (at least 16 bit color is supported by tegra driver).
no2chem said:
I managed to get dosbox compiled with mamaich's new updates
Click to expand...
Click to collapse
I'm using ARM/Tumb dynamic code created by M-HT: http://vogons.zetafleet.com/viewtopic.php?t=31787
I've changed just one function.
I'm also looking at putting at optimizing dynrec for thumb2, for example, it seems that CBZ might be useful (except it only lets you branch forward!!!)
Click to expand...
Click to collapse
Please post your code changes somewhere, so that I could use them in my project too
don't you think - wouldn't normal ARM apps crash if this was the case? I'm sure most stuff is compiled with interworking, maybe ms interworking convention is different?
Click to expand...
Click to collapse
NTARM kernel is a THUM2-only kernel. There is no interworking it it. So it is possible that MS code intentionally sets the T bit to THUMB when returning from interrupt or a task-switch. Unfortunately this means that we can't use ARM code in our programs (I have not seen any ARM code in MS progs, only THUMB2). Anyway, THUMB2 code is very efficient now, its speed can be the same as of ARM code, and size is much smaller.
Generally, THUMB2 should outperform ARM, yes. Also, the state of Thumb vs. Arm is actually set in a kind of weird way: rather than manually setting a processor bit, it's controlled by the jump address. Since even THUMB2 instructions are always 16-bit (2-byte) aligned, the least significant bit of an instruction address is never 1. Therefore, that bit is used (on any jump-type instruction) as a flag to indicate whether the processor should use ARM or THUMB2 mode. In theory, this should mean that the OS doesn't care about the mode being used... but I've never written an interrupt handler for ARM (a couple other platforms, but not ARM) and there might be something weird going on with them.
GoodDayToDie said:
rather than manually setting a processor bit, it's controlled by the jump address
Click to expand...
Click to collapse
This is true for normal jumps. And that is why we have to set lowest bit to 1 manually on indirect branches inside autogenerated THUMB code. But for CPU itself the mode is kept in T bit in CPSR. And "something" sets it to 1 "sometimes". I have not done much testing, but I think that this should be an interrupt, as I've seen that bit changed in the middle of a generated ARM code.
Anyway the THUMB2 code is better then ARM for us - it is more compact, so more code may be kept in cache (both in CPU and DosBox dynamic recompilator's).
updated with a thumb2 dynrec.
Touch input for mouse
For anyone that wants to use their touchscreen for mouse input, navigate to their %AppData%\Local\DosBox folder and modify the dosbox-0.74.conf file so that "Autolock=true" is "Autolock=false"
It works pretty well with the touchscreen and an on-screen keyboard for gaming and such

[DEV][APP] waut.ch! - Calibration for Android - version 145

Utility for background calibration, curation and tuning of the device towards an intuitive interface.
Subsystems being battery, entropy, encryption, disk, cpu, memory, filesystem, ui, scheduler, and network, all safe and open source technology.
Presented in this educational gaming metric format with infinite feedback and an interestingly assymetric chance. scribble anywhere, check in some stress, or find the 8!
waut.ch! does one hope to receive from this?
Well, increasing degrees and amounts of a certain "Je ne sais quoi" or responsiveness from the user interface for a start. Better battery life perhaps. Better quality of life, maybe.
And waut.ch! can only perhaps be described as "A qualified quantification of the placebo effect"
waut.ch! might benefit from this?
In the Android device space:
Designers
Users
Manufacturers
Recyclers
Developers
Compilers
Support personnel
OEMs
The Friendly Neighborhood Nerd/Technician.
“Make the most of yourself....for waut.ch! is all there is of you.” - Ralph Waldo Emerson ( paraphrase )
All along the waut.ch! tower - Bob Dylan
waut.ch! - Sometimes used in some colloquium as "watch!", keen upon reducing the TDP of mobile devices to 1.0 watt!
ARM variants of Android only Donut 1.6+
Please uninstall either Seeder or CrossBreeder prior to using this.
Root recommended, else reactivity metric is interesting and introduces uniqueness into the entropy pool anyway. Metric may demonstrate a certain asymmetry that is expected from predictable human actions. Efforts have been made to remove time seed logic from haveged in order to improve upon encryption and system-wide performance and security.
Also numerous other subsystems require careful calibration to facilitate this process.
Rewritten code, subset of functionality for upstream project - CrossBreeder ( https://forum.xda-developers.com/showthread.php?t=2113150 )
Please feel free to view and analyze source and functionality and report bugs and discuss etc on the XDA forum:
( https://forum.xda-developers.com/android/apps-games/app-waut-ch-calibration-android-t3549967 )
Google Play store:
( https://play.google.com/store/apps/details?id=ch.waut )
Please visit: /data/data/ch.waut/files/bin on the device itself for partial shell source code and XDA Downloads section and Github for full source code.
Reboot at convenience liberally or sparingly to reseed the entropy pool or as is known in common parlance, for good luck!
Thanks.
Havged source code:
https://github.com/Openand-I/haveged
Adhoc Payment URL to support development efforts : https://paypal.me/openand/10
Frappe ( "free-paid" ) same-version to support development efforts : http://waut.ch
Custom haveged source code as used in this piece of software: https://github.com/Openand-I/haveged
=====
Version Name: 59a6333e-9ed9-43f8-8dad-51ed46c17e28
cb.sh: cache pressure - 500
cb_io.sh: read_ahead - 0
cb_io.sh: nr_requests - 0
$ md5sum *.apk
661c30b02b2321300624af98feaa5bad *145-waut.ch.apk
661c30b02b2321300624af98feaa5bad *oi.apk
$ sha256sum *.apk
6d23b8da87dc5516583a55a3203c9f5068ea8fe8765ece489080ef663c8aee15 *145-waut.ch.apk
6d23b8da87dc5516583a55a3203c9f5068ea8fe8765ece489080ef663c8aee15 *oi.apk
https://github.com/Openand-I/haveged
https://github.com/openand-inc/waut.ch
https://forum.xda-developers.com/devdb/project/?id=19218#downloads
https://forum.xda-developers.com/android/general/app-waut-ch-calibration-android-version-t3858365
https://github.com/openand-inc/waut.ch/raw/cb0c60025f86a4fdc4778506e97ee80eb1c00b45/oi.apk
https://github.com/openand-inc/waut.ch/raw/master/145-waut.ch.apk
-------------------------
Recommended:
- ntp: automatic system time update from internet is enabled.
please check the clock and fiddle around with the timezone settings in case of any issues. one may need to set the timezone manually.
then simply run the app to initiate a time sync
the network time sync happens at around 3am. so the time to check is in the morning.
- Please disable mount namespace separation in the superuser app to take advantage of the mount optimisations.
- Reboot once and occasionally to reseed the entropy pool. It's good luck!
- Do ensure that the waut.ch service has started upon reboot. Just run if it doesn't start it automatically!
Note: Please note that the haveged binary in the APK is a static binary and works on both PIE and non-PIE environments. It is also UPX compressed. UPX for Android didn't compile. So UPX for linux was used to compress the executable file. It is an elegant solution as both on disk and in memory space(?) is reduced by 70% per executable. One is welcome to decompress the file using 'upx -d'.
Full source code is provided on Github and build scripts are attached here and on Github.
There is no license required to both install the app or distribute it, both within the developer ROM community or in commercial form. Adhoc payment URL to support development - https://www.paypal.me/openand/10
Again do note that the license to use the APP and source code is free worldwide and irrevocable in full or partial form. All other open source components simply inherit their license. But under no circumstances is any use thereof legally binding or relevant.
--------
Utility for background calibration, curation and tuning of the device towards an intuitive interface.
Subsystems being battery, entropy, encryption, disk, cpu, memory, filesystem, ui, scheduler, and network, all safe and open source technology.
Presented in this metric format with infinite feedback and an interestingly assymetric chance. scribble anywhere, check in some stress, or get lucky for that matter!
-----
ARM variants of Android only Donut 1.6+ ( should even be compatible with the latest ARM Android 9+ )
Please uninstall either Seeder or CrossBreeder prior to using this. And other "mods" or "tweaks".
Root recommended, else reactivity metric is interesting and introduces uniqueness into the entropy pool anyway. Metric may demonstrate a certain asymmetry that is expected from predictable human actions. Efforts have been made to remove time seed logic from haveged in order to improve upon encryption and system-wide performance and security.
Also numerous other subsystems require careful calibration to facilitate this process.
Rewritten code, subset of functionality for upstream project - CrossBreeder ( https://forum.xda-developers.com/showthread.php?t=2113150 )
Please feel free to view and analyze source and functionality and report bugs and discuss etc on the XDA forum:
( https://forum.xda-developers.com/android/apps-games/app-waut-ch-calibration-android-version-t3858365 )
Google Play store:
( https://play.google.com/store/apps/details?id=ch.waut )
Please visit: /data/data/ch.waut/files/bin on the device itself for partial shell source code and XDA Downloads section and Github for full source code.
The app will amongst other maintenance tasks tune sqlite databases regularly and reseed the entropy pool or as is known in common parlance, for good luck!
Thanks.
Payment URL: https://paypal.me/openand/10
XDAevDB Information
waut.ch!, App for all devices (see above for details)
Contributors
idcrisis
Source Code:
[url]https://github.com/Openand-I/haveged[/URL]
[url]https://github.com/openand-inc/waut.ch[/URL]
[url]https://forum.xda-developers.com/devdb/project/?id=19218#downloads[/URL]
Previous Version Information - 144
305bd30f-0c8a-40d8-baf5-330c68f62d51
Status: Stable
Created 2017-01-01
Last Updated 2020-08-18
$ md5sum *.apk
8ea8e8c132a584767a12e394f7975654 *144-waut.ch.apk
8ea8e8c132a584767a12e394f7975654 *oi.apk
$ sha256sum *.apk
4925066a106c83b18ac6e563f03331c56b72777e66973db591c9776d706595e3 *144-waut.ch.apk
4925066a106c83b18ac6e563f03331c56b72777e66973db591c9776d706595e3 *oi.apk
https://github.com/Openand-I/haveged
https://github.com/openand-inc/waut.ch
https://github.com/openand-inc/waut.ch/raw/master/144-waut.ch.apk
https://github.com/openand-inc/waut.ch/raw/f699d3763507ec1f91d82b9ce25c53036b460a9e/oi.apk
-----
Version notes:
haveged: static non upx binary used
cb.sh: lock fixes
Recommended:
- ntp: automatic system time update from internet is enabled.
please check the clock and fiddle around with the timezone settings in case of any issues. one may need to set the timezone manually.
then simply run the app to initiate a time sync
the network time sync happens at around 3am. so the time to check is in the morning.
- Please disable mount namespace separation in the superuser app to take advantage of the mount optimisations.
- Reboot once and occasionally to reseed the entropy pool. It's good luck!
- Do ensure that the waut.ch service has started upon reboot. Just run if it doesn't start it automatically!
Please support development, simply use https://paypal.me/openand/10 or the payment URL.
Or you can simply buy the "frappe" ( free-paid ) version of the app: http://waut.ch
Recommended:
- ntp: automatic system time update from internet is enabled.
please check the clock and fiddle around with the timezone settings in case of any issues. one may need to set the timezone manually.
the network time sync happens at around 3am. so the time to check is in the morning.
- Please disable mount namespace separation in the superuser app to take advantage of the mount optimisations.
- Reboot once and occasionally to reseed the entropy pool. It's good luck!
- Do ensure that the waut.ch service has started upon reboot. Just run if it doesn't start it automatically!
Issues:
- superuser - Please disable mount namespace separation in the superuser app ( for optional but recommended mount options ). Also please revisit the app entry inside the superuser app to ensure the waut.ch service can run on boot unattended. One can see the logs on another day to ensure that the scheduler ran correctly in the night.
- Non root users - User Interface can help in clearing the random device. Please try and obtain root to avail of most features
- Some Samsung users - One is also requested to raise a ticket with Samsung who may be running old PE detection rules that flag any compressed EXE.
- x64 users - Reports are that the binaries run on 64 bit as they are static! Please compile one's variant of the binary if required. Entropy generations removes CPU jitter and hence runs cooler and more secure.
- Intel users - User Interface can help in clearing the random device. Please compile one's variant of the binary if required. Entropy generations removes CPU jitter and hence runs cooler and more secure.
- Maintenance scheduler VACCUUMS and INDEXES "ALL" SQLITE databases. Some folks may not like that. But given that they're no WAL mode anyway, it's a bottleneck worth removing safely.
- There is a concerned effort to state that 32-bit ARM Android Go/One < 1 GB RAM devices are all that's required for long term functioning. Higher no issues.
- Please try and use a heap size of 96MB. Attempts have been made to set heap size dynamic but ideally this should be done in the build.
- Please clear cache or factory reset upon issues to gain at least another year of MTTR ( Mean Time To Recovery ) for each device.
Thank you!
License
License concerns:
haveged - inherited - https://github.com/Openand-I/haveged
busybox - inherited - https://github.com/openand-inc/busybox
- Busybox simple extract, possibly edit the .config file in 'vi' and type 'make'
The requisite tools are installed using:
apt-get install gcc-arm-linux-gnueabi
apt-get install libncurses5-dev
apt-get install gawk
The following is a step in another direction as the 'make' command works perfectly after extraction, but this is provided for posterity:
wget http://busybox.net/downloads/busybox-1.24.1.tar.bz2
tar -xjf busybox-1.24.1.tar.bz2
cd busybox-1.24.1/
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- defconfig
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- menuconfig
At the menu, you can configure BusyBox options. Once configured, you can build BusyBox:
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-
sqlite3 - inherited - https://github.com/openand-inc/sqlite
waut.ch - 'none', non legally binding, and non legally relevant on a worldwide scale and irrevocable ( derived works are allowed to add their own licenses as long as the import ( meaning ) of the phrase "non legally binding and non legally relevant" is implied throughout ( not required to include text at all )) and does not reflect upon future updates of this software in any manner adversely.
He is back! Welcome back! Looking forward to this great new project! The only thing I might be missing is the dnsmasq filtering, which in my opinion if by far better solution than any kid of firewall/blocker
Sent from my Galaxy Tab 2 3G using Tapatalk
qWantUS said:
He is back! Welcome back! Looking forward to this great new project! The only thing I might be missing is the dnsmasq filtering, which in my opinion if by far better solution than any kid of firewall/blocker
Sent from my Galaxy Tab 2 3G using Tapatalk
Click to expand...
Click to collapse
Mate!
There are some serious issues with DNSMASQ.
- It crashes on wrong syntax of any option. Cannot do for a server.
- Command line syntax changes between original branch, Android and Cyanogen MOD. Not merged.
- Android fork far behind main branch
- Consumes port 53 on server run without sharing interfaces. This is on the Android branch. Main branch has this resolved.
- CPU loop for most Android DNS versions that show up only upon server run, rather than the tethering run. Possibly hijacked open source branch.
- Tethering modifications to Android branch ( command line addons ) not required any more as the main branch has incorporated methods to dynamically change IP addresses on the fly.
- It is probably encumbered by coding standards and export issues with only one developer. It is emblematic of most such over-reused projects.
- Hash table in memory, therefore future block list will use up RAM. Some others have disk based caching.
- Static blocklists are fairly not scalable for 10 years hence. Needs wildcard blocklists.
- DNS cache poisoning/overloading ( leading to denial of service) possible by any process/app.
It is therefore recommended to use either of:
pdnsd
unbound
djbdns
and standalone dhcp clients and servers from the Entware repository. And also these are interesting:
DANE
DNSSEC over TLS
BTW, all Android is encumbered by simple denial of service attack simply by reading from /dev/random by any app.
In order to try and package this into a simplish product, deleting /dev/random was not implemented.
idcrisis said:
Mate!
There are some serious issues with DNSMASQ.
- It crashes on wrong syntax of any option. Cannot do for a server.
- Command line syntax changes between original branch, Android and Cyanogen MOD. Not merged.
- Android fork far behind main branch
- Consumes port 53 on server run without sharing interfaces. This is on the Android branch. Main branch has this resolved.
- CPU loop for most Android DNS versions that show up only upon server run, rather than the tethering run. Possibly hijacked open source branch.
- Tethering modifications to Android branch ( command line addons ) not required any more as the main branch has incorporated methods to dynamically change IP addresses on the fly.
- It is probably encumbered by coding standards and export issues with only one developer. It is emblematic of most such over-reused projects.
- Hash table in memory, therefore future block list will use up RAM. Some others have disk based caching.
- Static blocklists are fairly not scalable for 10 years hence. Needs wildcard blocklists.
- DNS cache poisoning/overloading ( leading to denial of service) possible by any process/app.
It is therefore recommended to use either of:
pdnsd
unbound
djbdns
and standalone dhcp clients and servers from the Entware repository. And also these are interesting:
DANE
DNSSEC over TLS
BTW, all Android is encumbered by simple denial of service attack simply by reading from /dev/random by any app.
In order to try and package this into a simplish product, deleting /dev/random was not implemented.
Click to expand...
Click to collapse
Very comprehensive explanation indeed! I am no expert in Linux, only an average user [emoji6]
I was not aware of so many obstacles using dnasmasq, but aware of it's unreliability while using it for blocking stuff in pfsense. But I must say, I had similar experience with unbound, however, probably due to the lack of linux knowledge.
The way I understand is that host blocking is no longer viable due to scales of blocking required today. But sadly there is no alternative for that on android. I have been trying another tool, called sharkmasq but developer sadly abandoned it, while it seems very unfinished and unreliable. I also understand the reasons behind leaving netfiltering out of your development, just, in desperation, I search for the hope [emoji3]
P. S.
Thank you very much for your reply and for your time putting it together. Nevertheless I will look forward to further developments of this new exciting project!
Sent from my LG-D855 using Tapatalk
Uhm, my device seems more responsive even when I use a very low CPU frequency, but is it normal that the app doesn't ask for root permission even if I'm fully rooted? I'm using a Nexus 5 running Nougat. Thank you
Inviato dal mio Nexus 5 con Tapatalk 2
Hi, yes, part of the design criteria. Without root, the metric is interesting. With root, it will automatically start the background processes including the daily cleanups.
Best way to know that the background processes are running is to install Seeder ( but whatever you do don't start it! . Entropy should show around 4096.
Still in the testing phase, does it worth excluding the app from "android optimisation list"? I also use "power nap", not sure will that cause any side effects...
Sent from my LG-D855 using Tapatalk
Let us know If Seeder has RNGD off and still reports ~ 4096 entropy then all is hunky dory.
The new update asked me for root permission, all right here now
Have been busy lately with life, but had occasionally played with phone. So my findings so far: I use this app on my old galaxy tab 2 3g on slim6. Seeder shows entropy being filled in no time at all. Can't really say much about performance, but it seems that it behaves a little smoother. I also use it on my daily LG G3 on stock MM rom with custom kernel, but seeder is reporting very low entropy, if seeded is left on for 1 minute, I can see that entropy never goes above 900? Goes up slowly but then gets used, and then very slowly goes up again and gets used.
Sent from my LG-D855 using Tapatalk
Reboot once after first install kicks in the animation changes. And good luck
New version uploaded with haveged updates. Build scripts also added.
Feedback solicited about the compiler flags etc. Basically the idea is that the ARM v5 binaries should work for all past and future versions of 32-bit Android without modification.
UPX binary could only be compiled for x86 linux.
To answer question, second device, not running, may need to check su logs
Also backend binaries are 32 bit, may not work on 64 bit. GUI's ok.
I am very confused about the frontend of this app that looks like some number game that I really don't understand . Also, I don't understand the description very well. Finding the 8??? Reboot when animation kicks in? What animation?? Nothing seems to happen. I just see a still of a beach with a red number
But something DID happen though, so I am in business. I will keep you curious for a moment, while firstly giving you a good reason for a hard laugh:
I am trying to revive an LG L3 E400! Yep, that's right . Just for fun and educion. In fact I already succeeded quite a bit. I flashed JellyCast V7rev1, used some stuff from V6 SuperCharger (not entropy thingie coz I already intended to use CrossBreeder for that) and 3 scripts from Fly-On Mod. I came here via the CrossBreeder thread (of course). I have already tried CrossBreeder 7.2.13 (I followed advise from @f3tus here, he advised to use that older version). Entropy went up (so it worked) but I did not experience much performance improvement. I did experience more "not responding" issues though. So I reverted to the pre-CrossBreeder backup. And then came your app.....
So that's a very short summary of the many, many, many hours I spent last 2 weeks on this ancient minimalistic but very adorable phone. Now... back to this Waut.ch! app. Presuming that it had done something, I checked the entropy level after first reboot: still a jumpy number between 150 and 200 or so. Despite the fact that the service shows up as running. Then, a lot more playing with the "game" without understanding what I was doing. Just hoping for some wonder. Some message.... just...... something! But nothing. Just a beach with a red number. And a clock. And a questionmark. Or exclamation mark. Pffffff.. So.... let's reboot again. Then eat a sandwich and drink (more) coffee. Then check entropy again. And there it is! Steady at 4089! At first I thought his app was failing because the number didn't change anymore. Just steady. With the old CrossBreeder 7.2.13 I got 4096 a lot but it was a very jumpy number, going even below 1000 a lot. Well, some time passed while writing this post so let's check again: Wow, right now it says 4091 (99%)! One hour later: 4092! (It had spontaneous reboots inbetween).
I am testing it now for a few hours and have these problems:
"Not responding" issues (same as old CrossBreeder).
Spontaneous reboots (3 already). One of them failing to boot at all (stuck at JellyCast logo).
Two times, after a reboot, GPS was disabled. Not consistently though.
No noticable performance gain.
By the way, I am testing like real world usage. Just opening, using and closing apps. I use the same apps all the time so that gives a very good feeling about performance. More meaningful than benchmarks to me. My goal is to make this phone suitable for normal daily usage.
Your app is using a combination of serveral mods, right? I would like to be able to enable / disable each one separately. How can I do that? For example, I have a gut feeling that my "not responding" issues (and reboots?) do not come from the entropy mod but perhaps from governor tweaks or something else. Any chance that you will add enable buttons in the app? Or supply seperate scripts?
Update: I DO have performance gain! Not consitently though. Apps sometimes starting faster than ever. Also still the aforementioned quirks. Last hours no more auto-reboots. Entropy reaching 4096 now, just a little bit wobbly. There seems to be a lot of potential.
The stutter was an issue with the last version.
Do try the latest one and let us know.
Also better if you install the Play store also so the updates are automatic.
Version name is changed GUID if comparing. Better than comparing version numbers.
GUI is non functional, just a visual representation of haptic feedback and actual randomness spread.
It's like drawing points in the whites of a poached egg. If you can see a pattern, you'll be rich in the stock market
Today I removed the app. The biggest problem is random reboots for me. As said, it has potential because sometimes apps started faster than ever on my ancient LG L3. But not consistently. So maybe I 'll try again later.
Sorry for my first post, probably tl;dr. I will repeat my most important question: can you provide us with separate scripts? I mean: I would like to test the entropy thing without the other tweaks as you described in the OP.
Dude!
You're probably using an old version. We'll never know. Will we? Will we now?
There's a small trick that borders on superstition. The famous random device block gremlin tends to disappear if you run the GUI in scribble mode. Basically what may be happening is the block is hit and then the stupid kernel makes whatever decision it makes. Usually at the cost of usability in favour of some stupid religious Linux fervour. Someone should report that to LINUX.
Or another entropy generator, haveged or RNGD or Seeder may be running. Which is a big no! Best to run it on stock ROMS or full custom ROMs ( please ask the developer if they're running any variant of above and to recommend coexistence or better variant of any subcomponents or piece of code. ). In other words, open source!
Also, the separate scripts are in the /bin directory on the phone itself. May require bash and ADB skills.
And full source on XDA and GitHub.
I'll construe this as a possible request to also post the shell scripts from GitHub into the XDA project as separate attachments. Which I can do. In due course
Cheers.
Well, I took it from Play Store. Is that an old version?

Categories

Resources