Related
I just noticed this again and it worries me alot! Devs here are usually posting their changes as being different than others, more even about compiler optimizations and yet most of you make huge mistakes.
Personally I've been running Gentoo linux for about ten years now and what I learned from running different setups with space limitations is that one should NEVER use -Os and -O3 together as this will not only take much more time to compile it also GROWS executables.
-Os is basically the same as -O3, but it adds some flags that won't benefit executable size vs memory usage.
In general executables compiled with only -Os are nearly as optimized as -O3/2 and resulting in smaller executable. While compiled with -O3 uses more memory allocation for extra speed. While using both together will result in larger executables, more memory usage AND much longer compile times vs just -O2.
So devs can you all please remove -Os from your "optimizations" and leave just -O2? Bragging about "more compiler optimizations" really fails if you fail to understand why the -Ox optimizations levels have been added in the first place... Which is to NOT use them together!
Also -O3 might introduce more IO operations, which might have a negative effect on our snappies.
The choice: Summarised
-O2 is the default optimization level. It is ideal for desktop systems because of small binary size which results in faster load from HDD, lower RAM usage and less cache misses on modern CPUs. Servers should also use -O2 since it is considered to be the highest reliable optimization level.
-O3 only degrades application performance as it produces huge binaries which use high amount of HDD, RAM and CPU cache space. Only specific applications such as python and sqlite gain improved performance. This optimization level greatly increases compilation time.
-Os should be used on systems with limited CPU cache (old CPUs, Atom ..). Large executables such as firefox may become more responsive using -Os.
Click to expand...
Click to collapse
From: http://en.gentoo-wiki.com/wiki/CFLAGS
Quote from the wiki articale you provided:
"Note that only one of the above flags may be chosen. If you choose more than one, the last one specified will have the effect."
tincmulc said:
Quote from the wiki articale you provided:
"Note that only one of the above flags may be chosen. If you choose more than one, the last one specified will have the effect."
Click to expand...
Click to collapse
My point is that in general people here seem to use "-Os -O3", which is dumb really as it had no benefits. -O2 should be better performance wise on our phones.
great post, thank you man!
Now silently pray for our devs to change their -O3 -Os to -O2.
Or I might need to do my own branch of "optimized" nightlies.
For example what you declare with -march=A -mcpu=B will intact be -march=A -mtune=B.
edit: I think the gcc compiler even throws a warning about l2p etc.
Wouldn't the best choice be just -Os? I would have thought that optimising for systems with limited cache would be appropriate for portable devices.
Pipe Merchant said:
Wouldn't the best choice be just -Os? I would have thought that optimising for systems with limited cache would be appropriate for portable devices.
Click to expand...
Click to collapse
I highly doubt executable sizes on our phones are big enough to benefit from -Os.
In the case of my router at home -Os doesn't give any benefit at all, in fact packet inspection is slightly slower vs -O2.
this guide is under construction still and not quite finished - check back soo
Hi guys, I'd like to share some knowledge and as I'm going to start a blog doing so, I felt this would be as good as anywhere to start.
This technique will allow you to create disk images of your chosen size on as many storage devices as you want and that your device supports. There are a few advantages and reasons someone would do this. Storage flexibility, performance, data redundancy, easy backups. The techniques outlined here will likely be obvious to any linux pros, but perhaps not thought about in a mobile context.
While the current instructions and application of this technique is relatively complex, part of my goal in starting this is to gauge interest on whether it would be worth making a tool to do this. Perhaps there is one already, or another how-to. I didn't find any in my research in setting this up, albeit as a linux guru I didn't have to do much.
I will also include an APPENDIX including instructions on how to automatically compresss a filesystems contents (READ&WRITE) to save space with fusecompress. I may also add one for an encrypted filesystem if someone requests it.
DIFFICULTY: Hard for noobs, Easy for Pros.
REQUIREMENTS;
- Android device (rooted) preferably with multiple storage (sub-)systems (one will do if all you want is imaging or compression)
- Currently LINUX MUST BE INSTALLED ON YOUR DEVICE AND HAVE A BIND MECHANISM TO ACCESS THE MOUNT POINT IN ANDROID. As mentioned above, if the interest is there I will port the utils and make an app to manage it. I recommend Galoula Linux Installer w/ Debian stable. This tutorial is based off that setup but will likely work w/ Ubuntu and other Linux methods with few changes. Once all the tools are ported/compiled for Android, a Linux installation will not be required. You could very well compile it yourself..
- Working apt-get package manager and an internet connection.
Linux packages/tools needed & requirements/dependencies:
- lvm2, fuse, fuse-utils, libfuse, gvfs-fuse, libfuse2, libfuse-devel*, gcc/compiling tools*, fusecompress*+, busybox binaries or equivilent, fdisk, dd, mke2fs (or equivilent for your chosen filesystem), and iozone for benchmarking if you want.
- You will need a kernel & linux that supports fuse/loopback devices (most do!) or you can load the loopdev module. You will need at least 1 or 2 or more free loop devices depending on your configuration. There is a kernel parameter to change the maximum. **TODO: Add instructions for that **
- * and +: You will only need libfuse-devel and gcc for compiling fusecompress ONLY IF YOU NEED/WANT COMPRESSION. (If your chosen distro has a working fusecompress package you can use that, however at the time of writing DEBIAN-STABLE has not included an updated package for it. Squeeze had it, wheezy has some patches and updates for it but is not released as a package nor can I get it to work stabily after I spent hours convincing it to compile. I should note that this package is based off the CPP aka 1.9x branch (v2.6) and isn't completely stable compared to the older (slightly slower) but stable 0.9x branch. The APPENDIX instructions fot fusecompress ARE FOR THE STABLE/0.9 BRANCH)
My setup (as a reference):
Motorola XT875 aka Targa aka Bionic running Android 4.2.2, Avatar ROM (based on Cyanogenmod CM10.1) Latest Weekly Beta
16G internal device/SD storage, 16G patriot class 10 MicroSDHC. Dual-core ARMv7 armhf @ 1.2ghz, using DEBIAN-STABLE (Wheezy) under Galoula Linux Installer installed to it's own separate Ext3 partition on the external SD and Android Bind active. Also using my tablet, an a13 clone, with same Linux configuration.
I have a 2GB disk image on the internal, and a 2GB image on the microsd and they are combined as RAID0 (striping) w/ 4k stripes, ext3 fs w/1kb blocksize, 4kb stripe stride and compressed w/LZMA compression level 3-8 depending on my mood. It is setup like this for storing source code (cm-10.1 source actually, I am going to attempt to compile it entirely on mg phone, or at the very least my own modified portions.
Step 1.
--> If you're a pro, install all the stuff above, skim this section (mainly the part on finding an open loop) and skip to Step 3. Otherwise continue reading.
1a.
Boot up your linux distro on your Android. For me this means opening TerminalIDE, launching a term window, typing 'su' to gain root and then 'linuxchroot.sh' to launch the linux environment.
1b.
Then, ensure you have loop/fuse support.
Code:
ls /dev/block/loop*
.. should list all your systems loop devices, you can also type:
Code:
losetup /dev/block/loopX
(where X is a device number) to find out if the device is free or used by another app.
A response like:
Code:
losetup: /dev/block/loop0: 0
generally means it's available, while
Code:
losetup: /dev/block/loop1: 0 /mnt/asec/com.some.app/data.img
means another app is using it and you shouldn't.
Remember this as you will need to find anywhere from 1-4+ unused loop devices depending on your desired configuration.
1c. *
Next, it's best to ensure you're distro is up-to-date, this is optional* but recommended.
Code:
apt-get update
..then..
Code:
apt-get upgrade
..then, again (for good measure: update before you upgrade, upgrade, then update the upgrades..)
Code:
apt-get update
Step 2.
2a.
Once that completes, start installing the required packages:
Code:
apt-get install lvm2 fuse fuse-utils libfuse libfuse2 gvfs-fuse
Press Y and agree to any dependencies, wait for it to finisn.
2b.
You will need to choose which filesystem to use. I recommend ext3, but you will have to choose which is best for your needs. FAT is supported almost anywhere out-of-box, but is CRAP. NTFS is a bit better but is likely unsupported by your device, exFAT may be an option even. Whichever you choose you will need the appropriate tools (mkfs) to create/format it.
For both FAT and EXT support (or rather creation ability - mounting & read/write support may be limited by your kernel/modules loaded):
Code:
apt-get install dosfstools e2fsprogs e2fslibs
Step 3.
In this step you will create the disk image(s), initialize the partition table(s) and file system(s) and the steps will vary based on your goals, so read carefully. Since we are dealing with images, there shouldn't be much risk for damaging your device/data as long as you're careful. I should note, for the less experienced, that incorrect parameters with some of these commands can wipe your device or SD, if you read the directions carefully and only use the disk image(s) with these tools you will be safe, of course, after you put data within the image, it can also be overwritten with some commands.
3a.
You will now decide which, and how many devices to put images and how large they will be, and most importantly, your logical volume (RAID, non-RAID, span/JBOD, etc.) configuration. I will go into brief detail on the different configurations and use cases.
RAID0 aka STRIPING
Uses/Performance/Pros/Cons
Primarily aimed at performance, NO REDUNDANCY PROVIDED.
RAID0 will write one X byte block per Y devices looping in order. This effectively splits/distributes your data evenly among member devices.
Disk size will be size=devices*device size (or *disk img size)
All images (or devices) need to be the same size, extra bytes will be left unused.
Performance (P) (aka throughput) should be/is about (in theory) S*N=P where S is the average device speed (among all members, assuming similar/identical devices & busses, also note that P will never really exceed the slowest devices avg/max speed*number of drives, so slower drives are a bottleneck, try to use equal or identical devices) and N is the number of devices. On a physical hard disk drive, seek times can be lowest avg seek*members at best sometimes, though seek latency isn't an issue on our flash devices.
My benchmarks hardly show any impressive performance increase however my internal and external sd are obv. not matched, and I haven't done a ton of testing/benching/optimising.
Raid0 examples: 2x4GB disk images, each on a class 10 MicroSDHC card will appear as one 8GB device that can perform as fast as both devices can work together, about twice as fast as one, seek times won't decrease much. 4x200GB SATA drives striped will be one 800GB drive that is almost 4x as fast, however seek times may decrease to 1/4.
RAID1 aka MIRRORING
Uses:
Primarily aimed at data redundancy.
RAID1 will write one X byte block to EVERY device. This effectively MIRRORS your data among member devices.
Disk size will be equivilent to the device/img size regardless of the amount of devices it's mirrored to.
WRITE PERFORMANCE will be the same as the slowest single devicd, READ PERFORMANCE can increase up to x*member devices as it can act 'sort of like' RAID0 for reading, it doesn't need to read EVERY byte from EVERY device, it only needs to read 1/N (n=num of devices) from each device reading all simultaneously.
I have not tested mirroring in this setup yet.[/b]
JBOD aka JussaBunchO'Disks
Uses:
Just basically spans the devices for extra space. No performance techniques, striping or mirroring, rolls off the end of one disk (or img) onto the next. Devices/imgs can be any size and there's no performance hit for using devices/imgs on the same bus or device.
OK -- I started this a while ago, and here's my attempt at finishing it.
Not sure if you need fusefs unless you want to do file system compression.
You do need the lvm2 package, however.
Get down to it:
Now you have to create your image files, loop them, and raid them.
To create the images, use this command (this creates a ~100mb image, adjust the size as necesary):
Code:
dd if=/dev/zero of=/mnt/sdcard0/.myRaidImage-0.img bs=1024k count=100k
Code:
dd if=/dev/zero of=/mnt/sdcard1/.myRaidImage-1.img bs=1024k count=100k
Then, loop them to two free loop devices (see above to find out if they're free):
Code:
losetup /dev/block/loop1 /mnt/sdcard0/.myRaidImage-0.img
losetup /dev/block/loop2 /mnt/sdcard1/.myRaidImage-1.img
Before using any of the lvm2 tools below, you should check that your /etc/lvm/lvm.conf is correct, you may have to change the dirs from /dev to /dev/block for example.
Now, use pvcreate to create lvm physical volumes with those images:
Code:
pvcreate /dev/block/loop[1,2]
And vgcreate to create a lvm volume group for your lvm volumes:
Code:
vgcreate vg0 /dev/block/loop[1,2]
Now, we can create either a JBOD/spanned/contiguous disk, a mirrored disk, or a striped disk.
First, we will create a mirrored logical volume, with a log (in contrast to lvm2's raid1 mode which has no log):
Note: I had to force lvm2 to manage devices in /dev instead of relying on udev (which my device doesn't have), this is quite possibly the case for you as well, the option is in /etc/lvm/lvm.conf.
Code:
lvcreate -m 1 --mirrorlog mirrored -l 40%VG vg0
If that is successful, you now have a usable logical volume. To actually use it however, you will have to activate a filesystem on the device.
Code:
mkfs.ext3 /dev/vg0/lvol0
Then, you can create a mount point and use it:
Code:
mkdir /mnt/mirrored
mount /dev/vg0/lvol0 /mnt/mirrored
NOW YOU CAN USE IT.
The gentoo wiki has a great page on lvm2, and how to use it: https://wiki.gentoo.org/wiki/LVM
Most of it can be applied here with my instructions.
I will come back later and add specific instructions for striping and other modes, including stream compression and perhaps encryption - however - it make take me a while, which is why I posted the link - you should be able to figure it all out with this & that
reserved!
Hi,
do you have any update?
I'm wondering know how to do it.
Using madam or something else?
Thanks.
Sorry, I was sidetracked and forgot to finish. I will try to remember to finish this soon.
So, I got around to finishing it, at least to the point where it's usable and demonstrates mirroring. Not much different process for the other modes, but I will get around to exact instructions later if needed.
cheers.
So, I got around to finishing it, at least to the point where it's usable and demonstrates mirroring. Not much different process for the other modes, but I will get around to exact instructions later if needed.
cheers.
I implemented a browser based encryption solution which runs on Windows RT (and many other Windows computers). All I wrote was the HTML page, I am leveraging Crypto.JS javascript library for encryption algorithm. I am using the HTML 5 File API implementation which Microsoft provides for reading and writing files.
I make no claim on this but seems to work good for me. Feel free to feedback if you have any suggestions. The crypto.js library supports many different algorithms and configuration so feel free to modify it to your own purposes.
You can download the zip file to your surface, extract it and load the TridentEncode.htm file into Internet Explorer.
If you want to save to custom directory you probably need to load it from the Desktop IE instead of metro IE (to get the file save dialog). I usually drag and drop the file onto desktop IE and from there I can make favorite. This should work in all IE 11 and probably IE 10 browsers... if you use other browsers you may need to copy paste into the fields since the File API implementation seems rather browser specific. Running the html page from the local filesystem means that there is no man-in-the-middle which helps eliminate some of the vulnerabilities of using a javascript crypto implementation. You could also copy the attached zip file to your skydrive to decrypt your files from other computers.
Skydrive files in theory are secure (unless they are shared to public) so this might be useful for adding another layer of protection to certain info.
Again, use at your own risk, but feel free to play around and test it, and offer any suggestions or critiques of its soundness, or just use it as a template for your own apps.
Ok... this is really cool! Nice idea, and a good first implementation.
With that said, I have a few comments (from a security perspective). As an aside, minimized JS is the devil and should be annihilated with extreme prejudice (where not actually being used in a bandwidth-sensitive context). Reviewing this thing took way too long...
1) Your random number generation is extremely weak. Math.random() in JS (or any other language I'm aware of, for that matter) is not suitable for use in cryptographic operations. I recommend reading http://stackoverflow.com/questions/4083204/secure-random-numbers-in-javascript for suggestions. The answer by user ZeroG (bottom one, with three votes, as of this writing) gets my recommendation. Unfortunately, the only really good options require IE11 (or a recent, non-IE browser) so RT8.0 users are SOL.
NOTE: For the particular case in question here (where the only place I can see that random numbers are needed is the salt for the key derivation), a weak PRNG is not a critical failing so long as the attacker does not know, before the attack, what time the function is called at. If they do know, they can pre-compute the likely keys and possibly succeed in a dictionary attack faster than if they were able to generate every key only after accessing the encrypted file.
2) Similarly, I really recommend not using a third-party crypto lib, if possible; window.crypto (or window.msCrypto, for IE11) will provide operations that are both faster and *much* better reviewed. In theory, using a JS library means anybody who wants to can review the code; in practice, the vast majority of people are unqualified to either write or review crypto implementations, and it's very easy for weaknesses to creep in through subtle errors.
3) The default key derivation function (as used for CryptoJS.AES.encrypt({string}, {string})) is a single iteration of MD5 with a 64-bit salt. This is very fast, but that is actually a downside here; an attacker can extremely quickly derive different keys to attempt a dictionary attack (a type of brute-force attack where commonly used passwords are attempted; in practice, people choose fairly predictable passwords so such attacks often succeed quickly). Dictionary attacks can be made vastly more difficult if the key derivation process is made more computationally expensive. While this may not matter so much for large files (where the time to perform the decryption will dominate the total time required for the attack), it could matter very much for small ones. The typical approach here is to use a function such as PBKDF2 (Password-Based Key Derivation Function) with a large number of iterations (in native code, values of 20000-50000 are not uncommon; tune this value to avoid an undesirably long delay) although other "slow" KDFs exist.
4) There's no mechanism in place to determine whether or not the file was tampered with. It is often possible to modify encrypted data, without knowing the exact contents, in such a way that the data decrypts "successfully" but to the wrong output. In some cases, an attacker can even control enough of the output to achieve some goal, such as compromising a program that parses the file. While the use of PKCS7 padding usually makes naïve tampering detectable (because the padding bytes will be incorrect), it is not a safe guarantee. For example, a message of 7 bytes (or 15 or 23 or 31 or any other multiple of 8 + 7) will have only 1 byte of padding; thus there is about a 0.4% (1 / 256) chance that even a random change to the ciphertext will produce a valid padding. To combat this, use an HMAC (Hash-based Message Authentication Code) and verify it before attempting decryption. Without knowing the key, the attacker will be unable to correct the HMAC after modifying the ciphertext. See http://en.wikipedia.org/wiki/HMAC
5) The same problem as 4, but from a different angle: there's no way to be sure that the correct key was entered. In the case of an incorrect key, the plaintext will almost certainly be wrong... but it is possible that the padding byte(s) will be correct anyhow. With a binary file, it may not be possible to distinguish a correct decryption from an incorrect one. The solution (an HMAC) is the same, as the odds of an HMAC collision (especially if a good hash function is used) are infinitesimal.
6) Passwords are relatively weak and often easily guessed. Keyfiles (binary keys generated from cryptographically strong random number generators and stored in a file - possibly on a flashdrive - rather than in your head) are more secure, assuming you can generate them. It is even possible to encrypt the keyfile itself with a password, which is a form of two-factor authentication: to decrypt the data that an attacker wants to get at, they need the keyfile (a thing you have) and its password (a thing you know). Adding support for loading and using keyfiles, and possibly generating them too, would be a good feature.
The solutions to 3-5 will break backward compatibility, and will also break compatibility with the default parameters for openssl's "enc" operation. This is not a bad thing; backward compatibility can be maintained by either keeping the old version around or adding a decrypt-version selector, and openssl's defaults for many things are bad (it is possible, and wise, to override the defaults with more secure options). For forward compatibility, some version metadata could be prepended to the ciphertext (or appended to the file name, perhaps as an additional extension) to allow you to make changes in the future, and allow the encryption software to select the correct algorithms and parameters for a given file automatically.
Wow thanks GDTD that's great feedback
Not sure about his minified sources, the unminified aes.js in components is smaller than the minified version (which I am using) in rollups. I'll have to look into what his process for 'rollup' is to see if I can derive a functional set of non-minified script includes. If I can do that it would be easier to replace (what I would guess is) his reliance on Math.random.
His source here mirrors the unminified files in components folder : https://code.google.com/p/crypto-js/source/browse/tags/3.1.2/src
msCrypto that would be great, I had no idea that was in there. I found a few (Microsoft) samples so I will have to test them out and see if I can completely substitute that for crypto.js. Would be more keeping in line with the name I came up with.
Currently this version only works for text files, I am using the FileAPI method reader.readAsText(). I have been trying to devise a solution for binary files utilizing reader.readAsArrayBuffer but as yet I haven't been able to convert or pass this to crypto.js. I will need to experiment more with base64 or other interim buffer formats (which Crypto.js or msCrypto can work with) until I can get a better understanding of it.
Metadata is a great idea, maybe i can accommodate that with a hex encoded interim format.
You seem extremely knowledgeable in the area of encryption, hopefully i can refine the approach to address some of the issues you raised by setting up proper key, salt, and IV configuration... I'm sure I will understand more of your post as i progress (and after reading it about 20 times more as a reference).
Too bad we don't a web server for RT, that would at least open up localStorage for json serialization (mostly for other apps I had in mind). I guess they might not allow that in app store though. Could probably run one of a developers license though (renewed every 1-2 months)?
nazoraios said:
Too bad we don't a web server for RT, that would at least open up localStorage for json serialization (mostly for other apps I had in mind). I guess they might not allow that in app store though. Could probably run one of a developers license though (renewed every 1-2 months)?
Click to expand...
Click to collapse
I cant comment too much on the encryption, GoodDayToDie has covered anything I could contribute and more. But there is a functioning web server on RT. Apache 2.0 was ported: http://forum.xda-developers.com/showthread.php?t=2408106 I dont know if everything is working on it, I dont own an RT device and last time I tried I couldnt get apache to run on 64 bit windows 8 anyway (needed it at uni, spent hours going through troubleshooting guides and it never worked on my laptop, gave up and ran it under linux in virtualbox where it took 2 minutes to have functioning the way I needed it to).
Curious about the performance. Speaking of encryption, 7-Zip has it built-in, and from the discuss in StackExchange, it seems pretty good.
One of the neat things about this thing (local web app? Pseudo-HTA (HTml Application)? Not sure if there's a proper name for such things) is that it runs just fine even on non-jailbroken devices. That's a significant advantage, at least for now.
Running a web server should be easy enough. I wrote one for WP8 (which has a subset of the allowed APIs for WinRT) and while the app *I* use it in won't be allowed in the store, other developers have taken the HTTP server component (I open-sourced it) and packaged it in other apps which have been allowed just fine. With that said, there are of course already file crypto utilities in the store anyhow... but they're "Modern" apps so you might want to develop such a server anyhow so you can use it from a desktop web browser instead.
Web cryptography (window.crypto / window.msCrypto) is brand new; it's not even close to standardization yet. I'm actually kind of shocked MS implemented it already, even if they put it in a different name. It's pretty great, though; for a long time, things like secure random numbers have required plugins (Flash/Java/Silverlight/whatever). Still, bear in mind that (as it's still far from standardized), the API might change over time.
Yep, I think of them as Trident apps since trident is what Microsoft calls their IE rendering engine, but I guess they are sort of offline web apps (which come from null domain). Being from null domain you are not allowed to use localstorage which is domain specific. You also are not allowed to make ajax requests. You just have file api and json object serialization to make do with I/O.
Another app I am working on is a kind of Fiddler app similar to http://jsfiddle.net/ where you can sandbox some simple script programs.
Kind of turning an RT device into a modern/retro version of a commodore 64 or other on-device development environments. Instead of basic interpreter you've got your html markup and script.
I have an attached demo version which makes available jquery, jquery-ui, alertify javascript libraries in a sandbox environment that you can save as .prg files.
I put a few sample programs in the samples subfolder. Some of the animation samples (like solar system) set up timers which may persist even after cleared so you might need to reload the page to clear those.
It takes a while to extract (lots of little files for all the libraries) but once it extracts you can run the html page and I included a sample program 'Demo Fiddle.prg' you can load and run to get an idea.
I added syntax highlighting editors (EditArea) which seems to work ok and let's you zoom each editor full screen.
The idea would be to take the best third party javascript libraries and make them available and even make shortcuts or minimal API for making it easier to use them. Common global variable, global helper methods, ide manipulation. I'd like to include jqplot for charting graphs, maybe for mathematical programs and provide api for user to do their own I/O within the environment.
These are just rough initial demos, and obviously open source so if anyone wants to take the ideas and run with them i'd be interested in seeing what others do. Otherwise I will slowly evolve the demos and release when there are significant changes.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Wandering through the forums and various websites dedicated to Android, we are constantly confronted with tips on how to increase the performance of the smartphone. Some recommend to include swap, others - add special value to build.prop, and others - to change variables of the Linux kernel. Such recipes in different ways you can find a huge amount of that on the XDA and 4PDA. But do they actually work
Tenacity with which some seemingly competent smartphone users are trying to push their ideas public optimum adjustment Android and the underlying Linux kernel. And the right to it was limited to a slight tuning virtual memory management subsystem, or the inclusion of experimental options. No, we usually offer a very long use scripts that change literally every variable core, remount filesystems with different odd options, including the swap, activate various system daemons and perform billions of different operations.
No, well, you can, of course, assume that the Linux kernel, Android and proprietary firmware for smartphones develop illiterate idiots, whose work must be radically alter, but in practice some reason it turns out that the most famous tuning tools, published on XDA, - it nothing but a hodgepodge of disparate huge number of recommendations, it is not clear who invented and no one knows why. The absurdity of the situation reaches that these instruments can be found rows copied unchanged from the scripts to increase Linux-server performance under heavy loads (I'm not kidding, look at the contents of the famous script ThunderBolt!).
In general, the situation is more complicated than. All advise all, no one suggests anything, and those who understand something, sitting and drinking tea and laughing over what's happening.
Swap
Let's start with the swap - the most absurd ideas of all that you can think of for use in smartphones. Its purpose is to create and connect the paging file, thereby manage to free storage space in memory. The idea itself is certainly sensible, but only if it is a server, which rests on the interactivity of nowhere. Using your phone regularly used pagefile will lead to lags arising from slips past the cache - just imagine what will happen if an application tries to display one of their icons, and it will be in the swap, which will have to re-load the disc, after freeing up space by placing data swap another application. Horror.
Some users can be argued that in fact after the swap no problems, but for this we must thank the mechanism lowmemorykiller, which regularly kills very swollen and have not used the application. Thanks to him, the device with 1GB of memory can never reach the necessary performance data in a swap. He is the reason why, in contrast to the Linux-desktop on the Android swap is not needed.
Verdict: A very stupid idea, the implementation of which is fraught with serious lags.
zRAM
The idea is so right that even Google recommends zRAM for KitKat-based devices in the event that the amount of RAM is less than 512 MB. Only snag is that the method only works for modern cost devices based on multi-core processors from the budget any MTK and 512 MB of RAM. In this case, the encryption stream can be taken to separate the kernel and do not care about performance.
On older devices with a single core, and which recommend the use of this technology, we again get the lags, and in fairly large quantities. The same, incidentally, applies to technology KSM (Kernel Same Page Merging), which allows you to combine identical memory pages, thus freeing space. It also recommended that Google, but on older devaysakh leads to an even greater lags, which makes sense, given the constantly active core thread that runs continuously from memory in search of duplicate pages.
Verdict: it depends on the device, in most cases, the system slows down.
Seeder
At the time this application has done a lot of noise and gave rise to many analogues. The network has a huge number of reports of alleged phenomenal growth performance of the smartphone after installation. Homegrown custom firmware collectors have begun to include him in their assembly, and the author was declared a savior. And all this despite the fact that Seeder not doing anything dirty hacks, but just corrected a stupid bug Android.
The bug consisted in the fact that some high-level components of the Android runtime actively used the file /dev/random to get entropy. In some moments buffer /dev/random devastated, and the system is blocked until it is filling the required amount of data. And as he filled that have been reported from various sensors, buttons and sensors of the smartphone, the time for this procedure took so much time to notice that lag.
To solve this problem the author Seeder took Linux-demon rngd, compiled it for Android inastroil so that he took random data from a much faster (but also much more predictable) /dev/urandom, and every second merges them into /dev/random, without allowing the latter exhausted. As a result - the system never experienced a lack of entropy and work quietly.
This bug was closed back in Google Android 3.0, and it would seem, we do not need to think about Seeder. But the fact that the application has since actively developed and even today, is recommended by many "experts" for the application. Moreover, the application has several analogues (eg, sEFix), and many of the creators of the scripts/tools to accelerate still include this functionality in their creations. Sometimes it is the same rngd, sometimes - the demon haveged, sometimes just symlink /dev/urandom on /dev/random.
Everyone who tried it, excitedly shouting about the effectiveness of the solution, however, according to Ricardo Cerqueira from the company Cyanogen, in newer versions of Android /dev/random is used in all three components: libcrypto (encryption SSL-connections, generating SSH keys and etc.), wpa_supplicant/hostapd (to generate the WEP/WPA-keys) and several libraries for generating random ID to create ext2/3/4 filesystems.
Application Efficiency in today's Android, in his opinion, is not connected with the completion of the pool /dev/random, and that rngd constantly awakens the device and causes it to increase the frequency of the processor, which has a positive effect on performance and negative on the battery.
Verdict: The placebo effect.
Odex
Stock firmware smartphones always odex. This means that along with the standard package for Android apps in APK format directory /system/app/ and /system/priv-app/ (since KitKat) are also of the same name files with the extension odex. They contain so-called optimized bytecode applications already passed through the validator and optimizer virtual machine and recorded in a separate file (this is done using dexopt utility).
Meaning odex files to offload virtual machine and thus speed up the launch of the application (runoff). On the other hand, ODEX files to prevent modifications to make to the firmware, create problems with the update, and for this reason many custom ROMs (including CyanogenMod) distributed without them. Return (or rather, generate) files odex a variety of ways, including using simple tools/scripts like Odexer Tool. Using them is easy, and many of the "experts" are advised to do so.
The only problem is that this is purely a placebo. Not finding odex-files in the directory/system, the system itself will create them the next loaded and placed in the directory /system/dalvik-cache/. This is what she does when loading a new firmware on the screen the message "Busy ... Optimizing Applications." In relation to applications from the convenience store is also, incidentally, works. But at the stage of installation of the software.
Verdict: The placebo effect.
Lowmemorykiller tweaks
The implementation of multitasking in Android is very different from other mobile operating systems and is based on the classical model. Applications can work quietly in the background, in the system there are no restrictions on the number, the functionality of the transition to a background execution is not curtailed. All as on the desktop, except for one detail: the system has every right to kill any background application in the case of lack of memory, or (since KitKat) excessive greed application resources.
This mechanism, called lowmemorykiller, was coined to, retaining features of a full-fledged multi-tasking OS, Android could live normally in a limited amount of memory and lack of swap-partition. The user can easily launch any application and quickly switch between them, and the system will take care of the long-unused application completion and to always remain free memory in the device.
In the early days of Android purpose of this mechanism for many users it was unclear why have become popular so-called task-killer - apps that from time to time to wake up and have completed all background applications. Profits in this case, it was considered a large amount of free RAM, which was perceived as a plus, though no advantages in this, of course, was not. But there were many disadvantages in the form of a longer switch between applications, increased battery consumption and problems c awakening in the morning the owner (Service also kills).
Over time, multitasking principles of understanding has come, and from task-killers gradually abandoned. However, they were quickly replaced by another trend - tuning of lowmemorykiller mechanism (for example, by MinFreeManager applications). The main idea of the method is to lift the boundaries of filling the RAM at which the system will start to kill background apps. A sort of the way ", and us and you", which allows you to free up some memory by regular means, without disturbing ideas Android multitasking.
But what it ultimately leads? For example, the standard value memory is full of borders - a 4, 8, 12, 24, 32 and 40 Mb, that is when the free storage space of 40 MB will be killed by one of the cached applications (loaded in memory, but is not running, is this optimization Android ), with 32 - Content Provider, has no customers, 24 - one of the seldom-used back-end application, then go to the expense of service processes application (for example, the music player service), visible on the app screen and the current running application. The difference between the last two is that the "current" - this application, which is currently dealing the user, and the "visible" - is that, for example, has a notification in the status bar or display on top of the screen any information.
In general, all this means that the smartphone will always be available 40 MB of memory, which is enough to accommodate another application, and then start LKM flow and begin cleanup. All OK, everyone is happy. The system uses the maximum memory. Now imagine what would happen if the user take advantage of advice homebrew "expert", and raise these values so that the latter would be, well, let's say, 100 MB (usually raised only the last three values). In this case, it happens one simple thing: the user will lose 100 - 40 = 60 MB of memory. Instead of using this space to store back-end applications, it is useful, as it reduces the time to switch to them, and the charge of the battery, the system will keep it free is not clear why.
It is fair to say that the LKM tuning can be useful for devices with very very little memory (less than 512) and Android 4.X on board, or to temporarily increase thresholds. Some developers tweaks directly recommend the use of "aggressive" setting only if you run heavy software like hi-end games, and all the rest of the time to stay on the standard. This really makes sense.
Verdict: better not to touch.
I/O tweaks
The scripts that are published on the forums, you can often find tweaks I/O subsystem. For example, in the same script ThunderBolt! has the following lines:
Code:
echo 0 > $i/queue/rotational;
echo 1024 > $i/queue/nr_requests;
The first gives the I/O scheduler to understand that he is dealing with a solid state drive, the second increases the maximum size of the queue IO 128 to 1024 ($i variable in commands contains a path to the tree of block device in /sys, eg /sys/block/mmcblk0/, the script runs on them in the loop). Hereinafter you can find the following line relating to the CFQ scheduler:
Code:
echo 1 > $i/queue/iosched/back_seek_penalty;
echo 1 > $i/queue/iosched/low_latency;
echo 1 > $i/queue/iosched/slice_idle;
This is followed by a few more lines belonging to other planners (by the way, pay attention to a whole extra semicolon at the end of instruction). What all of these lines is not so? The first two commands are pointless for two reasons:
1. Schedulers I / O in a modern Linux kernel itself able to understand what type of storage medium they deal.
2. Such a long input-output queue (1024) completely useless on a smartphone. Moreover, it is meaningless, even on the desktop and is used on heavy duty servers (of tuning recommendations which it, apparently, and got into the script).
The last three are meaningless for the simple reason that for a smartphone, where there is virtually no separation applications prioritized in the input-output and there is no mechanical drive, the best planner - is the noop, that is simple the FIFO-queue - who first turned to memory, he also got access. And this scheduler is not any special settings. Therefore, all these multi-screen lists commands better replaced by a simple cycle:
Code:
for i in /sys/block/mmc*; do
echo noop > $i/queue/scheduler
echo 0 > $i/queue/iostats
done
In addition to enabling noop scheduler for all drives it off the accumulation of statistics I/O, which should also have a positive impact on the performance (although this is only a drop in the sea, which will be completely invisible).
Another tweak that can often be found in the scripts tuning performance - this increase readahead values for memory cards up to 2 MB. readahead mechanism for early reading data from the media even before the application requests access to these data. If the kernel sees that someone long enough to read the data from the media, it is trying to figure out what data will be needed in the future application and pre-loads them into RAM, thereby enabling to reduce the time of their return.
It sounds cool, but, as practice shows, readahead algorithm is very often wrong, which leads to unnecessary operations of input-output and consumption of RAM. High values readahead (1-8 MB) are recommended for use in RAID-arrays, whereas on the desktop or smartphone is better to leave everything as is, that is 128 KB.
Verdict: in addition to noop, do not need anything.
Tweaks virtual memory management system
In addition to the subsystem I/O, it is also common to do tuning virtual memory management subsystem. Often, change affects only two kernel variables: vm.dirty_background_ratio and vm.dirty_ratio, which allow you to adjust the size of the buffer for storing the so-called dirty data, ie the data that has been written to disk application, but more are still in memory and waiting until they are written to disk.
Typical values of these variables in the desktop Linux-distributions and Android are as follows:
Code:
* vm.dirty_background_ratio = 10
* vm.dirty_ratio = 20
This means that when the "dirty" data buffer size in 10% of the total amount of RAM wake pdflush nuclear flow and starts to write data to disk. If the operation of recording data on the disk will be too intense and even though the job pdflush, the buffer will continue to grow, then when it reaches 20% of the amount of RAM the system switches all the subsequent write operation in synchronous mode (without pre-buffer) and the work of writing to disk application will be blocked until such time as the data is written to disk (in the terminology of Android is called a lag).
It is important to understand that even if the buffer size is not reached 10%, the system anyway pdflush starts the flow after 30 seconds. What we are given this knowledge? In fact, anything that we could use for their own purposes. The combination of 10/20% is quite reasonable, for example, on your smartphone with 1 GB of RAM is about 100/200 MB of memory, which is more than enough in terms of rare bursts records that speed is often below the speed record in system NAND-memory, or SD-card (when installing software or copying files from a computer). But the creators of scripts optimization with this, of course, disagree.
For example, in Xplix script can find something like this (in the original, they are much longer because of the checks on the amount of RAM and use BusyBox):
Code:
sysctl -w vm.dirty_background_ratio=50
sysctl -w vm.dirty_ratio=90
These commands apply to devices with 1 GB of memory, that is, set limits on "dirty" buffer equal to (approximately) 500/900 MB. These high values are absolutely meaningless for the smartphone, as only work under constant intense recording on the disc, that is, besides for heavy server. In a situation with a smartphone they are no better than the standard. By the way, in the script ThunderBolt! used much more reasonable (and close to the standard) values, but I doubt that by their application the user will notice at least some difference:
Code:
if [ "$mem" -lt 524288 ];then
sysctl -w vm.dirty_background_ratio=15;
sysctl -w vm.dirty_ratio=30;
elif [ "$mem" -lt 1049776 ];then
sysctl -w vm.dirty_background_ratio=10;
sysctl -w vm.dirty_ratio=20;
else
sysctl -w vm.dirty_background_ratio=5;
sysctl -w vm.dirty_ratio=10;
fi;
The first two commands are run on smartphones with 512 MB of RAM, the second - with 1 GB, and others - with more than 1 GB. But in fact there is only one reason to change the default settings - a device with a very slow internal memory or memory card. In this case it is reasonable to spread the values of the variables, that is, to make something like this:
Code:
sysctl -w vm.dirty_background_ratio=10
sysctl -w vm.dirty_ratio=60
Then, when a surge system write operations, without having to record data on the disc, up to the last will not switch to synchronous mode, which will allow applications to reduce lag when recording.
Verdict: better not to touch.
P.S.
There are numerous and smaller optimizations, including the "tuning" of the network stack, changing the variables of the Linux kernel and Android (build.prop), but 90% of them have no effect on the real performance of the device, while the remaining 10% or improve some aspects of behavior devices at the expense of others, or so insignificant increase productivity, you do not even notice it. From what really works, we can note the following:
****Acceleration. The small acceleration to improve performance and andervolting - save a little battery.
****Database Optimization. I seriously doubt that this will give a noticeable increase in speed, but the theory tells us that the work should be.
****Zipalign. Ironically, despite the built-in Android SDK feature content alignment within the APK-file in the store you can find a lot of software is not transmitted through zipalign.
****Disable unnecessary system services, removing unused system and seldom-used third-party applications.
****Custom kernel with optimizations for a specific device (again, not all nuclei are equally good).
****Already described I/O scheduler noop.
****Saturation algorithm TCP westwood +. There is evidence that he is in wireless networks more efficiently used in the default Android Cubic. Available in custom kernels.
Useless settings build.prop
LaraCraft304 from XDA Developers forum has conducted a study and found that an impressive number of /system/build.prop settings that are recommended for use "experts" do not exist in the source AOSP and CyanogenMod. Here's the list:
ro.ril.disable.power.collapse
ro.mot.eri.losalert.delay
ro.config.hw_fast_dormancy
ro.config.hw_power_saving
windowsmgr.max_events_per_sec
persist.cust.tel.eons
ro.max.fling_velocity
ro.min.fling_velocity
ro.kernel.checkjni
dalvik.vm.verify-bytecode
debug.performance.tuning
video.accelerate.hw
ro.media.dec.jpeg.memcap
ro.config.nocheckin
profiler.force_disable_ulog
profiler.force_disable_err_rpt
ersist.sys.shutdown.mode
ro.HOME_APP_ADJ
To be continued...
thanks man!
this helped me understand some things
"No, well, you can, of course, assume that the Linux kernel, Android and proprietary firmware for smartphones develop illiterate idiots, whose work must be radically alter"
this made me lol
This...
Was...
AWESOME!
Pure information furthermore useful bits and pieces. Sorry for the necro but I just felt plain wrong to tap a button to say thanks. I really felt like I'd be the minimum to write thanks personally. Thank you very much for this post.
I swear to you windowsmgr.max_events_per_sec works flawlessly, the higher number, the more content will be able to load on launcher without lag. I used to test it on my J7 pro android 8.0
I subscribe the OP's statements that we are constantly are searching for optimization(s) of our Android device. I used all kind of Android devices, like Sony, Oppo, etc for many years with varying satisfaction, also regarding Android updates. I know that what I'm going to say now, is very sensitive in the Android community, but nevertheless I will post it. After having used Fitbit as my fitness trackers/smartwatches, I noticed the release of the Apple iWatch 6 and I was stunned about the capabilities of this watch versus all other smarwatches. Alas I needed also an iPhone to get it up and running. Let me be clear, I never was an Apple fan due to it's too strict policy and the fact that you were not able to make your own homescreen. But, since the release of iOS 14 this all changed, so I decided to get over my Apple resistance and bought the Apple 12 Pro Max. Using this now for a few months, this is the best upgrade I ever made. Everything runs smooth, fast, lot's of space, smooth connection with the iWatch, etc etc. I designed my own home display even on an Apple device (see attachment). Looks exactly the same as the home display I had on all of my former Android devices. Hope this information is in some way helpfull for you all. Kindest regards kuzibri
P.S. If you've got interested in more information about the Apple 12, see my threads in the Apple fora of XDA:
1. https://forum.xda-developers.com/t/q-a-template-for-the-apple-iphone-12-pro-max.4322579/
2. https://forum.xda-developers.com/t/ask-away-thread-for-all-apple-iphones.4323471/
Utility for background calibration, curation and tuning of the device towards an intuitive interface.
Subsystems being battery, entropy, encryption, disk, cpu, memory, filesystem, ui, scheduler, and network, all safe and open source technology.
Presented in this educational gaming metric format with infinite feedback and an interestingly assymetric chance. scribble anywhere, check in some stress, or find the 8!
waut.ch! does one hope to receive from this?
Well, increasing degrees and amounts of a certain "Je ne sais quoi" or responsiveness from the user interface for a start. Better battery life perhaps. Better quality of life, maybe.
And waut.ch! can only perhaps be described as "A qualified quantification of the placebo effect"
waut.ch! might benefit from this?
In the Android device space:
Designers
Users
Manufacturers
Recyclers
Developers
Compilers
Support personnel
OEMs
The Friendly Neighborhood Nerd/Technician.
“Make the most of yourself....for waut.ch! is all there is of you.” - Ralph Waldo Emerson ( paraphrase )
All along the waut.ch! tower - Bob Dylan
waut.ch! - Sometimes used in some colloquium as "watch!", keen upon reducing the TDP of mobile devices to 1.0 watt!
ARM variants of Android only Donut 1.6+
Please uninstall either Seeder or CrossBreeder prior to using this.
Root recommended, else reactivity metric is interesting and introduces uniqueness into the entropy pool anyway. Metric may demonstrate a certain asymmetry that is expected from predictable human actions. Efforts have been made to remove time seed logic from haveged in order to improve upon encryption and system-wide performance and security.
Also numerous other subsystems require careful calibration to facilitate this process.
Rewritten code, subset of functionality for upstream project - CrossBreeder ( https://forum.xda-developers.com/showthread.php?t=2113150 )
Please feel free to view and analyze source and functionality and report bugs and discuss etc on the XDA forum:
( https://forum.xda-developers.com/android/apps-games/app-waut-ch-calibration-android-t3549967 )
Google Play store:
( https://play.google.com/store/apps/details?id=ch.waut )
Please visit: /data/data/ch.waut/files/bin on the device itself for partial shell source code and XDA Downloads section and Github for full source code.
Reboot at convenience liberally or sparingly to reseed the entropy pool or as is known in common parlance, for good luck!
Thanks.
Havged source code:
https://github.com/Openand-I/haveged
Adhoc Payment URL to support development efforts : https://paypal.me/openand/10
Frappe ( "free-paid" ) same-version to support development efforts : http://waut.ch
Custom haveged source code as used in this piece of software: https://github.com/Openand-I/haveged
=====
Version Name: 59a6333e-9ed9-43f8-8dad-51ed46c17e28
cb.sh: cache pressure - 500
cb_io.sh: read_ahead - 0
cb_io.sh: nr_requests - 0
$ md5sum *.apk
661c30b02b2321300624af98feaa5bad *145-waut.ch.apk
661c30b02b2321300624af98feaa5bad *oi.apk
$ sha256sum *.apk
6d23b8da87dc5516583a55a3203c9f5068ea8fe8765ece489080ef663c8aee15 *145-waut.ch.apk
6d23b8da87dc5516583a55a3203c9f5068ea8fe8765ece489080ef663c8aee15 *oi.apk
https://github.com/Openand-I/haveged
https://github.com/openand-inc/waut.ch
https://forum.xda-developers.com/devdb/project/?id=19218#downloads
https://forum.xda-developers.com/android/general/app-waut-ch-calibration-android-version-t3858365
https://github.com/openand-inc/waut.ch/raw/cb0c60025f86a4fdc4778506e97ee80eb1c00b45/oi.apk
https://github.com/openand-inc/waut.ch/raw/master/145-waut.ch.apk
-------------------------
Recommended:
- ntp: automatic system time update from internet is enabled.
please check the clock and fiddle around with the timezone settings in case of any issues. one may need to set the timezone manually.
then simply run the app to initiate a time sync
the network time sync happens at around 3am. so the time to check is in the morning.
- Please disable mount namespace separation in the superuser app to take advantage of the mount optimisations.
- Reboot once and occasionally to reseed the entropy pool. It's good luck!
- Do ensure that the waut.ch service has started upon reboot. Just run if it doesn't start it automatically!
Note: Please note that the haveged binary in the APK is a static binary and works on both PIE and non-PIE environments. It is also UPX compressed. UPX for Android didn't compile. So UPX for linux was used to compress the executable file. It is an elegant solution as both on disk and in memory space(?) is reduced by 70% per executable. One is welcome to decompress the file using 'upx -d'.
Full source code is provided on Github and build scripts are attached here and on Github.
There is no license required to both install the app or distribute it, both within the developer ROM community or in commercial form. Adhoc payment URL to support development - https://www.paypal.me/openand/10
Again do note that the license to use the APP and source code is free worldwide and irrevocable in full or partial form. All other open source components simply inherit their license. But under no circumstances is any use thereof legally binding or relevant.
--------
Utility for background calibration, curation and tuning of the device towards an intuitive interface.
Subsystems being battery, entropy, encryption, disk, cpu, memory, filesystem, ui, scheduler, and network, all safe and open source technology.
Presented in this metric format with infinite feedback and an interestingly assymetric chance. scribble anywhere, check in some stress, or get lucky for that matter!
-----
ARM variants of Android only Donut 1.6+ ( should even be compatible with the latest ARM Android 9+ )
Please uninstall either Seeder or CrossBreeder prior to using this. And other "mods" or "tweaks".
Root recommended, else reactivity metric is interesting and introduces uniqueness into the entropy pool anyway. Metric may demonstrate a certain asymmetry that is expected from predictable human actions. Efforts have been made to remove time seed logic from haveged in order to improve upon encryption and system-wide performance and security.
Also numerous other subsystems require careful calibration to facilitate this process.
Rewritten code, subset of functionality for upstream project - CrossBreeder ( https://forum.xda-developers.com/showthread.php?t=2113150 )
Please feel free to view and analyze source and functionality and report bugs and discuss etc on the XDA forum:
( https://forum.xda-developers.com/android/apps-games/app-waut-ch-calibration-android-version-t3858365 )
Google Play store:
( https://play.google.com/store/apps/details?id=ch.waut )
Please visit: /data/data/ch.waut/files/bin on the device itself for partial shell source code and XDA Downloads section and Github for full source code.
The app will amongst other maintenance tasks tune sqlite databases regularly and reseed the entropy pool or as is known in common parlance, for good luck!
Thanks.
Payment URL: https://paypal.me/openand/10
XDAevDB Information
waut.ch!, App for all devices (see above for details)
Contributors
idcrisis
Source Code:
[url]https://github.com/Openand-I/haveged[/URL]
[url]https://github.com/openand-inc/waut.ch[/URL]
[url]https://forum.xda-developers.com/devdb/project/?id=19218#downloads[/URL]
Previous Version Information - 144
305bd30f-0c8a-40d8-baf5-330c68f62d51
Status: Stable
Created 2017-01-01
Last Updated 2020-08-18
$ md5sum *.apk
8ea8e8c132a584767a12e394f7975654 *144-waut.ch.apk
8ea8e8c132a584767a12e394f7975654 *oi.apk
$ sha256sum *.apk
4925066a106c83b18ac6e563f03331c56b72777e66973db591c9776d706595e3 *144-waut.ch.apk
4925066a106c83b18ac6e563f03331c56b72777e66973db591c9776d706595e3 *oi.apk
https://github.com/Openand-I/haveged
https://github.com/openand-inc/waut.ch
https://github.com/openand-inc/waut.ch/raw/master/144-waut.ch.apk
https://github.com/openand-inc/waut.ch/raw/f699d3763507ec1f91d82b9ce25c53036b460a9e/oi.apk
-----
Version notes:
haveged: static non upx binary used
cb.sh: lock fixes
Recommended:
- ntp: automatic system time update from internet is enabled.
please check the clock and fiddle around with the timezone settings in case of any issues. one may need to set the timezone manually.
then simply run the app to initiate a time sync
the network time sync happens at around 3am. so the time to check is in the morning.
- Please disable mount namespace separation in the superuser app to take advantage of the mount optimisations.
- Reboot once and occasionally to reseed the entropy pool. It's good luck!
- Do ensure that the waut.ch service has started upon reboot. Just run if it doesn't start it automatically!
Please support development, simply use https://paypal.me/openand/10 or the payment URL.
Or you can simply buy the "frappe" ( free-paid ) version of the app: http://waut.ch
Recommended:
- ntp: automatic system time update from internet is enabled.
please check the clock and fiddle around with the timezone settings in case of any issues. one may need to set the timezone manually.
the network time sync happens at around 3am. so the time to check is in the morning.
- Please disable mount namespace separation in the superuser app to take advantage of the mount optimisations.
- Reboot once and occasionally to reseed the entropy pool. It's good luck!
- Do ensure that the waut.ch service has started upon reboot. Just run if it doesn't start it automatically!
Issues:
- superuser - Please disable mount namespace separation in the superuser app ( for optional but recommended mount options ). Also please revisit the app entry inside the superuser app to ensure the waut.ch service can run on boot unattended. One can see the logs on another day to ensure that the scheduler ran correctly in the night.
- Non root users - User Interface can help in clearing the random device. Please try and obtain root to avail of most features
- Some Samsung users - One is also requested to raise a ticket with Samsung who may be running old PE detection rules that flag any compressed EXE.
- x64 users - Reports are that the binaries run on 64 bit as they are static! Please compile one's variant of the binary if required. Entropy generations removes CPU jitter and hence runs cooler and more secure.
- Intel users - User Interface can help in clearing the random device. Please compile one's variant of the binary if required. Entropy generations removes CPU jitter and hence runs cooler and more secure.
- Maintenance scheduler VACCUUMS and INDEXES "ALL" SQLITE databases. Some folks may not like that. But given that they're no WAL mode anyway, it's a bottleneck worth removing safely.
- There is a concerned effort to state that 32-bit ARM Android Go/One < 1 GB RAM devices are all that's required for long term functioning. Higher no issues.
- Please try and use a heap size of 96MB. Attempts have been made to set heap size dynamic but ideally this should be done in the build.
- Please clear cache or factory reset upon issues to gain at least another year of MTTR ( Mean Time To Recovery ) for each device.
Thank you!
License
License concerns:
haveged - inherited - https://github.com/Openand-I/haveged
busybox - inherited - https://github.com/openand-inc/busybox
- Busybox simple extract, possibly edit the .config file in 'vi' and type 'make'
The requisite tools are installed using:
apt-get install gcc-arm-linux-gnueabi
apt-get install libncurses5-dev
apt-get install gawk
The following is a step in another direction as the 'make' command works perfectly after extraction, but this is provided for posterity:
wget http://busybox.net/downloads/busybox-1.24.1.tar.bz2
tar -xjf busybox-1.24.1.tar.bz2
cd busybox-1.24.1/
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- defconfig
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- menuconfig
At the menu, you can configure BusyBox options. Once configured, you can build BusyBox:
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-
sqlite3 - inherited - https://github.com/openand-inc/sqlite
waut.ch - 'none', non legally binding, and non legally relevant on a worldwide scale and irrevocable ( derived works are allowed to add their own licenses as long as the import ( meaning ) of the phrase "non legally binding and non legally relevant" is implied throughout ( not required to include text at all )) and does not reflect upon future updates of this software in any manner adversely.
He is back! Welcome back! Looking forward to this great new project! The only thing I might be missing is the dnsmasq filtering, which in my opinion if by far better solution than any kid of firewall/blocker
Sent from my Galaxy Tab 2 3G using Tapatalk
qWantUS said:
He is back! Welcome back! Looking forward to this great new project! The only thing I might be missing is the dnsmasq filtering, which in my opinion if by far better solution than any kid of firewall/blocker
Sent from my Galaxy Tab 2 3G using Tapatalk
Click to expand...
Click to collapse
Mate!
There are some serious issues with DNSMASQ.
- It crashes on wrong syntax of any option. Cannot do for a server.
- Command line syntax changes between original branch, Android and Cyanogen MOD. Not merged.
- Android fork far behind main branch
- Consumes port 53 on server run without sharing interfaces. This is on the Android branch. Main branch has this resolved.
- CPU loop for most Android DNS versions that show up only upon server run, rather than the tethering run. Possibly hijacked open source branch.
- Tethering modifications to Android branch ( command line addons ) not required any more as the main branch has incorporated methods to dynamically change IP addresses on the fly.
- It is probably encumbered by coding standards and export issues with only one developer. It is emblematic of most such over-reused projects.
- Hash table in memory, therefore future block list will use up RAM. Some others have disk based caching.
- Static blocklists are fairly not scalable for 10 years hence. Needs wildcard blocklists.
- DNS cache poisoning/overloading ( leading to denial of service) possible by any process/app.
It is therefore recommended to use either of:
pdnsd
unbound
djbdns
and standalone dhcp clients and servers from the Entware repository. And also these are interesting:
DANE
DNSSEC over TLS
BTW, all Android is encumbered by simple denial of service attack simply by reading from /dev/random by any app.
In order to try and package this into a simplish product, deleting /dev/random was not implemented.
idcrisis said:
Mate!
There are some serious issues with DNSMASQ.
- It crashes on wrong syntax of any option. Cannot do for a server.
- Command line syntax changes between original branch, Android and Cyanogen MOD. Not merged.
- Android fork far behind main branch
- Consumes port 53 on server run without sharing interfaces. This is on the Android branch. Main branch has this resolved.
- CPU loop for most Android DNS versions that show up only upon server run, rather than the tethering run. Possibly hijacked open source branch.
- Tethering modifications to Android branch ( command line addons ) not required any more as the main branch has incorporated methods to dynamically change IP addresses on the fly.
- It is probably encumbered by coding standards and export issues with only one developer. It is emblematic of most such over-reused projects.
- Hash table in memory, therefore future block list will use up RAM. Some others have disk based caching.
- Static blocklists are fairly not scalable for 10 years hence. Needs wildcard blocklists.
- DNS cache poisoning/overloading ( leading to denial of service) possible by any process/app.
It is therefore recommended to use either of:
pdnsd
unbound
djbdns
and standalone dhcp clients and servers from the Entware repository. And also these are interesting:
DANE
DNSSEC over TLS
BTW, all Android is encumbered by simple denial of service attack simply by reading from /dev/random by any app.
In order to try and package this into a simplish product, deleting /dev/random was not implemented.
Click to expand...
Click to collapse
Very comprehensive explanation indeed! I am no expert in Linux, only an average user [emoji6]
I was not aware of so many obstacles using dnasmasq, but aware of it's unreliability while using it for blocking stuff in pfsense. But I must say, I had similar experience with unbound, however, probably due to the lack of linux knowledge.
The way I understand is that host blocking is no longer viable due to scales of blocking required today. But sadly there is no alternative for that on android. I have been trying another tool, called sharkmasq but developer sadly abandoned it, while it seems very unfinished and unreliable. I also understand the reasons behind leaving netfiltering out of your development, just, in desperation, I search for the hope [emoji3]
P. S.
Thank you very much for your reply and for your time putting it together. Nevertheless I will look forward to further developments of this new exciting project!
Sent from my LG-D855 using Tapatalk
Uhm, my device seems more responsive even when I use a very low CPU frequency, but is it normal that the app doesn't ask for root permission even if I'm fully rooted? I'm using a Nexus 5 running Nougat. Thank you
Inviato dal mio Nexus 5 con Tapatalk 2
Hi, yes, part of the design criteria. Without root, the metric is interesting. With root, it will automatically start the background processes including the daily cleanups.
Best way to know that the background processes are running is to install Seeder ( but whatever you do don't start it! . Entropy should show around 4096.
Still in the testing phase, does it worth excluding the app from "android optimisation list"? I also use "power nap", not sure will that cause any side effects...
Sent from my LG-D855 using Tapatalk
Let us know If Seeder has RNGD off and still reports ~ 4096 entropy then all is hunky dory.
The new update asked me for root permission, all right here now
Have been busy lately with life, but had occasionally played with phone. So my findings so far: I use this app on my old galaxy tab 2 3g on slim6. Seeder shows entropy being filled in no time at all. Can't really say much about performance, but it seems that it behaves a little smoother. I also use it on my daily LG G3 on stock MM rom with custom kernel, but seeder is reporting very low entropy, if seeded is left on for 1 minute, I can see that entropy never goes above 900? Goes up slowly but then gets used, and then very slowly goes up again and gets used.
Sent from my LG-D855 using Tapatalk
Reboot once after first install kicks in the animation changes. And good luck
New version uploaded with haveged updates. Build scripts also added.
Feedback solicited about the compiler flags etc. Basically the idea is that the ARM v5 binaries should work for all past and future versions of 32-bit Android without modification.
UPX binary could only be compiled for x86 linux.
To answer question, second device, not running, may need to check su logs
Also backend binaries are 32 bit, may not work on 64 bit. GUI's ok.
I am very confused about the frontend of this app that looks like some number game that I really don't understand . Also, I don't understand the description very well. Finding the 8??? Reboot when animation kicks in? What animation?? Nothing seems to happen. I just see a still of a beach with a red number
But something DID happen though, so I am in business. I will keep you curious for a moment, while firstly giving you a good reason for a hard laugh:
I am trying to revive an LG L3 E400! Yep, that's right . Just for fun and educion. In fact I already succeeded quite a bit. I flashed JellyCast V7rev1, used some stuff from V6 SuperCharger (not entropy thingie coz I already intended to use CrossBreeder for that) and 3 scripts from Fly-On Mod. I came here via the CrossBreeder thread (of course). I have already tried CrossBreeder 7.2.13 (I followed advise from @f3tus here, he advised to use that older version). Entropy went up (so it worked) but I did not experience much performance improvement. I did experience more "not responding" issues though. So I reverted to the pre-CrossBreeder backup. And then came your app.....
So that's a very short summary of the many, many, many hours I spent last 2 weeks on this ancient minimalistic but very adorable phone. Now... back to this Waut.ch! app. Presuming that it had done something, I checked the entropy level after first reboot: still a jumpy number between 150 and 200 or so. Despite the fact that the service shows up as running. Then, a lot more playing with the "game" without understanding what I was doing. Just hoping for some wonder. Some message.... just...... something! But nothing. Just a beach with a red number. And a clock. And a questionmark. Or exclamation mark. Pffffff.. So.... let's reboot again. Then eat a sandwich and drink (more) coffee. Then check entropy again. And there it is! Steady at 4089! At first I thought his app was failing because the number didn't change anymore. Just steady. With the old CrossBreeder 7.2.13 I got 4096 a lot but it was a very jumpy number, going even below 1000 a lot. Well, some time passed while writing this post so let's check again: Wow, right now it says 4091 (99%)! One hour later: 4092! (It had spontaneous reboots inbetween).
I am testing it now for a few hours and have these problems:
"Not responding" issues (same as old CrossBreeder).
Spontaneous reboots (3 already). One of them failing to boot at all (stuck at JellyCast logo).
Two times, after a reboot, GPS was disabled. Not consistently though.
No noticable performance gain.
By the way, I am testing like real world usage. Just opening, using and closing apps. I use the same apps all the time so that gives a very good feeling about performance. More meaningful than benchmarks to me. My goal is to make this phone suitable for normal daily usage.
Your app is using a combination of serveral mods, right? I would like to be able to enable / disable each one separately. How can I do that? For example, I have a gut feeling that my "not responding" issues (and reboots?) do not come from the entropy mod but perhaps from governor tweaks or something else. Any chance that you will add enable buttons in the app? Or supply seperate scripts?
Update: I DO have performance gain! Not consitently though. Apps sometimes starting faster than ever. Also still the aforementioned quirks. Last hours no more auto-reboots. Entropy reaching 4096 now, just a little bit wobbly. There seems to be a lot of potential.
The stutter was an issue with the last version.
Do try the latest one and let us know.
Also better if you install the Play store also so the updates are automatic.
Version name is changed GUID if comparing. Better than comparing version numbers.
GUI is non functional, just a visual representation of haptic feedback and actual randomness spread.
It's like drawing points in the whites of a poached egg. If you can see a pattern, you'll be rich in the stock market
Today I removed the app. The biggest problem is random reboots for me. As said, it has potential because sometimes apps started faster than ever on my ancient LG L3. But not consistently. So maybe I 'll try again later.
Sorry for my first post, probably tl;dr. I will repeat my most important question: can you provide us with separate scripts? I mean: I would like to test the entropy thing without the other tweaks as you described in the OP.
Dude!
You're probably using an old version. We'll never know. Will we? Will we now?
There's a small trick that borders on superstition. The famous random device block gremlin tends to disappear if you run the GUI in scribble mode. Basically what may be happening is the block is hit and then the stupid kernel makes whatever decision it makes. Usually at the cost of usability in favour of some stupid religious Linux fervour. Someone should report that to LINUX.
Or another entropy generator, haveged or RNGD or Seeder may be running. Which is a big no! Best to run it on stock ROMS or full custom ROMs ( please ask the developer if they're running any variant of above and to recommend coexistence or better variant of any subcomponents or piece of code. ). In other words, open source!
Also, the separate scripts are in the /bin directory on the phone itself. May require bash and ADB skills.
And full source on XDA and GitHub.
I'll construe this as a possible request to also post the shell scripts from GitHub into the XDA project as separate attachments. Which I can do. In due course
Cheers.
Well, I took it from Play Store. Is that an old version?