Related
I was wondering if there is a method to cache DNS requests? I use CM latest and was browsing around in the system folder, when i came by /system/etc/resolv.conf. Here you find the nameservers used by android to resolve DNS.
This got me thinking, because I use a DNS cache on my Ubuntu Box. Might there be a way to do this on Android, to speed up browsing regularly visited websites? Does anyone have any idea how to use a local DNS cache to speed up browsing?
Here's the deal:
- In /system/etc/resolv.conf you find 4 nameservers (4.2.2.5 - 4.2.2.2)
- Doing a ping test (count = 1000) to those, i find big differences in latency. Average was 800 on my test on 3G (I live in the Netherlands). I found this number rather large.
- On IRC, when asking this question, they mentioned porting dnsmasq to android and change nameserver to localhost
A ported version of dnsmasq is found here: http://code.google.com/p/android-wifi-tether/source/browse/trunk/res/raw/dnsmasq?spec=svn120&r=120
Copied the file to /system/bin, chmod +x the file, and this one works.
Now here's the thing:
- dnsmasq --help mentions a configurationfile: /data/local/dnsmasq.conf. This one have to be made for android
- how to configure dnsmasq so it uses the DNS servers and create a cache, thus speeding up browsing (in my humble belief)
- how to start dnsmasq on booting android (and making sure it stays alive)
And finally, does it make sense to use this kind of method? Like to apply for a think tank to make this work
That does sound like a good idea. For me it isn't just 800ms to find out the DNS but on edge rather something like about 3 seconds! this would seriously speed the whole thing up! However, how are you gonna find the IP adresses for the first time?
Well dnsmasq creates a cache, using nameservers found in the config file, if i am correct. I am looking into the dnsmasq.conf examples to see what workaround i need.. Should be pretty straightforward
resolv.conf should have nameserver 127.0.0.1, and dnsmasq should use configured nameservers like 4.2.2.5 and create a cache. Therefore apps use the dnsmasq as DNS server.
Furthermore, i am trying to find out what nameservers are the fastest. 4.2.2.2-5 are Verizon, while i am in the Netherlands. OpenDNS has a datacenter in Amsterdam, so i figured changing nameservers to OpenDNS would speed things up some more..
Even better would be unbound, though the stated overhead of ~11 MB might be too much for G1/mT3G v1.
Also, do you have to do any tricks to prevent resolv.conf from being overwritten when you switch networks? I know that usually dhclient will rewrite the file, and there's a couple of workarounds, but I have no idea which ones would work on Android.
CM builds already have a dnsmasq binary in /system/bin. It's used for tethering. You could launch it with a different config file if needed (there's a dnsmasq.conf in /system/etc already for tethering). Also, the values in resolv.conf aren't really used by much anymore unless you have stuff linked with uclibc. The resolver in Bionic uses the values of the net.dns* system properties.
cyanogen said:
CM builds already have a dnsmasq binary in /system/bin. It's used for tethering. You could launch it with a different config file if needed (there's a dnsmasq.conf in /system/etc already for tethering). Also, the values in resolv.conf aren't really used by much anymore unless you have stuff linked with uclibc. The resolver in Bionic uses the values of the net.dns* system properties.
Click to expand...
Click to collapse
Thanks for the reaction... You are right indeed, i found the dnsmasq.conf:
Code:
no-resolv
no-poll
server=4.2.2.2
server=4.2.2.3
interface=usb0
dhcp-authoritative
dhcp-range=192.168.77.100,192.168.77.105,12h
dhcp-leasefile=/data/misc/dnsmasq.leases
pid-file=/data/misc/dnsmasq.pid
user=dhcp
no-negcache
What you're saying is that i should create another one, but i am wondering what values i should use..
interface=?
Also, regarding your remark on DNS, how to see the values of net.dns* and how to change them? sysctl -n doesn't show these values, I must be looking in the wrong direction...
Hey,
You don't have to set an interface at all.
The interface delcaretion limit dnsmasq to listen only on a specific interface (for both DHCP and DNS requests).
As far as I know dnsmasq is caching dns by default you can limit the cache size and set not to cache negative queries but it will cache by default so no special settings is needed.
In addition, Dnsmasq is also function as dhcp server and if you don't want it to try removing all lines that declaring on dhcp settings.
True on that one, i want to use dnsmasq mainly on 3G
On my remark about the dns properties, already found out to use getprop and setprop
getprop shows different values on DNS compared to resolv.conf:
[net.dns1.195]: [84.241.226.140]
[net.dns2.195]: [84.241.226.9]
[net.dns1]: [84.241.226.140]
[net.dns2]: [84.241.226.9]
You need to be aware of one thing with regards to the resolv.conf file.
It is NOT USED BY ANDROID.
If you use the command "getprop", you will see several dns entries in there -- none of which matches the static dns servers set in resolv.conf. The resolv.conf is used by TERMINAL BINARIES.
zrubi said:
Hey,
You don't have to set an interface at all.
The interface delcaretion limit dnsmasq to listen only on a specific interface (for both DHCP and DNS requests).
As far as I know dnsmasq is caching dns by default you can limit the cache size and set not to cache negative queries but it will cache by default so no special settings is needed.
In addition, Dnsmasq is also function as dhcp server and if you don't want it to try removing all lines that declaring on dhcp settings.
Click to expand...
Click to collapse
might as well restrict it to the local interface
Code:
interface=lo
you can then turn off dhcp with
Code:
no-dhcp-interface=lo
I use OpenDNS on my home network which is claimed to be better than ISP dns servers. When you signup/setup you are required to input their values for dns servers, perhaps you set these values from your phone?
Not sure if Android supports this... but the easiest and least resource intensive way to do this would be to simply add your commonly used domains to the hosts file. I believe some ad blocking software used to use this to block ads.
Aside from that... Bind can be run as a caching nameserver and would probably work in conjunction with setprop to overwrite the nameserver values android tries to use...
Another interesting idea might be to run a squid cache with data stored on the SD card... this should theoretically be faster than pulling the pages over 3G, but could be memory constrained.
FWIW I did some testing with timing page loads over 3G/2G/Wifi... and I found that the bottleneck for page loads on the G1 was not in fact the speed of the network connection(3G and WiFi were virtually identical - 12Mbps cable, 2G slightly slower), but CPU time spent in rendering(well, possibly memory related as well, but the G1 is always memory constrained so its hard to tell). The biggest speed improvement I experienced was in turning off Javascript.
I also tested Stock vs Dolphin vs Steel and found Stock & Steel to have very similar numbers, with Steel having a slight edge sometimes. Dolphin was always orders of magnitude slower.
IMHO The only real way to speed up browsing on the G1 would be to utilize an external compression proxy to reduce the amount of data being sent across the network to the G1, or to rewrite the browser/Dalvik VM/whatever to be more efficient. I tried the external proxy method but couldn't seem to get it to work.
good thought...i'll tinker w/ this some
also dot folder
@equid0x Good thoughts... I used Opera on Android, which uses a compression proxy. Runs fast. Might be an idea to look into this for the native browser and abandon the idea to cache DNS.
cyanogen said:
CM builds already have a dnsmasq binary in /system/bin. It's used for tethering. You could launch it with a different config file if needed (there's a dnsmasq.conf in /system/etc already for tethering). Also, the values in resolv.conf aren't really used by much anymore unless you have stuff linked with uclibc. The resolver in Bionic uses the values of the net.dns* system properties.
Click to expand...
Click to collapse
I think I have dnsmasq playing nice w/ this config:
Code:
no-resolv
no-poll
server=4.2.2.2
server=4.2.2.3
listen-address=127.0.0.7
interface=lo
pid-file=/data/misc/dnsmasqcache.pid
user=dhcp
no-negcache
it's listening on UDP port 53 -- seems to be doing its thing..
you're right..resolv.conf doesn't affect anything I do.
but I don't see any entries like net.dns* when i
#sysctl net
What am I missing about how Bionic does things?
alapapa said:
but I don't see any entries like net.dns* when i
#sysctl net
Click to expand...
Click to collapse
Use:
# getprop
Brut.all said:
Use:
# getprop
Click to expand...
Click to collapse
Thanks.
i can:
# setprop net.dns1 127.0.0.1
# setprop net.dns2 127.0.0.1
and it uses dnsmasq for a while, then they change back to t-mobile's (seems like it happens when i change towers or it goes from 3g->edge or vice-versa
does net.dnschange have any ability to control this?
know where I can find any documentation on the dhcp process that populates these values?
edit: just tested again and the settings persisted all night. performance wasn't noticeably different than normal. I'll try to do some more scientific tests and report back..
was this project dropped? anything goin on here?
dnsmasq cpu problem
Hi Guys,
I've implemented the dnsmasq with my adhoc connection. My config file is:
no-resolv
no-poll
server=10.50.30.254
listen-address=127.0.0.1
interface=lo
pid-file=/data/misc/dnsmasqcache.pid
user=dhcp
no-negcache
addn-hosts=/data/misc/dnsmasq-host
cache-size=65536
local-ttl=86400
where the dnsmasq-host file is a copy of http://www.mvps.org/winhelp2002/hosts.txt to avoid advertising site.
After few minutes I run dnsmasq I get the cpu at 100% and the process sayd:
dnsmasq: Unknown cmd ''
dnsmasq: Unknown cmd ''
dnsmasq: Unknown cmd ''
I've tried only with few config options but the result is the same.
Do you know what is that?
Hi,
This is a potential game changer, DNS server for Android, the open source, DNSMasq and some specific config files for it.
It runs on your Android device and you point your DNS queries to it instead of your remote ISP DNS server.
Using this setup will speed up your Android online experience many fold because DNS (specially over 3G) is one of the slowest blocking components of the entire browsing activity.
Features/Points to note -
- Caching, multithreaded. The replies from the cache are instantenous, hence browsing speed is enhanced significantly
- Query multiple DNS servers simultaneously. Fastest reply is used. Hence redundancy is also achieved along with speed. Currently set to Google DNS servers, Open DNS servers and your home Wifi router. Best not to use more than 5.
- Works with all versions of Android from 1.5 -4.x and all Android phones and tablets
- You can add your ISP's (both 3g and home) DNS servers to /data/local/dnsmasq.conf.
- Don't use your ISP's servers (and remove your Wifi router) if anonymity is a concern.
- Can help circumvent DNS restrictions.
- If your home router has DNS and an IP other than 192.168.1.1, you can change the entry in /data/local/dnsmasq.conf
- It may not start automatically if your ROM doesn't have init.d support. In that case and in either case, you can download SManager and set /etc/init.d/97dns to 'su' (run as root), 'boot' (run on boot) and 'net' ( run on network change)
- To make it stick all the more, you can download SetDNS and enable it and set the DNS server to 'Custom' -> 127.0.0.1 primary and 8.8.4.4 secondary ( backup Google DNS in case DNSMasq is not running for some reason). Make sure that you have Wifi and 3G options checked.
- Will significantly speed up your tethering experience if you set your DNS server to the Android phone and to use this DNS server. Currently the dhcpd option is disabled. You can either enable it or tell your tethering DHCP software to send DNS server=192.168.x.x ( or your Android phone IP) to your PC.
- Block Ads with the MVPS HOSTS file located at /data/local/dnsmasq-host
- VPN connections with and without split tunneling are supported. Check out commented sections in config files. Can use different DNS servers for different domains.
- You need to use this version of DNSMasq. Others ( eg those distributed with most ROMs) have a runaway CPU problem or don't work without the -d (debug) option. Any help resolving those issues welcome.
- You'll need root and su. busybox optional. Simply run Install.bat to install it and configure it automatically. Then make it stick with SManager and SetDNS as mentioned above.
- It's not mine, I just found the correct version and configured it.
- Enjoy! Please click the Thanks button if this helps you.
25th May - Fixed bug where Wifi DHCP wasn't working
Any idea why this breaks my native tether on an og evo 4g on team dirt cm9 port?
Is it necessary to use the install.bat from a pc to make this work or can I just add the dns script to the init.d folder?
Firstly, thank you.
I am trying to manually setup this so I can use some dns redirection (using the --server option) but when I run the install.bat file, it breaks tethering (which uses dnsmasq as well). Would you please help me out understanding what steps the script did to get it running?
Hi,
I think that this version of DNSMASQ may need some different startup options to support dhcp. I have specifically disabled dhcp in the config file.
Or it (dhcp) simply may be compiled out of the binary and it currently may not be usable for both DNS and DHCP.
YMMV. BTW the install script backs up the dnsmasq binary so you can replace it with the backup in case of any issues.
Cheers.
I will try this one if it'll bring any good to my connection. Thanks.
There is a particular web site I was not able to connect to from my android devices but connected just fine from my windows PCs/laptops.
After some playing around with wireshark and Shark for Root, I saaw the server never answered the SYN from the android clients. Comparing the SYN packets between the windows 7 and android stacks, the only differences were in the options, and I was able to determine that either the server, or more likely a router/firewall in the path is choking on the TCP Timestamps option (option #8)
I if I temporarily disable the timestamps option echo 0 > /proc/sys/net/ipv4/tcp_timestamps, the android device connects just fine.
Once I identified the problem I found at least one thread showing that some servers still don't handle this 20 year old RFC properly (see http://serverfault.com/questions/23...-a-syn-ack-packet-in-response-to-a-syn-packet, I didn't have to muck with windows scaling in my case).
writing to /proc/sys is only a temporary solution though. it gets cleared when the device reboots.
Under linux, I would modify /etc/sysctl.conf to make the change permanent, but it doesn't look like android uses the sysctl.conf system (at least it's not present on any of my devices, custom ROMs or stock)
what's the proper way to disable the Timestamps option permanently?
I run SmoothROM on my nexus 7 and it has support for init.d, so at least one workaround was to create a /data/lcoal/userinit.sh that writes 0 to /proc/sys/net/ipv4/tcp_timestamps at every boot.
ParanoidAndoird on my Galaxy Nexus seems to also support init.d (edit: yup it does)
My main phone, a Nexus 4, is running 4.2.2 stock (albeit rooted) and doesn't seem to support init.d....
looks like installing busybox will install sysctl, so it might be an avenue to explore, although it's getting too late for me to try today.
I also saw mention somewhere of the later versions of busybox implementing init.d?
install-recovery.sh appears to be run at each boot, so I can add my tweak to that file on my rooted Nexus 4, or I could implemnt the full init.d solution described in this thread, although it's overkill for me: http://forum.xda-developers.com/showthread.php?t=1933849
Bottom line is maybe it's klugy, maybe it's the only way, but I got the tmestamps option disabled on all my android devices and can now connect flawlessly.
Hello XDA Developers, I have a Debian subsystem of sorts on my phone which is created by an application called Lil' Debi. For those of you unfamiliar with it, it essentially creates a Debian install on an .iso that can be mounted onto the disk. Once mounted, a user can access a shell to interact with this Debian subsystem by running /debian/shell as root, which will chroot to its own directory system separate system accessible from the Android Terminal.
Within this Debian subsystem I have created a non-root user account for the purpose of running a few networking applications that if compromised for some reason, won't give the attacker root privileges to break everything on my phone. There's only one small problem with this setup: I can't access the internet from a non-root account.
Both my terminal emulator and Lil' Debi have full network access, even when not run as root. I am curious then, why a non-root user account should have an incapability of accessing the network. A sample of wget on my phone using Google's IP address (I use the IP address because it cannot do DNS lookup obviously) gives a Permission Denied error. At the current moment I am not sure whether this problem lies with Android or with Debian. Does the user need to be explicitly granted permissions to use the network through Debian, or is the application somehow only able to access the network if it's root?
Additional information: The ROM used is PAC ROM, so you can assume any settings changes that could be made from Cyanogenmod or Paranoid Android can be made if necessary. The phone itself is a Oneplus One. No I don't have invites, so don't bother asking.
Opinions on the matter?
Also, on an unrelated note, g++ will only run under root. If I launch it as a non-root user, it will tell me that execvp failed because cc1plus doesn't exist. Why?
Thread's fallen onto the third page, so I'm going to bump.
One day has passed, and no help offered. Bump again.
Another bump. I thought XDA was supposed to be the most knowledgeable forum on Android.
Daily bump until this problem is solved...
Still bumping...
I hope people aren't just looking at the number of replies and assuming it's resolved...
Bumping again. At least 100 people have seen this thread, and not a single one has anything to say.
Bump again. It's now been a week since I asked this question.
Bump.
Hi, i'm very interested in privacy and security but I'm a complete noob when it comes to android and phones so hopefully i can learn something from this forum.
I currently own a samsung A51, unmodified. I'm unsure if i should root it.
I have recently read that samsung collects a lot of data and sells it to third parties, google and apple collects data just as well but they don't sell data to third parties. I'm worried about this.
My win10 machine has been hardened pretty well, it doesn't call home to microsoft in any way i know. Simply by setting the rules to "deny by default" unless something has specifically been whitelisted by, and blacklisting microsoft IP's.
Can this be done on a stock samsung phone as well? Is there any firewall app that can do this? Something like tinywall? Or IPtables in linux? to prevent any data connection from samsung?
Read my post, depending on who wants access to your phone, there isnt anything you can do. With permissions of most apps any by default most operating systems have backdoors within them, and with your advertising i.d u can be followed from website to website and tracked just by having wifi turned on. ..
E.g owning a samsung ssd, in their privacy statement they tell you they are gonna identify you from your ssd serial number and use it the same way the telemnttry u are blocking does, so unless u wanna block and change every device i.d u got...
Long story we all fuuuuucd
Windows OS has a hosts file, Linux OS has a host file, MacOS has an hosts file and Android OS has a hosts file, too. That's the place people use to block unwanted Internet connections.
jwoegerbauer said:
Windows OS has a hosts file, Linux OS has a host file, MacOS has an hosts file and Android OS has a hosts file, too. That's the place people use to block unwanted Internet connections.
Click to expand...
Click to collapse
Yeah i get that but that dont work with my issue as the access to my devices goes around the layer of the operating system