Related
Hi,
I have looked around and cant seem to find an application quite fitting.
All I want is an app that will wirelessly sync just a folder automatically when I have my WiFi turned on, is there an app that can do this?
i.e. Come home from work, turn on wifi and it just syncs.
Many thanks
To any developers about: How hard would something like this be to make?
I think dropbox does that?
Dropbox syncs a folder on multiple devices with a folder on dropbox server.
There is also an app called PCFileSync that syncs a folder over wifi between the phone and a samba share. I am turning it on with Tasker once it connects to my home ssid. Works great!
Sent from my Desire HD using XDA App
samba share? hm........
rsync and ssh would be a much better choice and cleaner, just a script to run.
synflex said:
samba share? hm........
rsync and ssh would be a much better choice and cleaner, just a script to run.
Click to expand...
Click to collapse
http://a-more-common-hades.blogspot.com/2010/07/backup-photos-with-rsync.html
Tasker+Rsync, and combine it with a ssh key and you can automate it easy.
BTW, I decided to do this today.. I have one which backs up /sdcard/DCIM, and another which backs up the entire sdcard, both over wifi to my Fedora Linux workstation. I'll have to figure out how I want to execute these, but I can do it from GScript Lite, Tasker, whatever floats my boat.
Here's the notes I took, just in case I have to do it again in the future, hope it's useful to some folks..
# Get dropbear/dropbearkey from http://www.elkins.org/dropbear.tar.gz
# Get ssh off device and follow instructions (references at the end) to fix it, name fixed file "sshfxed"
# Remount /system rw
~/AndroidSDK/tools/adb shell
su
mount -o rw,remount device /system
#exit back to Linux shell
Click to expand...
Click to collapse
# Install dropbear and sshfixed from adb push
~/AndroidSDK/tools/adb push sshfixed /system/xbin/sshfixed
~/AndroidSDK/tools/adb push dropbear /system/xbin/dropbear
~/AndroidSDK/tools/adb push dropbearkey /system/xbin/dropbearkey
Click to expand...
Click to collapse
# Now get into adb shell and execute in Android..
mkdir /system/etc/dropbear
chmod 4755 /system/bin/dropbear
chmod 4755 /system/bin/dropbearkey
Click to expand...
Click to collapse
# Create keys
dropbearkey -t rsa -f /system/etc/dropbear/dropbear_rsa_host_key
dropbearkey -t dss -f /system/etc/dropbear/dropbear_dss_host_key
Click to expand...
Click to collapse
# Copy and paste the public DSS key the dss command above outputs and put it in ~/.ssh/authorized_keys on intended host. If you need to know more here, google for how to use ssh keys on Linux hosts. Basically if you're going to [email protected], on machine1 under user account "user" you create a file named authorized_keys in the .ssh directory with the output of the previous command, it tells you it's the 'public key', which is what you want to copy. It has to be all one line.
# Now actually use it all and rsync sdcard to your host For debugging, I suggest trying the "sshfixed -l USER -y -i /system/etc/dropbear/dropbear_dss_host_key" part just by itself and make sure you can ssh to your machine WITHOUT any prompting, password, etc.
~/AndroidSDK/tools/adb shell
# the following is all one line..
rsync -rltDv --chmod=u=rwX,g=rX,o=rX -e "sshfixed -l USER -y -i /system/etc/dropbear/dropbear_dss_host_key" /mnt/sdcard [email protected]:/your/path
Click to expand...
Click to collapse
-- References
Get dropbear - http://forum.xda-developers.com/showthread.php?p=8220181
Set up dropbear - http://forum.xda-developers.com/showthread.php?t=442754
Fix ssh client to use urandom - https://www.heiher.info/1592.html
Use it all to rsync over wifi - http://a-more-common-hades.blogspot.com/2010/07/backup-photos-with-rsync.html
Another vote for Dropbox, it works great! Look for referral codes in the comments on Market for some additional free storage. I use it all the time for pictures and vids for the family.
Sent from my HTC Desire using the XDA App
I use S&K sync.
Horrible user interface but once things are set up you need only to start the client on your phone and press the "Start Sync" button.
Supports multiple configuration sets (local/remote directory pairs, ip address, etc), each can be started separately.
Google: "SK sync android"
khaytsus said:
http://a-more-common-hades.blogspot.com/2010/07/backup-photos-with-rsync.html
Tasker+Rsync, and combine it with a ssh key and you can automate it easy.
Click to expand...
Click to collapse
I'm actually syncing to 2 android phones, with dropbox, rsync and dropbear.
Dropbox is good for documents and such, with small filesize and possiblity of sharing with others.
Using rsync for large files like movies and music folders, also some pictures, notes, keepass, and stuffs I like having around, things that's only going around between my laptop and phone.
Running a crontab that scans for my phone in adb devices, if found, pull a "last sync status" file, if interval > 1 hour or changes made locally, do a local notify-send till rsync is done, then execute dropbear on phone, rsync remote and local, kills dropbear, then push last sync file.
This way, no resources is wasted on the phone, all scanning and verification of device, system, and files were done on laptop.
At the same time I'm monitoring my development phone with frequent ssh rsync when it's in range, push updates, kernels and such across in proper folder structure.
This will ensure changes or new kernels were pushed to phone ready for testing.
Could use the same script for both phone, be it adb+ssh+rsync or pure ssh+rsync, but I personally don't like having sshd running in background when I'm connected to wifi hotspot.
Cleaner this way, less resources wasted on phone, and I can have full control over what, where, and when to sync.
I do realise that there isn't much choice for syncronization in the wild, thus you may like yo try something along the same idea, and you may get your own perfect combo to sync.
P.S. backing up, restoring or syncing bookmarks from browser.db and such could be done this way.
I just wanted you to inform you about my first Android application: rsync backup for Android. It is available in Market (for free, ad-supported).
Comments, suggestions, feature requests and bug reports are welcomed.
Just curious about your instructions.... You mention generating keys multiple times, but not sure why?
Typically to use ssh keys, one must generate keys on the client, transfer the public key of the client to the server. Then the client can connect into the server using the certificate rather than passphrase authentication.
Your instructions confused me.. It could be due to not having coffee yet, but might want to streamline the info a bit or users might never get to actually using it.
I'll try it myself later, as I already do this but curious how you've done it on your end. Here's what I run manually about once a week from GScript:
rsync -rltDv --delete --chmod=u=rwX,g=rX,o=rX --exclude=".android_secure" --exclude "Music" -e "sshfixed -l user -y -i /sdcard/bin/etc/dropbear/dropbear_dss_host_key" /mnt/sdcard [email protected]:/media3/android/microsd/rsync-backup/hourly.0/
Click to expand...
Click to collapse
And the server itself does a hard-link copy and move of directory names etc which maintains a set of 4 snapshots, I also do the same thing for my backups on the machines themselves.. Handy to find stuff I just realized I hosed a month ago..
Biggest annoyance was having to dupliate ssh (dropbear) and fix it so it uses /dev/urandom, as /dev/random isn't a viable source for ssh.
These are three different ways of generating private+public key pair. I'll add some description, because it could be confusing, as you said.
_sammael_ said:
These are three different ways of generating private+public key pair. I'll add some description, because it could be confusing, as you said.
Click to expand...
Click to collapse
Ah, yes that makes more sense, give them options on how to generate the keys.. However, you can only create keys on the client itself, one mention seemed like it was referring to using a Linux box to create them?
You can always generate keys on Linux box, then transfer private key to your Android and append public key to authorized_keys.
_sammael_ said:
You can always generate keys on Linux box, then transfer private key to your Android and append public key to authorized_keys.
Click to expand...
Click to collapse
To connect from Linux to Android... Yes. But pretty sure that won't work for Android to Linux, which is what I'd think you'd normally do?
He's saying, move both keys. Generally the private key is left on the system that generated it, but that doesn't need to be the case. At least as far as I know.
First off, brilliant app! It's exactly what I have been looking for to save me a huge amount of time and effort. Currently I was doing this by hand because I've had bad experiences with sdcards suddenly just stop working meaning I loose everything.
I have set a range of profiles up and had Tasker kick them off daily however I am at a loss how to get --exclude-from '/sdcard/excludefile.txt' working. When ever I run a profile it states:
rsync: failed to open exclude file '/sdcard/excludefile.txt' : No such file or directory (2)
rsync error : error in file IO (code 11) at exclude.c(1062) [client=3.0.6]
Click to expand...
Click to collapse
What I'm wanting to do is have the following profiles
/ -> /home/user/mobile/dellstreak/
[exclude sdcard exclude*]
/sdcard -> /home/user/mobile/dellstreak/sdcard
[exclude DCIM audio ebooks video media rings downloads temp exclude*]
/sdcard/DCIM -> /home/user/mobile/dellstreak/sdcard/DCIM
Click to expand...
Click to collapse
You can see what I'm wanting to do, I want a little more control over some profiles i.e. so I can backup DCIM whilst out and about or back up the sdcard without backing up all my music, videos etc.
Any help would be appreciated.
Try filename without the quotes:
--exclude-from /sdcard/excludefile.txt
Currently custom parameters parser takes arguments literally. Splitting is done using space as a delimited. That's why spaces in filenames won't work and rsync think that your filename is '/sdcard/excludefile.txt' (file excludefile.txt' in directory '/sdcard) instead of /sdcard/excludefile.txt
khaytsus said:
To connect from Linux to Android... Yes. But pretty sure that won't work for Android to Linux, which is what I'd think you'd normally do?
Click to expand...
Click to collapse
Yes, it will work. If you don't believe me, check it for yourself. In fact I initially generated my private key on my Linux box (using dropbearkey), then tested method with dropbearconvert.
Please notice that what you're generating is key pair, which means that it will work as long as you put public key in remote side's authorized_keys (no matter what system it is and where it was generated) and authenticate using corresponding private key.
Been using it for weeks now, I love it.
You should document a few things, such as how to create a dropbear key from a standard key.
Confirmation number: 9X233376XY8982101.
It is documented on application's webpage (see Menu -> Help).
_sammael_ said:
It is documented on application's webpage (see Menu -> Help).
Click to expand...
Click to collapse
So it is. I'm not sure if that was there when I first started using, but you've added the things I would have suggested (being able to change the command line options, etc.)
_sammael_ said:
Try filename without the quotes:
--exclude-from /sdcard/excludefile.txt
Currently custom parameters parser takes arguments literally. Splitting is done using space as a delimited. That's why spaces in filenames won't work and rsync think that your filename is '/sdcard/excludefile.txt' (file excludefile.txt' in directory '/sdcard) instead of /sdcard/excludefile.txt
Click to expand...
Click to collapse
The exclude files I'm parsing don't have spaces in anyway so luckily I didn't have to worry about that (coming from a linux background has taught me to avoid spaces where ever possible, not to mention keep everything lower case )
Your suggestion of removing the quotes (') seems to have done it, I use rsync on my linux boxes which require you to use quotes when parsing an exclude file.
Will check that it's parsing the exclude files correctly and will report back.
As per documentation that someone has suggested, I found the current documentation to work find, however it seems a little sparse. Maybe creating a wiki site for the documentation might help with this?
Again thankyou for a great app.
The exclude list seems to be working as expected, I now have it syncing as follows
/ -> /home/user/mobile/dellstreak/sdcard
/sdcard -> /home/user/mobile/dellstreak/sdcard
/sdcard/DCIM -> /home/user/mobile/dellstreak/sdcard/DCIM
Click to expand...
Click to collapse
I've also significantly increased my excludes to cover directories that have permission issues or that are (re)created on boot.
##rsyncrootexclude##
/sdcard
/acct
/cache
/config
/d
/mnt
/proc
/dev
/sys
Click to expand...
Click to collapse
##rsyncsdcardexclude##
/sdcard/Update*.zip
/sdcard/update*.zip
/sdcard/video
/sdcard/audio
/sdcard/ebooks
/sdcard/download
/sdcard/media/music
/sdcard/music
/sdcard/xuluan.podcast
Click to expand...
Click to collapse
So one suggestion, is it possible to add a scheduler to this?
I use Titanium Backup to make backups of my apps/data daily and would like that pushed automatically to my rsync server.
I didn't plan to add this, you can use Tasker or Locale.
As _sammael_ says, tasker works great with it.
I get it to automagically rsync when ever I connect to my wifi, and then on a timed scheduled.
ssh required?
great app.. like others, I was using rsync manually.
Do you only support rsync over ssh? I currently use the (insecure) rsync protocol to sync a "TV" folder on my Windows Media Center box. I can install a SSH server, it just seems overkill for my needs.
Robert
Currently only rsync over ssh is supported, but I guess I can add selection whether user want to use ssh or not. Am I thinking correctly that it will be as simple as skipping rsync's -e option?
I have SSHFS working from a Debian chroot. I have a SGS2 (SGH-i777) with CyanogenMod 9 (2012-06-12 nightly), and everything is working as the root user. I can read the SSHFS mount and the files it contains. My problem is that a normal, non root user cannot read the SSHFS mount. The UID/GID of the mount is 1000/1000, which is the UID/GID of the user that owns the source of the mount (my primary workstation user).
I'm trying to use KeePassDroid to read my KeePass database in place, so any changes I make get immediately reflected in my master file, no need to replicate the file. If I could grant KeePassDroid sudo privileges, I think that would be good enough for me. But I'm stuck. I don't know enough about how users are managed on Android, and I don't know how to give superuser privileges to an arbitrary application (it doesn't appear that the Superuser app that shipped with my root ROM allows for that). Any ideas?
SSHFS... almost got it working... a little closer now....
I've finally gotten a chance to look at this further. It looks like sshfs (the mount utility) has the capability of using an arbitrary UID for the owner of the mount. So, I looked in /data/system/packages.list to determine the UID of the KeePassDroid application. Seems the UID for that app is 10079 on my system. But that doesn't work, because if I log into a shell as that user ("su - 10079"), the keepass directory doesn't even show up in an ls directory listing. I've also tried 10080, which is apparently the UID of the OpenIntents file browser used by KeePassDroid to select the KeePass database. It obviously didn't work.
My next try will be to set the entire chroot to be owned by the 10079 user. I'll post back here if I find success.
[SOLVED]SSFS... got it working...
I finally got it to work, had to pass the -o allow_other option to sshfs, and now my KeePassDroid app can read my database. Next task is to have it load the chroot and sundry stuff automatically, so I have access to my KeePass database wherever I'm at.
Hopefully this information will be useful to the next schmo who tries this. I can already see that this thread shows up on Google.
This probably sounds like a total noob question, but its legit. I've become very comfortable with the Linux File Structure over the years as they are self explanatory.
/home - where home folders to users exist.
/root - OK, Don't know why it wasn't put in home, but understandable to what it is
/etc - config files go here
/lib - library files go here
/sbin - superuser binaries/executables go here.
/bin - binaries/executables
/usr - user shared files
/var - variable files such as logs?
I'm sure we get the gist of it, but why did android not keep this structure? Does android's structure better suite it? To me it just seems like crap is lying everywhere with barely any forethought as to where it should be placed. Could someone explain the file structure to me?
Android wannabe dev,
Darksurf
Sent from my ASUS Transformer Pad TF700T using xda premium
I don't think there is so much difference if you have a better look at it. Most of it is just in /system, for the same reason (one of them, actually) you were wondering why /root isn't in /home under Linux - you can have different partitions at these paths (security, storage/quota management, etc.). And as there are no user accounts, there is no need for /home.
Plus, there is need for more modules that are loaded and unloaded, there's different apps cache management under Android, these things have to be in separate places.
Having said that, I sometimes miss simple /var/log/ stuff and things like that by default too (you forgot /dev, btw). I would like the /system partition to be more Linux-like in case of directories, but I like how /data and /sdcard are handled, it's pretty clear and easy to manage.
IMHO the main difference is the division into /system and /data (+ /sdcard or other removables mount points in the case of the Infinity), which, as you've probably noticed, has its very practical purposes and have to be clearly distinguished, especially on a non-rooted device, where the permissions are clearly set (but comes helpful when rooted, so you won't mess up too much by a missclick ;>).
There are a lot of howtos about pulling/pushing content over wifi around. Taking adb or AirDroid may be reliable for common use, but i was looking for a real two-way-sync without utilizing a cloud, also i wanted not too much 3rd Party-Software involved, neither on my box nor at my phone.
Although Unison is platform-independant and can be run from Windows, i'm focussing on the Linux part (coz i've got no Windows box to test). I won't go deeper into things like howto install the required applications on your box. You won't have to compile anything, precompiled binaries should be available for your distro. Guess pacman, yum and apt-get will do their job, but if in doubt, Google will lead the way.
Requirements
A rooted device
SSHDroid installed on your Android
OpenSSH, SSHFS and Unison properly installed on your Box
SSH
After ensured the requirements above, you should be able to ssh to your android device. It's ip is shown by SSHDroid.
Code:
ssh [email protected]<yourdeviceip>
First connect will take some time, coz ssh will handshake your device and save the fingerprint. After the connect succeeded, you will be easily able to connect your device anytime over wifi.
SSHFS
SSHFS allows us to mount the whole android device to a mountpoint on your box. Compare it like a external harddisk you plug into your box usb.
First we'll create a directory in our home and then mount the device into it.
Code:
mkdir wifidroid && sshfs [email protected]<your device ip>:/sdcard/ ~/wifidroid/
SSHFS will ask for your root password anytime you try to connect. If not configured different within SSHDroid, it will be admin. You're ought to change it for safety reasons. After above steps, your device is mounted into wifidroid. You can easily browse the contents using your preferred linux filemanager (like dolphin or nautilus). To unmount device, use fusermount -u ~/wifidroid/ later.
UNISON
Unison will handle the syncing between your box and your device. It comes with an cli and even with a gui. Whatever you will use is up to your personal preference.
First of all we're going to create the unison profile directory and a basic example profile for syncing pictures.
Code:
mkdir ~/.unison && touch pictures.prf
Open pictures.prf with your preferred editor (nano, vi, kate, gedit, orwutevah) and write down some config stuff.
Code:
root=/home/<username>/Pictures
root=/home/<username>/wifidroid/Pictures
path=Friends
path=Landscape
path=Misc Stuff
perms=0
ignore=Name .*~
ignore=Name *~
ignore=Name .*
The first root=/home/<username>/Pictures points to your local pictures directory, whereas the second one, root=/home/<username>/wifidroid/Pictures points to the pictures-folder at your mounted android device. Unison syncs recursevly, so it would simply sync anything beneath the directory Pictures. If you want to sync only some Subfolders, use path=Friends i. e. to include Pictures/Friends. You may edit this to your own convince. We don't want unison to set permissions on synced files, so we set perms=0. Rest should be self explanatory.
Okay, now just fire up unison with unison pictures for cli-version, or unison-gtk pictures for the gui-version and syncing should begin.
Keep in mind that, depending on your wifi speed, it will take a huge amount of time to sync a whole mp3/picture library for the first time. After first sync things should be butter smooth
Above example is just a basic profile. You can easily use unison to backup your whole device, or, as i do, make regular backups of your Camera-Roll, WhatsApp and Threema Backups. If you wanna go deeper into this, you should take a look at the Unison Reference Guide.
Genericxx said:
There are a lot of howtos about pulling/pushing content over wifi around. Taking adb or AirDroid may be reliable for common use, but i was looking for a real two-way-sync without utilizing a cloud, also i wanted not too much 3rd Party-Software involved, neither on my box nor at my phone.
Although Unison is platform-independant and can be run from Windows, i'm focussing on the Linux part (coz i've got no Windows box to test). I won't go deeper into things like howto install the required applications on your box. You won't have to compile anything, precompiled binaries should be available for your distro. Guess pacman, yum and apt-get will do their job, but if in doubt, Google will lead the way.
Requirements
A rooted device
SSHDroid installed on your Android
OpenSSH, SSHFS and Unison properly installed on your Box
SSH
After ensured the requirements above, you should be able to ssh to your android device. It's ip is shown by SSHDroid.
Code:
ssh [email protected]<yourdeviceip>
First connect will take some time, coz ssh will handshake your device and save the fingerprint. After the connect succeeded, you will be easily able to connect your device anytime over wifi.
SSHFS
SSHFS allows us to mount the whole android device to a mountpoint on your box. Compare it like a external harddisk you plug into your box usb.
First we'll create a directory in our home and then mount the device into it.
Code:
mkdir wifidroid && sshfs [email protected]<your device ip>:/sdcard/ ~/wifidroid/
SSHFS will ask for your root password anytime you try to connect. If not configured different within SSHDroid, it will be admin. You're ought to change it for safety reasons. After above steps, your device is mounted into wifidroid. You can easily browse the contents using your preferred linux filemanager (like dolphin or nautilus). To unmount device, use fusermount -u ~/wifidroid/ later.
UNISON
Unison will handle the syncing between your box and your device. It comes with an cli and even with a gui. Whatever you will use is up to your personal preference.
First of all we're going to create the unison profile directory and a basic example profile for syncing pictures.
Code:
mkdir ~/.unison && touch pictures.prf
Open pictures.prf with your preferred editor (nano, vi, kate, gedit, orwutevah) and write down some config stuff.
Code:
root=/home/<username>/Pictures
root=/home/<username>/wifidroid/Pictures
path=Friends
path=Landscape
path=Misc Stuff
perms=0
ignore=Name .*~
ignore=Name *~
ignore=Name .*
The first root=/home/<username>/Pictures points to your local pictures directory, whereas the second one, root=/home/<username>/wifidroid/Pictures points to the pictures-folder at your mounted android device. Unison syncs recursevly, so it would simply sync anything beneath the directory Pictures. If you want to sync only some Subfolders, use path=Friends i. e. to include Pictures/Friends. You may edit this to your own convince. We don't want unison to set permissions on synced files, so we set perms=0. Rest should be self explanatory.
Okay, now just fire up unison with unison pictures for cli-version, or unison-gtk pictures for the gui-version and syncing should begin.
Keep in mind that, depending on your wifi speed, it will take a huge amount of time to sync a whole mp3/picture library for the first time. After first sync things should be butter smooth
Above example is just a basic profile. You can easily use unison to backup your whole device, or, as i do, make regular backups of your Camera-Roll, WhatsApp and Threema Backups. If you wanna go deeper into this, you should take a look at the Unison Reference Guide.
Click to expand...
Click to collapse
Thanks will try
Nice job mate. I will try it.
Thanks @Genericxx, it looks complicated at first, but after reading a while it makes complete sense,
For anyone who wants an easier way for syncing between PC windows and Android, i'd recommend ES File Explorer and Goodsync (on windows PC)
First enable FTP Remote manager in ES through a WiFi same with ur PC windows. Then open Goodsync and setup a sync wizard and choose FTP and then the ip address from ES on the Right (destination)sync folder.
It works great on the Note 2 and it is awesome app! 5 star on market and thanks here.
No need to be root
Great guide, thanks a lot!
I have two things to add.
1. You do not need to root your Android device to use SSHDroid.
2. As the user of a non-rooted device, I ran into the problem of permissions when using unison. The solution:
Apart from having the
perms = 0
line in your .prf file, also add
dontchmod = true
So, my file looks like this:
Code:
perms=0
dontchmod = true
root=/home/<username>/Pictures
root=/home/<username>/wifidroid/Pictures
ignore=Name .*~
ignore=Name *~
ignore=Name .*
Placing "perms=0" and "dontchmod = true" at the beginning of the file may be important (I read it somewhere), but maybe it is not.
And it works nicely with my non-rooted device. Thanks again!
Check also UnisonSync:
https://play.google.com/store/apps/details?id=net.danielroggen.unisonsync
It's a Unison client for Android with GUI.
The advantage of unison client (rather than server) on the answer Android device is that you can connect to your server even if your shoe Android device is on a different subnet or behind a firewall (as long as your server is accessible of course).
Cheers,
D.