[Q] Website Copier/Downloader/Ripper for Android? - General Questions and Answers

Hi,
Does anyone know of an app that can create a mirror of a website, or download the whole website (with filters for filename or filetype). In particular I need to rip off images from the site.
On a PC this can be done with Httrack Web Copier or offline browsers.
Thanks!!

thedesolatesoul said:
Hi,
Does anyone know of an app that can create a mirror of a website, or download the whole website (with filters for filename or filetype). In particular I need to rip off images from the site.
On a PC this can be done with Httrack Web Copier or offline browsers.
Thanks!!
Click to expand...
Click to collapse
wget is the easiest way, and I think it's included with busybox

Theres also an app called "SD Card Offline HTML Browser".

Thanks guys.
SD Card Offline HTML browser can only view sites stored on the SD that were ripped from wget etc. But that will be handy too.
I'm looking more into wget (I do have it just need to learn how to use it). I guess this is only for root though.

My app WebLiveWallpaper can do this for images as a byproduct. Select 'Dump all images' in 'System Settings'.
You can do a lot of stuff there in the 'Advanced Settings' with regular expressions.
I am also working on extracting and enhancing this functionality (also dump complete websites) into a separate app but this will take some more time ...

_miha_ said:
My app WebLiveWallpaper can do this for images as a byproduct. Select 'Dump all images' in 'System Settings'.
You can do a lot of stuff there in the 'Advanced Settings' with regular expressions.
I am also working on extracting and enhancing this functionality (also dump complete websites) into a separate app but this will take some more time ...
Click to expand...
Click to collapse
man, your app is really cool...it looks like something different but it has a ton of options...if u can develop this further into a complete image/content extraction tool. this would be awesome
ive been playing around with it but havent been able to do what i have wanted yet...will try some more!

Firstly, I am having some problems with wget: it does not resolve hostnames
Even when I give it an IP address, it still picks up another hostname from somewhere that it cant resolve...I think its a known problem, but I dont know how to update my busybox or wget
Secondly, WebLiveWallpaper is a deceptively good program, but I have 2 problems with it. Firstly it is VERY slow (even with WiFi) in downloading and dumping the images. Secondly I had problems in grabbing images from multiple levels above. It works fine if all the images are on the same page. But nevertheless it does the trick.

WebLiveWallpaper app: if you did not already use you should try advanced settings static gallery set to a high number. That will speed up loading after the first picture very, very much. The app needs to explain everything better but I have not enough time ...
Sent from my HTC Magic using XDA App

Website Downloader on Android
play.google.com/store/apps/details?id=com.library.webdownloader&feature=search_result#?t=W251bGwsMSwxLDEsImNvbS5saWJyYXJ5LndlYmRvd25sb2FkZXIiXQ..
I have found this one. It is downloading the page with all texts and images. :angel::angel::angel:

Related

PDF & DOC attachments

Did a search, so sorry if I missed it. But, a couple of questions.
1) If you have root, is it possible to download and save PDF and Word files that come in on POP3 (or even gmail) email?
2) Otherwise, does anyone know the phone folder where these files are temporarily store while on the phone, and can they be accessed that way?
I spend a lot of time overseas, and going back and forth to the web to retrieve is a pain and expensive. Thanks
Forget it, I solved the issue.
Informing use how you solved the issue might help others
I'd be really interested in how you managed this, I've searched here, and Googled everywhere (almost). But I still haven't found what I'm looking for...
Mikey1022 said:
Informing use how you solved the issue might help others
Click to expand...
Click to collapse
lol...gotta love these types of posts right?
I'm able to save attachments like this using k-9 mail., for those interested
Sorry to go AWOL -- had to leave the country for a bit. My apologies.
Yeah, I also "solved" it by using K9. But, I find the program very buggie.
On a larger, related question, how does the Android platform make its file associations? In other words, when I receive an email attachment (with the stock app), it will allow me to download and save that attachment to my SD card only if there is an onboard, associated program. I can download .jpg, because Android has an application built in that will read a jpg. So, if I want to download a .doc file, I can't unless there is an appropriate app onboard. (Although k9 seems to have found a way). Where, in the Android platform, is this association made? And, is it possible (with a rooted phone) to somehow manipulate those associations to allow downloading of ANY (or a chosen few) files?
For instance, I find that if I change the file extension on a Word doc (for example) from "doc" to "jpg", my onboard email will allow me to download and save to SD card. So, it obviously CAN do it, what's stopping it?
Again, sorry for "skipping out" in the midst of a thread.
Download Crutch Lite in the market allows you to download any file by making Android believe it can open any filetype.
Theres a app called download crutch on the market that allows downloading of files that may not have associated applications. Not exactly sure what file types it supports but its worth a shot.
EDIT: Fushichou got it before me but heres a link to Cyrket http://www.cyrket.com/search?q=download+crutch
Thanks, but my experience is that Crutch will not work on email attachments. It seems to be an internet based device. I used it for a while. Partially, this is a kind of tech question -- how are the files associated?

Run a website on your Android Phone

*edit- For some reason, this thread got posted in Android Software and Hacking general instead of Android apps and games :-|
Hey everyone, its me again.
I have recently been using an app called Paw Server for Android. Though it may seem stupid, this app can really come in handy. The app itself is a utility for managing messages, contacts ect.. But what many people do not realise is that you can host your own website on it, wherever you go. Though this will only work over wifi, you can optionally do some port forwarding to make the site accessible over the internet.
To put a custom website on the server, do the following
1. Install PAW Server for android.
2. Mout your phone's SD-Card On your computer
3. This step has two options:
A Website:
Put your website files in /paw/html/ and make sure the index.html is in this directory. (If you need help creating a site, please visit http://cyanstatic.com)
A File explorer:
If creating a website is too much of a hassle, you can just put some files that you would like to access from your browser in /paw/html/. I Put a couple of songs and videos in the folder and had no trouble streaming the files to my browser.
4. Now open PAW and press the 'Start" Button.
Congratulations!
Your website/chosen files should now be accessible from your browser
i use swiftp, does pretty much that, though it is dedicates to do so, so i think it's a bit more comfortable
This is a great tool
Will give it a shot
doenis said:
i use swiftp, does pretty much that, though it is dedicates to do so, so i think it's a bit more comfortable
Click to expand...
Click to collapse
Really? So can swiftp run a web server or just file services?
Wow amazing!
Sent from my HTC Vision using XDA App
I used PAW Server to host a WIKI on my Nexus One
I used PAW Server to host a WIKI on my Nexus One....
And it almost worked!
tiddlywiki (google it, I'm new and can't do links) is a one page wiki written in, I can't really tell because the tiddlywiki page doesn't do much about talking about how it works.
But PAW serves the pages, and some of the read/browse functions seem to work -- but I was unable to figure out how to edit a page -- and that could just be my fault.
And installing the wiki was pretty easy. I downloaded the zip file to my pc, moved it into my dropbox directory, unpacked it into a directory called wiki, and then on my phone, I synced drop box, and I moved the wiki directory into the paws/html directory. And it served up the page just fine.
But I couldn't edit it.
Why does it ask for permissions to everything? ie Camera, Vibrate, etc....
This maybe unintentional with the manifest, but it looks awfully rogue when programs look for permission to things they don't use.
Very cool thanks for the share! Tough it doesn't make much sense serving via mobile, but i might give it a shot and see if I can use some DNS servers too, starting with dyndns.org
Check out Wifi Remote Access for Windows Mobile: You can do the same thing, but it also comes with a whole range of other stuff.
To edit it, you just need to go into program file, and add a folder inside public (I think) with all your website files. Then change Wifi Remote Access's default.html to redirect you there,instead of to /fs/
mik101 said:
Why does it ask for permissions to everything? ie Camera, Vibrate, etc....
This maybe unintentional with the manifest, but it looks awfully rogue when programs look for permission to things they don't use.
Click to expand...
Click to collapse
This is because it comes with a site on it for managing your phone, like take pictures from the browser and send sms.
With the server it is also possible to develop web applications that use the Android API.
That's also a reason (apart from the web interface mentioned by GlobaL_TroTTer) why there are so many permission requested.
Great find! This is going to be very beneficial to me if I ever misplace my phone around the house or at school. It also has several features that are invaluable IMHO.
Great App - but not working on Verizon network
PAW Server is a real gem... it works great in a LAN/WiFi environment.
Unfortunately, it appears that Verizon Wireless doesn't route ANY traffic that is not initiated by the phone/user.
So thi nice little personal web server is unreachable over the air.
Same for Telnet, FTP, etc.
If anyone has a working app (on a Verizon network) - reachable FROM the net - please let me know!!
Android Port Forward?
GlobaL_TroTTer said:
*Though this will only work over wifi, you can optionally do some port forwarding to make the site accessible over the internet.
Click to expand...
Click to collapse
I'm trying to access the server without connecting through wifi. Is this even possible?
Thanks
I've played around with PAW Server. I like the PHP Plugin. I created an offline UPC scanner web app (for retail inventory) in PHP/JS/WebSQL which synched with a remote server. It eventually turned into a Phonegap app.
Directory Permissions
Does anyone have a link to some documentation regarding "Directory Protection" part of PAW?
It mentioned that granting permissions to certain folders must follow the following format
relative-directory:auth-type:realm:users
with a sample of
/test:basic:Test Real:user
But I don't know what auth-types are available. I'd like to grant a certain folder read permissions to everybody.
I just have to say that setting up my own website with PAW was super easy (10 easy steps):
1) Install PAW and start up server: google PAW android
2) Find external IP: google "what's my IP"
3) Go to router (192.168.0.1 or 192.168.1.1 ) and enable port forwarding to your android for the port you used.
4) Set up PHP: Use browser to hit server(192.168.1.?), sign in, go to add-ons, setup PHP, transfer unzipped folder to /sdcard/paw/html/app/plugins/ and restart PAW server
5) create file (mypage.php) with contents(<html><head></head><body><?php echo '<p>Hello World again</p>'; ?> </body></html>) and place it in /sdcard/paw/html/app/
6) Navigate to [myexternalIP]:[myport]/app/mypage.php (note requires login) Unless you changed the port number it defaults to 8080
7) Register for a free domain: no-ip.com
8) Modify host to have "Port 80 Redirect" to the port you configured (typically 8080)
9) Fix permissions on certain folders so joe blow can hit my server <---- How to do this part????
10) Profit!

[Q] App for RSS reader where each post is a PDF file

Alright, so this use is very specific and I'd be surprised if there is an app that could do what I need but I'm going to try anyway.
I have a RSS feed where each new article has a PDF attachment - in fact, the RSS feed is to get this particular periodical which is released in the PDF format.
I've tried a couple of RSS readers and they sync the new post but not the PDF file. I want to be able to download PDF files from the reader and either read it in the reader or open it using another PDF reader.
Are there any RSS readers which can download attachments?
I don't know about Metro-style apps, but the built in RSS feature of IE can do file download links. It's not the best of interfaces (by a considerable margin), though. Outlook's reader may suit you better except it's not available for RT.
GoodDayToDie said:
I don't know about Metro-style apps, but the built in RSS feature of IE can do file download links. It's not the best of interfaces (by a considerable margin), though. Outlook's reader may suit you better except it's not available for RT.
Click to expand...
Click to collapse
I didn't know that IE had an RSS feature. I'll check that out. Thanks!
GoodDayToDie said:
I don't know about Metro-style apps, but the built in RSS feature of IE can do file download links. It's not the best of interfaces (by a considerable margin), though. Outlook's reader may suit you better except it's not available for RT.
Click to expand...
Click to collapse
Thanks! This worked out fine. I was able to subscribe to the RSS feed and select "Download attached files" in the options. This worked perfectly for me.
The non-touchiness (did I just make up a word?) is an issue but once I had the setup, I don't ever need to look at the feed itself. I can just look at the folder where files are downloaded and other stuff happens in the background.
Thanks again!
One more question though - is there a way to specify the location of the downloaded attachments in IE? By default, it stores attachments in /temp/Internet/ ... blah. I was able to pin this folder to the start menu for easy access but it would be nice to tell IE where to store the attachments.
Any help here would be appreciated.
You're welcome! I'm glad that worked out for you. Yeah, IE has actually had an RSS/ATOM feed reader since version 7, but it's gotten more and more hidden with each release (back in IE7, you could select options like having it play a sound when a feed was discovered, for example). IE10 has improved the reader a bit, but it's still kind of hidden.
I didn't find a way to specify where to deliver the downloaded files, sadly. The best option I know of is to use a directory symbolic link so that when IE thinks it's writing to the temp folder, it's actually writing to a more accessible location that you selected. However, setting up one of these requires using an admin command line, and carries some risk of messing things up. If you would like to do so anyhow, specify the full paths to your temp folder (where the downloads are going now) and the folder you want to put them in instead, and I'll give you a simple script to make it happen if possible, or at least tell you what you would need to do.
Thanks for the offer! Let me dig around a little bit more and if all else fails, I'll come back to you

Great comic book reader:ComittoN

First its the right price FREE second I can connect to windows share folders and read comics from computer.Easy to setup great options for tweaking the look of comics , haven't had any problems opening any comics so far.I own Comic Rack,Mobi Comic reader and a few others but I use this one the most.:good:
jonmanson said:
First its the right price FREE second I can connect to windows share folders and read comics from computer.Easy to setup great options for tweaking the look of comics , haven't had any problems opening any comics so far.I own Comic Rack,Mobi Comic reader and a few others but I use this one the most.:good:
Click to expand...
Click to collapse
Thanks a lot!!! This is what I needed.
This app is so close but fails. I have a large comic collection, 200GB+ and I find a lot of the times when I go to open a comic off the SMB Share it will sit at Loading File List and count up endlessly. Makes it unusable. Unless someone here knows a fix?
Rodimus80 said:
This app is so close but fails. I have a large comic collection, 200GB+ and I find a lot of the times when I go to open a comic off the SMB Share it will sit at Loading File List and count up endlessly. Makes it unusable. Unless someone here knows a fix?
Click to expand...
Click to collapse
I have a collection of 300GB+ and no problems here. I should note, though, that I have my comics organized into folders and subfolders, not just 25,000 files sitting in one directory. My first suggestion is to try creating a folder with 50-100 cbr/cbz files and see if you have a problem loading the list then.
jonmanson said:
First its the right price FREE second I can connect to windows share folders and read comics from computer.Easy to setup great options for tweaking the look of comics , haven't had any problems opening any comics so far.I own Comic Rack,Mobi Comic reader and a few others but I use this one the most.:good:
Click to expand...
Click to collapse
Hey,
So I know this is an old thread but I was hoping for some help. I just found this app and it sounds great, but I am a huge NOOB when it comes to some things. I have my comics folder being shared on my windows 8.1 PC, but I have no idea how I access it via the app. I am able to find the part where I have to input the server path into the app, but I'm not sure what format that takes. I have the IP address, then backslash, computer name, backslash folder name. Not working. How do I know what the path name should be exactly?
Thanks.
Hello,
I've tested this app in order to resolve my main concern about reading comics. As you folks, I have 300Gb+ collection in SMB share, so I need an easy way to navigate through my collection, and ComittoN is really FAST navigating through folders. I also used Perfect Viewer, which also hace CIFS/SMB support, but is painfully slow regarding folfer navigation, specially when using thumbnails.
BUT, as far as I've tested ComittoN, it's only working in Manga mode (reading right to left). I have a lot of manga, but only means 25-30% of the collection, and reading western comics right to left is really weird. Is there any config menu to select reading mode in ComittoN?
Thank you.
---------- Post added at 07:46 PM ---------- Previous post was at 07:41 PM ----------
mikelav456 said:
Hey,
So I know this is an old thread but I was hoping for some help. I just found this app and it sounds great, but I am a huge NOOB when it comes to some things. I have my comics folder being shared on my windows 8.1 PC, but I have no idea how I access it via the app. I am able to find the part where I have to input the server path into the app, but I'm not sure what format that takes. I have the IP address, then backslash, computer name, backslash folder name. Not working. How do I know what the path name should be exactly?
Thanks.
Click to expand...
Click to collapse
I have my collection in Airport shared, in 192.168.1.10. Attached hard drive is named "Cache", and it has one directory, called "Comics".
So, format is as follows:
IP: 192.168.1.10/Cache/Comics
Username: *****
Password: ****
Workgroup: default
I think your issue is with naming convention in shared resources. You have to use "IP"/"Shared resource"/"Directory". You're using "computer name" as shared resource, but I think you should use there the name of the shared resource, not computer name (*I think*).

[APP][4.1+] GIF Share Overlay - Search & share GIFs without leaving your favorite app

[APP][4.1+] GIF Share Overlay - Search & share GIFs without leaving your favorite app
GIF Overlay
GIF Overlay provides an overlay that draws on top of other apps and lets you quickly search and share GIFs without leaving your favorite app.
Features
Search and share GIFs - Powered by GIPHY.
Overlay on top of other apps. Search and share without leaving your favorite app.
Multiple theme support.
Support for multiple window positions.
Safe: Only Internet and Draw on top permissions needed. No personal data collection.
Free, no in-app purchases.
Download
Get it on Google Play
You are going good
Hi I need pba ic scrap 8 gb 16gb so my contact no is 9899961391 same as what's app
good app
Doesn't work for me. App themes work, but nothing shows in the gif window no matter what I search. Share button doesn't do anything either, but that could just be because there is nothing to share. Gif Overlay is set to be allowed to draw over other apps in my device btw.
bogarty said:
Doesn't work for me. App themes work, but nothing shows in the gif window no matter what I search. Share button doesn't do anything either, but that could just be because there is nothing to share. Gif Overlay is set to be allowed to draw over other apps in my device btw.
Click to expand...
Click to collapse
That should only happen if there is an error contacting the server for the search or parsing the search response data.
Could you check adb logcat messages? You should see some error there.
rumboalla said:
That should only happen if there is an error contacting the server for the search or parsing the search response data.
Could you check adb logcat messages? You should see some error there.
Click to expand...
Click to collapse
Got it to work! For some reason it didn't work until I disabled my VPN. Then after a successful gif search I rebooted the device and it seemed to be working fine with my VPN. It works great now for most apps, but for the few that it doesn't, is it possible to maybe allow for the Gif to be somehow copied right into whatever text field you're in or is sharing to the app the only way? Great app though btw. Thank you!
bogarty said:
Got it to work! For some reason it didn't work until I disabled my VPN. Then after a successful gif search I rebooted the device and it seemed to be working fine with my VPN. It works great now for most apps, but for the few that it doesn't, is it possible to maybe allow for the Gif to be somehow copied right into whatever text field you're in or is sharing to the app the only way? Great app though btw. Thank you!
Click to expand...
Click to collapse
I could copy the url to clipboard on long press a gif.
rumboalla said:
I could copy the url to clipboard on long press a gif.
Click to expand...
Click to collapse
That would be cool. I guess there's no way to really copy the image itself to clipboard is there? Only text? I've been looking for an app that does that myself for a long time.
bogarty said:
That would be cool. I guess there's no way to really copy the image itself to clipboard is there? Only text? I've been looking for an app that does that myself for a long time.
Click to expand...
Click to collapse
As far as I know, you can copy the image to the clipboard. The problem is that the app where you are pasting will not understand you are pasting an image unless they have implemented that.
rumboalla said:
As far as I know, you can copy the image to the clipboard. The problem is that the app where you are pasting will not understand you are pasting an image unless they have implemented that.
Click to expand...
Click to collapse
Generally, when you touch and hold an image in an app you can share or download it and share it to wherever, but i don't think I've ever seen a "Copy to Clipboard" option except when copying text. I see what you mean about pasting the image though and maybe a gif would be even more complicated to do that with.
I wonder if it would be better if you set it the touch and hold the image function to download the gif instead. Then you could add it anywhere.
Changelog 1.4
Added button to download GIF.
Added button to copy GIF URL.
rumboalla said:
Added button to download GIF.
Added button to copy GIF URL.
Click to expand...
Click to collapse
Thank you! I tried it out and for some reason I'm still having connection problems. Even turning off my VPN doesn't seem to make it search. I noticed that sometimes when turning off my WiFi connection it will search right away, but not all the time. I wonder if I'm the only one having this issue. When it does search it does appear to download, but even when searching my entire sdcard for "gif" or ".gif" or even the name that it says it saved as afterwards I can't seem to find where it saves. It seems as if maybe it's not really saving, at least as a gif. Can't find it when opening the gallery either. Opening it in gallery from the download notification does show it, but it doesn't allow you to view any details on it either, nor does it let you share it when opening it like that. URL copy works fine, though maybe a toast message saying it's copied would help to let people know that it has copied, otherwise, unless you try to paste somewhere it appears as if it's done nothing. Share works fine as usual. Anyhow, I hope this helps. Thank you!
bogarty said:
Thank you! I tried it out and for some reason I'm still having connection problems. Even turning off my VPN doesn't seem to make it search. I noticed that sometimes when turning off my WiFi connection it will search right away, but not all the time. I wonder if I'm the only one having this issue. When it does search it does appear to download, but even when searching my entire sdcard for "gif" or ".gif" or even the name that it says it saved as afterwards I can't seem to find where it saves. It seems as if maybe it's not really saving, at least as a gif. Can't find it when opening the gallery either. Opening it in gallery from the download notification does show it, but it doesn't allow you to view any details on it either, nor does it let you share it when opening it like that. URL copy works fine, though maybe a toast message saying it's copied would help to let people know that it has copied, otherwise, unless you try to paste somewhere it appears as if it's done nothing. Share works fine as usual.
I've a attached a verbose log here for you to look at in case you need it. Had to zip it because apparently 2kb is too big for text on XDA? Anyhow, I hope this helps. Thank you!
Click to expand...
Click to collapse
Can't see anything weird in the logs.
The download triggers the android downloadmanager so the file can go to some weird place like /data/data/com.android.providers.download. I could change it so it goes to the downloads folder but then I need to add an extra permission.
And yeah adding some sort of message or toast is a good idea.
rumboalla said:
Can't see anything weird in the logs.
The download triggers the android downloadmanager so the file can go to some weird place like /data/data/com.android.providers.download. I could change it so it goes to the downloads folder but then I need to add an extra permission.
And yeah adding some sort of message or toast is a good idea.
Click to expand...
Click to collapse
That would be great. I think I figured out that it goes to "/data/data/gif.overlay/cache/image_manager_disk_cache". The files in there were ".0" format, but when I changed it to ".gif" they showed the gifs I was trying to download. Assuming those aren't just the cached images from the search that is. Regardless you can't really access them or share them when they're outside of the sdcard.
bogarty said:
That would be great. I think I figured out that it goes to "/data/data/gif.overlay/cache/image_manager_disk_cache". The files in there were ".0" format, but when I changed it to ".gif" they showed the gifs I was trying to download. Assuming those aren't just the cached images from the search that is. Regardless you can't really access them or share them when they're outside of the sdcard.
Click to expand...
Click to collapse
No the ones inside the gif.overlay folder are either cached images from the search (those are not the final version but downsampled images to make it faster) or files saved before sharing. The downloaded ones should end up as giphy.gif somewhere where the downloadmanager puts it (in my case /data/data/com.android.providers.download).
rumboalla said:
No the ones inside the gif.overlay folder are either cached images from the search or files saved before sharing.
Click to expand...
Click to collapse
Yeah. Just checked and you're right. The gifs do go to"/data/data/com.android.providers.download/cache/". Strange that they weren't showing up in my system wide search. Regardless, it would be cool if they could go to the "downloads" or "pictures" or a "/sdcard/Gif Overlay/" folder or something. Then there would be no end to the possibilities.
Now I just have to figure out why it's only connecting like half the time. Just tried again and it worked. Earlier though it wasn't. It must be something on my end. Not sure.
EDIT: Downloaded "Advanced Download Manager" from the Play Store and set it to auto incercept copied links for now, so that now I just have to click the copy function in the app and ADM asks me where I want to save it to. Clean flashed a new rom too and now I'm not having any more connection issues btw. Must've been something blocking it on my old configuration.
Changelog 1.5
* Added option to select window position.
* Bug fixing.

Categories

Resources