25000 .txt files - problem - Eee Pad Transformer Q&A, Help & Troubleshooting

Hi,
I am musician and have big lyric collection, around 25.000 files.
They are all saved in .txt format.
Since i got Transformer idea was to have it with me on my gigs instead of having huge 17inch laptop.
I tried to copy files, from external HDD, over WIFI, from SD card ... and its taking so long, last time i stopped it after almost an hour of transfer. And i think i got only around 3000 files copied.
Do you have suggestion on how to quickly copy/manage all those files on tablet? (no problems on PC whatsoever )
I appreciate your help.
Thank you.

Buy a damn fast SD card and reader, and copy the files to the SD card through the PC.
Or get an USB HDD and plug it into the TF.
Copying millions of small files always takes ages, and flash chips are especially bad at this. And the flash chips in the TF isnt exactly the fastest performing available.
Even my SSD disc on my PC slows down a lot during small-file writing (sometimes as low as 10MB/sec, compared to ~530MB/sec on larger files).
So basically, copy them within, to or from the tablets internal memory will take years. Only "solution" would be to zip all the files so you end up with 1 big zip file instead of 25k small files.

Thanks for answer.
As i mentioned, i tried to copy them from external USB Hard Drvie, as well as SD card, Class 10.
I forgot to mention that i tried to zip/rar files, copy to internal SD and then extract to a folder. Same thing happens. Veeeeery slow. Takes ages.

Yeah, the issue is youre writing to the internal memory, which isnt capable of writing very fast.
Your best solution would be to just keep the files on the USB HDD (if you got USB port for your TF) or the external SD card

Not to mention that transfers of many small files is generally the slowest of all, where large single files are generally much more efficient. No matter what you do, you still can't avoid that problem. My suggestion would be to do as others have said and connect the SD card directly to your computer, transfer them, and then put the SD in your transformer.

looks like that will be my only option
thank you guys
Sent from my Transformer TF101 using XDA

akyboy said:
Hi,
I am musician and have big lyric collection, around 25.000 files.
They are all saved in .txt format.
Since i got Transformer idea was to have it with me on my gigs instead of having huge 17inch laptop.
I tried to copy files, from external HDD, over WIFI, from SD card ... and its taking so long, last time i stopped it after almost an hour of transfer. And i think i got only around 3000 files copied.
Do you have suggestion on how to quickly copy/manage all those files on tablet? (no problems on PC whatsoever )
I appreciate your help.
Thank you.
Click to expand...
Click to collapse
dont know if this method will help but, you could use 7zip to compress the files into 1 file from a PC, there is a 7zip app for android that can open it up and view the files. so you dont have to uncompressed it on the tablet
7zip http://www.7-zip.org/download.html
7zip on anrdoid https://play.google.com/store/apps/details?id=ru.zdevs.zarchiver&feature=search_result#?t=W251bGwsMSwxLDEsInJ1LnpkZXZzLnphcmNoaXZlciJd
with 25k files, i think you need database. may help make it easier to search and sort through so much data.

*Phi* said:
7zip http://www.7-zip.org/download.html
7zip on anrdoid https://play.google.com/store/apps/...=W251bGwsMSwxLDEsInJ1LnpkZXZzLnphcmNoaXZlciJd
with 25k files, i think you need database. may help make it easier to search and sort through so much data.
Click to expand...
Click to collapse
Tnx for tip, i will try 7zip definitely!!
Regarding Db,.. thats the problem i am trying to find db app or similar for 3 years now. Wanted to do this on my cell phone too.
On windows i simply use TreePad software and everything is in one big file, search-able(indexed) so i can search for any word inside text and i will get results in matter of seconds.
I wish there is something similar available on android

akyboy said:
Tnx for tip, i will try 7zip definitely!!
Regarding Db,.. thats the problem i am trying to find db app or similar for 3 years now. Wanted to do this on my cell phone too.
On windows i simply use TreePad software and everything is in one big file, search-able(indexed) so i can search for any word inside text and i will get results in matter of seconds.
I wish there is something similar available on android
Click to expand...
Click to collapse
well, if you looking for a db app these isnt any out there as far as i know, BUT
http://paw-android.fun2code.de/
you could try this. its like running a mini web server on your tf. and it allows you to make connections to sqlite, which is already built into android. so in theory you could be able to make a web app that stores your lyrics and allows you to access it via web browser on your tf (offline too).
it can run PHP, i've only used it to run web pages for demos nothing really heavy. never tried connecting to sqlite though.
http://www.vogella.de/articles/AndroidSQLite/article.html
if you are interested.

Tnx for answer Phil,...
I tried, and still have PAW setup on my tablet with PHP module installed.
There is option in dB program i use on Windows (TreePad) to export searchable (requires php) WebPage of my database, but looks like that their export is kinda buggy and That Exported site sometimes works sometimes doesnt. I contacted them and i am waiting for their answer.

I am still looking for something in this regards,... any other ideas or solutions is appreciated!
Thank you

Related

Direct copy of files via BT to Memory Card

Hi folks,
is there a possibility to have incoming files via BT to be copied to memory card instantly?
Currently files are first being copied to main disk on WM6 (after you agree to receive) and then you can specify where you want it saved in.
Due to lack of space, I would prefer if it is possible to override the current way and have files going to memory card instantly.
Would be happy if there is an solution?
regards
Horst
There is a BT utility called "Dr YAR" something. Sorry I can't recollect the entire name. One of it's features is it allows you to transfer files directly to the storage card. However I must inform you that in all the times I have tried it over various ROMS and devices, I have never had it successfully complete large files. It always goes through to about 50% and then hangs.
I really hope somebody can create a simple app that can cover for this one annoying aspect of WM.
Hi,
thx for the Info. i also do know of a program called "Sniper", which fulfills the job more or less as well.
But what I was wondering, if there aint a way to get files directly to the card.
regards
Horst
Waggl Daggl said:
But what I was wondering, if there aint a way to get files directly to the card.
Click to expand...
Click to collapse
Without third party software? I don't think so.
There are 2 ways a file is transfered to your
1. Information Exchange
2. File Transfer
Usually when you transfer a file from a normal non-smart phone, it sends a file using the Information Exchage, OBEX push, i think... This is used by WM mobiles for transferring Outlook files like vCard, etc. So when you recieve files from another Phone to ur WM via OBEX push, you most probably, wont have have control over where u can save the file, atleast by default firmware.
Now in the second case, Say transfering a file from a Laptop, it uses "File Transfer" to send a file. This uses a shared folder on your WM phone, and the file is transfered to your shared folder. This Shared folder is defined in the "Advance..." button of your "File Transfer" service, under the "Services" tab in Bluetooth Settings.So basically select a folder on your card to define where you recieve the files. i have transfered a fairly large file (24Mb , thats more than the free memory i have on my ROM) directly tothe card. kindly note that the files in the folder that you have decided to share will be available/ accessible to the host bt device sending the file, so its a good idea to define a new folder that for the purpose.
Hope this info was helpful
official hermes AT&T WM6 does not have FILE TRANSFER!
Hello,
I'm linking to my other posts in the thread below, since nobody has replied to that one. Anyway, the HERMES offcial AT&T ROM does not have the OBEX file transfer service, only the default Inbox service. In that case I have managed to transfer direct to the SD Card using this service, but I would like to improve the process by removing the UI prompt. Please read the other thread and let me know if you can duplicate my results and/or if there is a way to remove the prompts. Thanks,
http://forum.xda-developers.com/showthread.php?t=268697

[Q] [HOW TO] Content transfer - Cross Platform

How do you get to transfer in the easiest way - songs from an android device to a fully unlocked Nokia Lumia 710?
also..
Where do the attachments in my hotmail get saved on my local phone storage? I have fully unlocked L710 and HTC Titan. Someone sent me a few clips attached to hotmail. I can play it from my email attachment but would like to find the folder where it is locally stored after download, and then be able to play the content from Zune. Is there any homebrew app for such file management? I have File Explorer, but don't know where these attachments can be found, then moved so that they are playable in Zune Hub.
this is a great question that I would also like to know the answer to. There has to be some way to mount the storage and manage it from a pc like you can with Android.
Skywalker134 said:
this is a great question that I would also like to know the answer to. There has to be some way to mount the storage and manage it from a pc like you can with Android.
Click to expand...
Click to collapse
I did not really bother trying to find the answer. But someone who uses Android in my family tried to send me a few clips of music that I really really wanted to hear. Now ofcourse I have got it via bluetooth and that I can access. However, he sent a few more by email as my bluetooth transfer was slow. I had presumed I can find these songs in the same place. BUT nope!
However, here is my progress.
File explorer -> Applications data -> Volatile -> Email attachments -> go to each folder. Each folder represents one attachment.
So I was happy after posting this that I found where the attachments are.
However, when I copy and paste the same under my Zune contents, I cannot see it in my Zune Hub and cannot ofcourse play it from Zune hub!
Zune on the phone uses a database of media files. Merely moving your files under \My Documents\Zune\Content\* isn't going to help. The only way that I know of to add arbitrary files is to copy them to the PC, then sync them using the PC Zune software. Nobody has yet figured out how to do it more directly, although I'm sure it's possible.
Mounting the phone as USB Mass Storage is just not even vaguely likely to happen. The amount of driver writing and re-configuration that would be needed... I don't think you realize what you're asking for. There are deinitely options, though; for example, my Root Webserver app lets you browse the phone's filesystem (including uplodaing and downloading files) from your PC. You just have to use a web browser instead of a traditional file manager.
GoodDayToDie said:
Zune on the phone uses a database of media files. Merely moving your files under \My Documents\Zune\Content\* isn't going to help. The only way that I know of to add arbitrary files is to copy them to the PC, then sync them using the PC Zune software. Nobody has yet figured out how to do it more directly, although I'm sure it's possible.
Mounting the phone as USB Mass Storage is just not even vaguely likely to happen. The amount of driver writing and re-configuration that would be needed... I don't think you realize what you're asking for. There are deinitely options, though; for example, my Root Webserver app lets you browse the phone's filesystem (including uplodaing and downloading files) from your PC. You just have to use a web browser instead of a traditional file manager.
Click to expand...
Click to collapse
That does add some light to the scenario! Thanks!
So does that mean that even though we do have homebrew file explorers, we cannot really control our attachments i.e. sandbox issue? I have realised in last two days that our file explorer functionally isn't same as those traditional WM6.5 or Android ones. Hence theoratically speaking yes we do have file explorer but not to its full functionality..which leads me to another question. So ideally what do you use your file explorer for?
Aside from aiding my research and testing my hacks, here are a few things I use file browsers for:
Copying ebook files (.MOBI and .PRC) to the Kindle directory.
Manually editing the Marketplace configuration files (for example, some downloads are even bigger than the 200 MB that Marketplace Config will allow you to specify).
Freeing up space on the phone's storage (deleting temprorary files of my choosing, rather than simply purging the whole phone).
Backing up application data to my PC.
Turning music files into ringtones/alarms/etc.
A number of these can currently be automated using apps, and there are others that could be more useful to other people, but that's what I've found to be useful.
Content Transfer
Well I have been in same situation , I went with wifi transfer option,caz its quite secured and helpful for content transfer, like you can use smartIO App, which is good op
drupad2drupad said:
How do you get to transfer in the easiest way - songs from an android device to a fully unlocked Nokia Lumia 710?
also..
Where do the attachments in my hotmail get saved on my local phone storage? I have fully unlocked L710 and HTC Titan. Someone sent me a few clips attached to hotmail. I can play it from my email attachment but would like to find the folder where it is locally stored after download, and then be able to play the content from Zune. Is there any homebrew app for such file management? I have File Explorer, but don't know where these attachments can be found, then moved so that they are playable in Zune Hub.
Click to expand...
Click to collapse

Best App to Move Photos from PC to Infinity?

I have both the ASUS Extension Kits - one for USB and the other as an SD Card Reader.
I've tried moving photos from my PC with both kits and the File Manager App with no success. The File Manager somehow corrupts JPG files when using the SD Card Reader and it does not recognize my USB stick.
I can move the photos one or two at a time as an email attachment, but I have several hundred that I'd like to put on the Infinity.
Suggestions? TIA!
DUTCH Van Atlanta said:
I have both the ASUS Extension Kits - one for USB and the other as an SD Card Reader.
I've tried moving photos from my PC with both kits and the File Manager App with no success. The File Manager somehow corrupts JPG files when using the SD Card Reader and it does not recognize my USB stick.
I can move the photos one or two at a time as an email attachment, but I have several hundred that I'd like to put on the Infinity.
Suggestions? TIA!
Click to expand...
Click to collapse
A little looking, and I answered my own question. Photo Transfer App works very well wirelessly. I used it on my iPad, and it works well here, too.
Also, there is the wireless option via FTP. I use an old app called SwiFTP. It's no longer developed, but Im sure you can find the apk somewhere. It gives full access to your device and it's FREE! There are other apps like it on the Market as well. I use it whenever I don't feel like grabbing my USB cable.
Alitheia said:
Also, there is the wireless option via FTP. I use an old app called SwiFTP. It's no longer developed, but Im sure you can find the apk somewhere. It gives full access to your device and it's FREE! There are other apps like it on the Market as well. I use it whenever I don't feel like grabbing my USB cable.
Click to expand...
Click to collapse
I'd recommend File Expert. It works through HTTP, so on any computer on the same wifi network with a web browser can get into it provided the user has the correct password. The web page UI for managing files is pretty nice, if a little buggy. Only flaw is that its UI is slow (at least on my Atrix) and not tablet optimized.
Jotokun said:
I'd recommend File Expert. It works through HTTP, so on any computer on the same wifi network with a web browser can get into it provided the user has the correct password. The web page UI for managing files is pretty nice, if a little buggy. Only flaw is that its UI is slow (at least on my Atrix) and not tablet optimized.
Click to expand...
Click to collapse
Photo Transfer App is also http and works A-OK with no bugs. It will transfer photos or videos in either direction at very good speed. Photos can be batched up to 100 photos, abd videos are done singly. There are both Android and iOS Apps so I can take photos/videos off of my iPad and move them to my TF700.
Even though probably adding a 'solution' to a problem no longer in existence might I add that Total Commander supports both internal and external memory (with root access), (S)FTP, WEBDAV, LAN shares and more, and is still in active development? Also, you would not be limited to transfer of grahical files. (I haven't used specific photo transfer apps yet, so I do not know if they'd provide (a) definite advantage(s). If you think I'm missing out on something, please chime in and release me from my blissful ignorance. )
I've used MyPhoneExplorer for all file transfers. Works over my WIFI network although it will also work over cable.
AirDroid is fantastic...
Airdroid is the best app to move anything between android and windows
Sent from my ASUS Transformer Pad TF700T using xda app-developers app
Airdroid +1
Sent from my Samsung Galaxy S2 using XDA Premium app

what is the importance of usb mass-storage mode?

if you have something like samsung kies or airkies?
thanks.
randomandroid said:
if you have something like samsung kies or airkies?
thanks.
Click to expand...
Click to collapse
Because I rather be able to plug in my phone and access my files like a normal hard drive in windows, mac, linux, whatever than having to run a whole bloated software to which is nothing more than a file explorer. It's like iTune. I rather be able to open my USB storage device, copy a folder of mp3s and go play them on my my phone. not open up some stupid massive bloated program that will convert my mp3's to another format, rename them to unfindable file names, organize them in stupid structures so I can never retrieve my mp3's off the device, loose quality converting all over the place, just to be able to play a damn song.
On the flip side of things, I use adb, scp, or rsync most of the time, and occasionally pull the microSD and put it in a reader. I have literally, in all the time I have owned my phone, never put it in mass-storage mode.
Me too. Most of the time I use dropbox to avoid phone disassemble.
sent from me

MTP / Media Storage issues –*need to deactivate indexing for one directory

Hi all,
I have a quite special issue. I hope someone here can help me or point me into the right direction.
Huawei P9 with Android 6.0, rooted.
For my purposes (*) I use a MicroSD card with 128MB capacity that contains a lot of small files (abt. 1.5 million files).
I wondered for weeks, why the phone becomes incredibly slow and hangs from time to time.
I learned that the Media Storage service could be responsible, because it tries to index all files and that can take a very long time if there are lots of files.
I deactivated Media Storage (using Titanium Backup) and BINGO! My phone is like new. Snappy, well usable.
Since I don't need MTP access to the phone, I thought it might be good now.
HOWEVER:
I just found out that ring tones don't work anymore, neither for phone ringing, nor for the alarm clock etc.
It seems that those apps also rely on the file index, that's gone now.
Without any ring or alarm tone, the phone is almost unusable again.
Do you know of any way to make the MTP / Mediastorage server ignore one directory when indexing? That would help, as I could simply let it ignore the directory on SD card that contains those 1.5 million files.
Or can you think of any other good solution for my problem?
Thank you so much!
Daniel
(*) to explain my usecase, why I need so many files on SD: I am a photographer, doing gigapixel panorama photography. For presenting my work to customers, I put my demo panoramic tours onto the SD card. These are HTML files with a data directory containing JPG tiles of the gigapixel panoramas. One panorama image consist of up to a few 100.000 files. A webserver running on the phone makes the panorama tours accessible to onboard browsers or, via WiFi hotspot, to other devices in the surrounding, e.g. a PC or tablet of the customer.
I considered solutions like an external storage (USB OTG) connected to the phone only when I present that stuff to the customer, or an entirely separate device (Pandora / Dragonbox Pyra or so) to host and serve the panorama tours, but I like to carry only one device, the small smart phone, to do it all. An extenal OSB OTG storage solution would almost fit my needs. But.. only almost, because a small storage is something that can get lost or stolen easily and that needs additional handling and provides additional points of failure, which I don't want to have in front of the customer.
Android is capable of doing it all, and my philosophy is, that a computer needs to adapt to my way of work than vice versa. This was the intention when Neumann and Zuse invented programmable computers. And I don't really want to eat humble pie here.
daniel908 said:
Hi all,
I have a quite special issue. I hope someone here can help me or point me into the right direction.
Huawei P9 with Android 6.0, rooted.
For my purposes (*) I use a MicroSD card with 128MB capacity that contains a lot of small files (abt. 1.5 million files).
I wondered for weeks, why the phone becomes incredibly slow and hangs from time to time.
I learned that the Media Storage service could be responsible, because it tries to index all files and that can take a very long time if there are lots of files.
I deactivated Media Storage (using Titanium Backup) and BINGO! My phone is like new. Snappy, well usable.
Since I don't need MTP access to the phone, I thought it might be good now.
HOWEVER:
I just found out that ring tones don't work anymore, neither for phone ringing, nor for the alarm clock etc.
It seems that those apps also rely on the file index, that's gone now.
Without any ring or alarm tone, the phone is almost unusable again.
Do you know of any way to make the MTP / Mediastorage server ignore one directory when indexing? That would help, as I could simply let it ignore the directory on SD card that contains those 1.5 million files.
Or can you think of any other good solution for my problem?
Thank you so much!
Daniel
(*) to explain my usecase, why I need so many files on SD: I am a photographer, doing gigapixel panorama photography. For presenting my work to customers, I put my demo panoramic tours onto the SD card. These are HTML files with a data directory containing JPG tiles of the gigapixel panoramas. One panorama image consist of up to a few 100.000 files. A webserver running on the phone makes the panorama tours accessible to onboard browsers or, via WiFi hotspot, to other devices in the surrounding, e.g. a PC or tablet of the customer.
I considered solutions like an external storage (USB OTG) connected to the phone only when I present that stuff to the customer, or an entirely separate device (Pandora / Dragonbox Pyra or so) to host and serve the panorama tours, but I like to carry only one device, the small smart phone, to do it all. An extenal OSB OTG storage solution would almost fit my needs. But.. only almost, because a small storage is something that can get lost or stolen easily and that needs additional handling and provides additional points of failure, which I don't want to have in front of the customer.
Android is capable of doing it all, and my philosophy is, that a computer needs to adapt to my way of work than vice versa. This was the intention when Neumann and Zuse invented programmable computers. And I don't really want to eat humble pie here.
Click to expand...
Click to collapse
There's an Xposed module that can do exactly what you're asking. But I'm not sure how nicely the xposed framework will work with your Huawei device (assuming you're still stock). You can check it out if you feel its worth any hassle that might be involved. Here's the link to the forum and a screenshot of the app's description & capabilities
https://forum.xda-developers.com/xposed/modules/xposed-xposed-media-scanner-optimizer-t2942609
GREAT! THANKS!
I didn't know that an XPosed module existed that could solve that problem.
An attempt to install XPosed a few months ago has failed.
But I retried today with the Wanam build (official one failed back then) and it worked well. Downloaded this module now and will see how it works.
Thank you so much!
Quick question:
The manual for the Media Scanner Optimizer module says
"Directories: If checked the media scanner will only scan directories that have a .scanMedia file (i.e. an empty file with a dot as first part of the name similar to the .noMedia file). Use a file explorer app to create this file. This file can be placed in a directory that may contain subdirectories. The subdirectories will be scanned as well. If unchecked all directories will be scanned (except for the ones containing a .noMedia file - this default behavior of the media scanner is not altered by the Xposed Media Scanner Optimizer)."
(source: http://repo.xposed.info/module/com.thomashofmann.xposed.mediascanneroptimizer)
Does that mean, in my case I coud simply have placed a .noMedia file in the directory with the millions of files and my problem would have been solved, even without XPosed?
Somewhere I read that the media storage service does NOT care about the .noMedia files, so I refused experimenting with this quite early in my research phase.
Do you have experiences, if this really works or not?
daniel908 said:
Quick question:
The manual for the Media Scanner Optimizer module says
"Directories: If checked the media scanner will only scan directories that have a .scanMedia file (i.e. an empty file with a dot as first part of the name similar to the .noMedia file). Use a file explorer app to create this file. This file can be placed in a directory that may contain subdirectories. The subdirectories will be scanned as well. If unchecked all directories will be scanned (except for the ones containing a .noMedia file - this default behavior of the media scanner is not altered by the Xposed Media Scanner Optimizer)."
(source: http://repo.xposed.info/module/com.thomashofmann.xposed.mediascanneroptimizer)
Does that mean, in my case I coud simply have placed a .noMedia file in the directory with the millions of files and my problem would have been solved, even without XPosed?
Somewhere I read that the media storage service does NOT care about the .noMedia files, so I refused experimenting with this quite early in my research phase.
Do you have experiences, if this really works or not?
Click to expand...
Click to collapse
I initially thought about the .nomedia alternative. But I wasn't clear on if the 1.5 million files you have were actually media files. If they are then creating a .nomedia file should do the trick. If not, then you have the Xposed module
I personally have a few .nomedia files here and there on my SD card. And it does what its supposed to, ie; stops any media file that's present in those folders from showing up in the Gallery/music player.
So I guess its safe to assume that they weren't indexed either
Freewander10 said:
I initially thought about the .nomedia alternative. But I wasn't clear on if the 1.5 million files you have were actually media files. If they are then creating a .nomedia file should do the trick. If not, then you have the Xposed module
I personally have a few .nomedia files here and there on my SD card. And it does what its supposed to, ie; stops any media file that's present in those folders from showing up in the Gallery/music player.
So I guess its safe to assume that they weren't indexed either
Click to expand...
Click to collapse
I thought the .nomedia trick would only work, if the Xposed module was there?
Or does it work by default, even on a non-rooted stock ROM?
Thise mana files that I have are almost all JPG files. There are a few .js and .xml files, too, that control how the .jpgs are rendered on the client's browser, also some video files that are embedded into the virtual tours, but about 99.5% of the files are .jpg.
daniel908 said:
I thought the .nomedia trick would only work, if the Xposed module was there?
Or does it work by default, even on a non-rooted stock ROM?
Thise mana files that I have are almost all JPG files. There are a few .js and .xml files, too, that control how the .jpgs are rendered on the client's browser, also some video files that are embedded into the virtual tours, but about 99.5% of the files are .jpg.
Click to expand...
Click to collapse
It doesn't matter what type of files you have. If the folder has a .nomedia file, it will not be scanned for anything.
This is standard android behavior, and does not require xposed to be installed.

Categories

Resources