I'm working on a personal modding project, where I take the AOSP Dialer and add some features that I'd like to have. Long story short, for a component of this, I need to figure out how Google has been able to inject arbitrary audio into the conversation/call audio stream.
For years, discussion online, and especially on Stack Overflow, has insisted that:
There are too many upstream limitations, this is impossible
This is impossible, you have to play it over the speaker and hope the microphone picks it up
You can't do this, even Google says so
Indeed, even Google's up-to-date MediaPlayer documentation clearly shoots this down and doesn't mince words:
You cannot play sound files in the conversation audio during a call.
Click to expand...
Click to collapse
However, we know this isn't true. At least, not anymore, and not on Pixel devices. Google's Call Screening feature can "talk" to someone calling your phone, but that synthesized audio is never played audibly to the user. In other words, Google has been able play a Text-To-Speech stream of audio to someone calling your device, doing so silently, clearly to the listener, and without requiring the handset speaker or user's microphone to be "on."
Despite the fact that this is "possible" by virtue of "it has already happened," I can't find any discussion, documentation, info, or anything helpful about how Google has been able to do this. So, what do you do?
The next logical step is to start decompiling the app, but that's easier said than done. I'm by no means an expert in reverse engineering Android apps. Admittedly, you could consider me a beginner. Still, I've found a few things that seem useful, so here's what I've been able to find:
First off, when compared to the AOSP Dialer, the Google Dialer requires an additional privapp-permission that may be of interest: android.permission.MODIFY_AUDIO_ROUTING. I can't find much about what this permission does or how it is used, but it's definitely used by the Google Dialer, and on a stock Pixel 3 XL ROM, that permission is defined in /product/etc/permissions/privapp-permissions-google-p.xml as a privapp-permission.
Next, when decompiling the app, one of the first things I noticed is that a special "IMPL" package containing a playInternal function is class injected/loaded dynamically, and it's adjacent to MediaPlayer code that seems to play audio over a certain channel. The class it tries to load is:
Code:
com/android/dialer/audio/impl/CallAudioPlayer
However, that package isn't present in the list of decompiled classes (there's no "audio" folder under "dialer"), and despite playInternal being explicitly called by a string, there aren't any other classes that seem to define the playInternal function.
I don't know where XDA stands on posting decompiled code, but if you're using JADX, the area of interest is in defpackage/bhk.java.
But if I'm correct, this means that:
You can use MediaPlayer to play over the call stream, contrary to Google's documentation; you just need a special IMPL that allows for that behavior
This (probably?) requires the aforementioned MODIFY_AUDIO_ROUTING permission
If one were to obtain/locate and re-implement CallAudioPlayer.java, you could probably reproduce this behavior in the AOSP Dialer, or any other system app with the necessary permissions
If I'm not correct, then chasing down CallAudioPlayer will be a dead end. Still, the fact stands that Google did this somehow, so the answer must be somewhere.
So... that's where I'm at with this. I don't feel like I'll get much further without some help from more knowledgeable people, since I don't know where this CallAudioPlayer class is located. If it's in the base apk, but obfuscated, I can't find it. And if it's in a system framework or overlay APK, it must be using a different name, since no instances of CallAudioPlayer or playInternal exists in any of the relevant .apks on my system.
If this requires more sophisticated Android system/API modifications, that's fine too. This will end up on a custom ROM, so even if part of this behavior extends outside of the APK, any potential solution can be implemented in an AOSP ROM to achieve this functionality.
If you have any advice on how this may have been done, I really appreciate any and all discussion I can get on this. And if you don't know, I encourage you to ask a friend or someone who might be more knowledgeable when it comes to reverse engineering. Even if no one outright knows the answer to this, I hope to get at least some recent discussion on this topic, so that people investigating this in the future will at least have some sort of starting point.
Thanks for reading!
Hi. I'm no expert on Android (far from that haha) or Reverse Engineering. Though, I'm making an assistant installed as system app in a rooted Android and can also be compiled with the hidden and internal methods and classes so they can be used, like ITelephony, for example (https://github.com/anggrayudi/android-hidden-api) - btw, have you tried to mess with those classes and methods? (Sorry for my ignorance, I've no idea how it is to code/recode a ROM, what you need to use and stuff.) And anyways, I found something you didn't mention up there, so I'm unsure if you already know or not. From what Google says here (https://support.google.com/phoneapp/answer/9118387) in the "Screen calls manually" section, "Your Google Assistant screens the call and ask who's calling and why. You'll get a real-time transcript of how the caller responds.". Though, if it's their assistant doing that, not sure why that permission is on Dialer and not only on the assistant. Anyways, maybe that could be a good place to look at?
EDIT: Maybe also you (or anyone, of course) could look at sending DTMF tones over the call (I only made a quick search on Google, but there may be more that could help on this?). I think it's the same thing as it's inserting audio into the call. Though I can't be sure - btw, if it's really sending audio, then any Dialer app can already do that! But may not be that simple, so no idea at all. Maybe that's hardware thing and not software (>95% probable?). Just trying to give to ideas on where more to research.
Here (https://issuetracker.google.com/issues/36906273#comment107) it's said "I found this app, that can send dtmf after the call is made and active: https://play.google.com/store/apps/details?id=mobi.drupe.app (...)" - name: "drupe - Contacts & Caller ID" (just in case the app gets deleted from PlayStore and then people don't have their name to search for, only the package). If it's audio that it's being injected then it's possible, since that app seems to do it already, and the answer could be there too and might be good to look for the keys' frequencies on the code? Again, sorry for my ignorance on anything wrong I said. As I said in the beggining, I'm far from expert/experienced on Android.
I'll also be trying to search on how to send DMFT tones over a phone call. Could help, maybe. And if only a frequency could be sent for any reason, at least it's already cool to send some beep haha (preferably different from the ones of the keys or it might be confused with a key press by the other side, depending on who we're calling).
Late reply, did you solve this? I would like to build call features on Aosp. Best!
Related
Development thread for a software method to morph your voice during a phone call
The idea is to develop a software method which let's you morph your voice during a phone call. There are many ways to achieve this and I am amenable to suggestions.
Since I was not able to find any API which let's you configure the voice or the stream from the microphone or something similar I came up with the idea to write a driver for the microphone, which would give me the ability to change my voice directly within it, but after some research it was clear to me that it can't be that easy.
Updates:
05.05.2010 - Microphone driver solution died. Researching for a new way...
Windows Mobile doesn't seem to have any influence on the microphone during a phone call. Deeper research about the hardware of the phone is necessary (processors, DSP/MDSP,....).
HELP IS APPRECIATED AND ANY SUGGESTIONS ARE WELCOME
I will update this thread as some news appear.
Please write your suggestions and your ideas directly as post in this thread.
Good idea !
good luck , i'll watch this thread
You'll get a better response in this forum...
http://forum.xda-developers.com/forumdisplay.php?f=523
You've posted in a HD2 specific forum, but that one is specific to Windows Mobile software development.
Device specific
The reason why I posted it here is that I am going to develop the driver on this device (HD2 Leo) and because the only solution so far i figured out is developing a driver, which is device specific.
good idea, i have been after an app similar to the "i am t-pain" (autotune)app from iphone
please can you use autotune?
God speed to you good sir!
pferdismus said:
The reason why I posted it here is that I am going to develop the driver on this device (HD2 Leo) and because the only solution so far i figured out is developing a driver, which is device specific.
Click to expand...
Click to collapse
Fair enough, but surely it wouldn't be a device specific driver to do this? Never mind.
This will be great for extortions. The question is whether we can find a way to disgiuse the phone's ID and location (which can be found using cellular triangulation) otherwise we have achieved nothing here.
lol this would be a fun one... hopefully you can work some magic and create a special treat for us hd2 users
Amirul0608 said:
good idea, i have been after an app similar to the "i am t-pain" (autotune)app from iphone
please can you use autotune?
Click to expand...
Click to collapse
Autotune is an app for iphone, which obviously can't help us here.
In addition autotune doesn't morph your voice during a phone call.
The main problem here is not the morphing itself (such apps already exist for WinMo), but the fact that it needs to be done while you make an active call.
pferdismus said:
Autotune is an app for iphone, which obviously can't help us here.
In addition autotune doesn't morph your voice during a phone call.
The main problem here is not the morphing itself (such apps already exist for WinMo), but the fact that it needs to be done while you make an active call.
Click to expand...
Click to collapse
Live morphing during a active phone call would be nice..
There is a software name Voice Twister for Windows mobile which can do live morphing but not during active phone calls..
Noam23 said:
This will be great for extortions. The question is whether we can find a way to disgiuse the phone's ID and location (which can be found using cellular triangulation) otherwise we have achieved nothing here.
Click to expand...
Click to collapse
We have a serious man here.
Hi all,
I'm a noob app developer trying to do it for the first time. I'm trying to create an event calendar app for company's internal purposes. So that everyone in the company (which uses a lot of android phones) can be updated on company events that are coming up.
My approach is to create a database with MySQL for events input, and to use PHP to connect the database which is in a server and to push it to the android devices that are used in the office as that's how I found it in a tutorial from HelloAndroid.
My questions are:
1. is this the best approach to create such app?
2. Can I use Android's default calendar to input these events? if not is there some kind of calendar template that I can use to display the information to the users.
Probably will have more questions as I progress, but please help if anyone is more experienced in these matters than I am.
Thank you very much, and really appreciate the inputs that I'm gonna get
A simple solution would be to use Google Calendar.
By using GC you can add/edit events either from a PC (work or home) or your Mobile anywhere in the World.
All that is required is for someone to first set up the calendar on Google and push it out to all those who you want to see it.
This way you are not isolating anyone who does not have an Android Phone.
that's a really good point. Especially in a closed environment like an office. I didn't think of that before. THank you very much.
However, if let's say I want to try to build an app for that for public use, not just confined to limited numbers of people in a company.
Would that be the best approach or if there's any better way to approach it. I may decide to build the app after all just to learn and gain experience in app development.
thanks again for the feedback, really appreciate it.
I can't really give you advice on developing an App sorry.
All I can say is the method I mention is the one which I use to keep all my Operational Team (Surgeons/Doctors/Nurses/Admin/etc) updated and advised.
There's nothing short in you developing your own Calendar App but you would have to ensure it would be available to all and that includes anyone anywhere regardless of the fact they are at work, home, holiday, with Android or not, have a Phone, PC, Mac, etc... etc...
For that there's already a stable tool by the name of Google Calendar.
If you do develop one I'd still be interested in seeing what ideas you could bring into practice that would have a benefit over GC.
I understand your point of view, and actually you got me thinking, maybe if I really want to do the apps, I can create an app that calls the Gcalendar events, and display it from the app. I can find out the API to do that, and that would be much simpler.
Thank you again for your input, really appreciate i
racdyn said:
I understand your point of view, and actually you got me thinking, maybe if I really want to do the apps, I can create an app that calls the Gcalendar events, and display it from the app. I can find out the API to do that, and that would be much simpler.
Thank you again for your input, really appreciate i
Click to expand...
Click to collapse
No problem.... I wish you well on your venture.
Do please let us all know how you get on.. You never know, if it becomes that good I may even use it myself over the stock calendar App or any of the other's that already link to GC.
Hi all,
This thread pertains to the inbuilt Internet Calling feature on Gingerbread. By default, it uses g.711 but I use an audio broadcast unit which requires g.722.
Is anyone able to help me, with finding out where the prefs are for the default SIP client, and how to enable g.722 as Mon codec de preference?
Many thanks
Perhaps I should post this in Dev...
Well, according to the rules, I'm not allowed to post a development based question in the dev forum. Apparently "If it's not about a ROM, it's not allowed in the development area" - perhaps the 'Dev' area should be renamed 'ROMs', and a separate Development Area should be opened?! That way, serious questions won't get overlooked and moved down so quick with the masses of repeated questions from new members.
Anyway - thats my rant...
Here's my actual question - reposted:
Hi folks
OK, this isn't ROM related as such, but it is definitely development related, hence posting here. I'm not a programmer, but I know my way around config files etc. usually. What I'm trying to find out, is this:
Can I adjust the 'built-in' VoIP Client to use a different codec? There's no options available in software, but the config for the client must be somewhere - question is, WHERE would it be? If anyone can point me in the right direction, I'll report my findings
There's a few VoIP things I want to customise, but using the native client, rather than a third party app.
I'm not after a spoon-fed solution - just the clues where to look so I can sort it out.
Many thanks
Just unzip any ROM and look inside,
That would spare you own phone and you can check every single file.
Btw, apk are zip files as well...
I've had a look around, but still haven't found the right things.
I guess the 'Internet Calling' function is part of the main 'phone' app, but I can't find any settings or config files for this.
It should be such a simple change to make; but I'm stuck at the first hurdle!!
Hi there,
I'm new to this android stuff, I use a... Uh, okay I use a different phone...
I'm blind and am exploring android with one goal... To see again. Okay, maybe that was a little over blown but let me explain before asking for some help from you lovely people.
There is an app called 'the vOICe', by seeing with sound, their home page here:
(Noticed I can't post links, do a search for seeing with sound and the voice and you'll find the one, it's also on the app market, just don't mistake it for the app accompanying the TV series...)
Who are using a system to help blind see. In a nutshell, it scans an image from left to right and layers sound using different frequencies to represent the image. I'm sorry if I'm a bit vague about this as I've not really had a chance to fully experiment with it. In short, it turns visual images into sound and it would appear that some fantastic results are coming from it and actual stimulation of the visual cortex is happening. In effect, the android device is the a to d, and the user's brain is the d to a. Amazing.
The problem is, currently it's not very usable. It can process a live stream from a video camera built into an android device but, of course, no one wants to wonder around with a 6 inch sudo-tablet strapped to their forehead. There was some excitement with the idea of google glass, however, aside from it not coming to light, I'd not really want to spend several hundred pounds on a device which I can use for just one thing. Okay, now the specifics...
I want to be able to route a webcam, or a similar camera into an android device, the device to accept the images from the camera and to process them into sound using the vOICe app. From what I understand from the Doctor who is pushing this forward and the inventor of the system, android does not yet allow external cameras to act as the native camera device. Can you help?
My perfect rig, and I'm making the huge assumption that it's the rig that most blind people would settle for, is the phone in my pocket, a hat or headband containing a camera, and a pair of headphones, preferably bone conduction as they give better awareness.
My question, my request is for someone to help make my device accept a video camera through usb and use the vOICe app to process the image.
Thank you in advance.
Olz
PS... I'm new to all of this, so I apologise if I've posted in the wrong place. If admins could move this post to the right place if I have messed up, I'd be very grateful.
O
Hi, Thanks for using XDA assist
There is a general forum for android here http://forum.xda-developers.com/android/help where you can try to ask to get expert's help.
Good luck.
So I am going to post this here, because the development section is ferboten to people trying to offer useful suggestions. Geez, I hope this is the right section.
Let me set it up. ANDROID IS BASED ON THE LINUX KERNEL. Everything, and I mean everything after that fact becomes android and bloat. Given this as the basis of all things android, I have a simple set of guidelines that should be used to create a solid, reliable, bullet proof operating system for devices able to use the android operating system.
#1. Create the kernel boot section of the basic platform that supports the very basic hardware features, including touch screen, radios, power/charging/battery management, wifi, bluetooth, nearfield, audio, microphone, s-pen,etc. Basics only. Root access is standard and can be turnd on/off just like developwer functions. No special tricks or addon hoops to jump through.
#2 At this point STOP. Every single process or service to be run on the device from this point forward should be able to be individually selectable - or not. Yes, I understand that individual services or apps may have dependancies to other processes and that thouse need to be functional in order for a particular app to work. That is why, each additional service or app must list the dependencies and in the selection process, the installion will be required to verify you have the proper services installed and functioning, if not to list them and allow you to make the decision to proceed. Viola, we have NO MORE BLOAT WARE.
#3 Make selecting additional services/apps selection process a menu driven, tag selectable process. Make the unselection process smart to verify and identify the other apps dependant on the item you are killing/removing.
#4 Allow a built in snapshot option to capture the entire system as you have customized it for yourself and allow it to be backed up to external memory with the ability to bring it back, AT WILL. With no big hassles.
Those 4 items are a good start. By themselves alone, it would put all of us in the drivers seat of controlling and living with our devices.
I am aware that such a system is not for everyone. It does require some basic technical understanding of the process. But for the vast majority of users, I am certain, that an a la carte system is far superior to the bloated monstrosities being forced down our devices.
I would appreciate any refinements to my suggestions.
The silence is deafening. It has been days. Kind of says something, don't you think?
I am so very happy (not) that these boards a compartmentalized to the point where you can't get through.
Pretty much a waste of my valuable time, especially considering the fact that if my suggestions where applied, people would be clamoring for an installation with those features. I guess there is not much true "developer" in xdadevelopers anymore.
Funny guys, with the pay to remove ads spam here. Really? Why? For what.
I leave here disappointed. Not angry. Just sad.
I'm not a developer so I have nothing valuable to contribute to your suggestions in your original post but I can offer an observation regarding your perception of the level of contribution to your thread and that is the fact that is particular area of the forums probably isn't the ideal spot to have a discussion such as this. You figured that out already so sure you can call me Capt. Obvious if you wish but I'm just pointing out that yes the forums are compartmentalized (as you've stated) in such a way that development for each device is separated out. This development is centered around AOSP based ROMs or "stock" ROMs (using manufacturer released source) so if you're looking to have higher level discussions about what AOSP should look like then deep down at this device specific level probably isn't where you want to be. So there's that . . . then there's the fact that for this particular device you can basically hear a pin drop in the sub-forums as it is since nothing much is going on by way of development.
Where specifically such a discussion would see more input I'm not sure, perhaps the main general discussion section?
Thank you for your sobering reply. I get frustrated. Already had my post bounced out of "wrong" forum while trying to speak directly to "developers" (i wanted air quotes because the term does not accuratly describe its target) So I tried this. The note pro 12.0 is still the best hardware giving the ipad a run for its money. The problem its monkeyshines kiddie software running it, or not running, or barely running it. ANDROID, indian word for crappy wannabe software, developed by clueless kids.
Again, thanks for the gentle nudge. I appreciate someone willing to conduct a conversation
Are you looking for an Ubuntu or SUSE type of setup?
I think what you are looking for is similar to apt-get type of installs, I could be wrong of course. This might be helpful for many "users". I think one of the barriers is that there are slight differences between the models of tablets, and creating the logic to put in for the sub-dependencies might prove problematic. You might get more answers to this possibility by asking one of the developers directly and sharing with them. Not a developer myself.