Identify new trackers #40
Comments
U039b
added
enhancement
|
To add a new tracker, follow this schema of description: |
AppLovin
Avocarrot
NativeX
|
|
trying to untangle Baidu location tracking... the maps and location are so closely related in the code I've seen. Baidu Maps
WeChat Location
|
|
A quick note - older versions of Tune tracker use the com.mobileapptracker name for the SDK. Tune
|
|
Updates to SafeGraph. Much more detail at https://github.com/YalePrivacyLab/tracker-profiles/blob/master/trackers/SafeGraph.md SafeGraph OpenLocate
|
HyperTrack
|
Uber Analytics
|
Lisnr
|
SilverPush
|
Shopkick
|
Alphonso
|
Smaato
|
Scandit
|
|
we need to decide how to handle / untangle Google Maps and Location services as well. At the least, the presence of the location services listener should be considered a tracker. Google Maps
Google Location Service
|
Inrix
|
Signal360
|
|
Signal360 use the Manchester decoder for logic 1s and 0s. This is probably similar methodology for other audio beacon companies. |
|
thanks for the heads up, I'm sure you're right about this being the most common method. Some of these audio beacons use amplitude, but that's very limited (FidZup's method, if we trust the patent applications). Most seem to use frequency and what they call "frequency shift keying", which is slight changes in frequencies for 0s and 1s. Hypothetically, they could do much more frequency shifts within that 18kHz to 20kHz range (LISNR claims up to 22kHz but I don't know of devices that have that capability), and then they could do hex or the alphabet even. What's unclear to me is how they have enough bandwidth to get complex data across the wire... the amount of time that someone is in proximity to a speaker with their microphone could be very limited. |
|
one technique is this: so if the sdk process hears the carrier frequency it can then start listening for the repeated modulated signals, create a historical cache of recorded signals and then process them for any candidates. |
|
we should talk off-thread, but that's potentially ~24KB per minute at most? something like that? |
Byyd (Adfonic)
Mixpanel
Phunware
Gimbal
|
Google Usage Stats
|
|
I just came across Segment (https://segment.com), a tracker that happens to be integrated into Mattermost, a self-hostable chat platform that is very popular now in the FLOSS community. One of their client, whose use of the data looks the most cynical to me: https://segment.com/customers/xo-group They do seem to be collecting data from Android as well: https://segment.com/docs/sources/mobile/android/. Interestingly enough, their Android client/library (I am not sure what I am talking about) seems to be open source. |
|
Thanks. We do have Segment listed as a tracker in Exodus, but it would be great if you could provide more detail in this thread so that we can fill out the tracker profile more completely. Try to take a look at some of the more detailed profiles above, or the ones we did at https://github.com/YalePrivacyLab/tracker-profiles |
NewRelic
|
|
Changes to Signal 360: Signal360
|
|
thanks. right now I'm going through that uniq_list file and removing obfuscated portions, google and android classes, and some things that look generally innocuous. Kinda interesting. Finding some stuff I hadn't seen before. |
|
I just went through and added/updated all of the ones I'd collected info for. Do you all expect to reanalyze the apps for any new trackers that've been identified? And does the progress bar beside each tracker on the https://etip.exodus-privacy.eu.org site mean it won't be ready until 100% completed? Is someone going through our entries and making fixes? Like, I know some of the gradle entries I put are probably not always going to be a particular version number and some domains are randomly generated (ex: 234234135.mobileapptracking.com or whatever) so I didn't know if they get a better rule written. |
Opentracker
|
|
Not sure which of these count as "trackers" (so please don't just copy them over unverified), but all of the below fall into the category "mobile analytics": Codahale Metrics
Microsoft Azure Analytics
Parse.com
Splunk MINT
FlowUp
Keen Java Clients
|
|
Are you still accepting submissions here for new trackers? |
|
@kaputnikGo with the issue not closed, I assumed so 🙀 |
|
I would just ask @uo39b for access to their etip website. I moved all of my submissions into it directly. But it doesn't look like any I did submit were ever officially added which stinks because I have several hundred more I could likely find in here :/ https://raw.githubusercontent.com/jawz101/MobileAdTrackers/master/hosts |
|
@IzzySoft @kaputnikGo There's a way to get an account on that site by emailing him here |
|
Exodus Privacy has new leadership and may just not be aware of this github issue. I can send them upstream via https://etip.exodus-privacy.eu.org but you're right, let's figure out a workflow that works for everyone. Thanks all! |
|
also, you can just put these in our YalePrivacyLab repo for tracker profiles, where we're also gathering new info... I will invite all of you as contributors. |
|
@seandiggity I'm not sure if I'll report trackers regularly – but sure it's good to know where to put them, and they hopefully will make their way into Exodus. You also can find my full library list (which not only contains trackers, but all kinds of libraries used in Android apps) in my GitLab repo if you're interested. Further I'm not sure if I can provide full descriptions as you keep them in your repo. Is it OK to commit partially filled samples? Do you want them submitted directly to your repo, or via PRs? |
|
Hi all! We invite you to share/sync trackers info between ETIP and the Yale Privacy lab repo. Cheers! |
|
Thanks for the heads-up, @U039b! Waiting for advice concerning "incomplete records" (I can't provide full ones as I've got no idea how to fill the gaps – especially network signature, Maven specifics and gradle; I'm not a dev) and "distribution guidelines". If that's permitted, I'll accept the invitation and share my findings. Speaking of which: are there any issues with your scanner currently? For several hours now I'm always told to come back later as the queue is filled. Something hanging? |
|
@IzzySoft it seems that some tasks are stuck in the queue, we will investigate ;-) Regarding ETIP fields, |
|
@IzzySoft no more "come back later" ;-) |
that was my assumtion, too.
@U039b just noticed – thanks a lot! 👍 👏 🕺 🤸♂️
Yes, so far I got. But it's domain names the corresponding tracker contacts, right? I've got no idea how to figure that. I'm just performing a basic static analysis of path names on the Smali, which is how I found some hundred libraries – those above trackers among them. So if I want someone else (here: you) to fill the gaps, you'd need a sample? Or could I simply skip this as well?
That's good to know! Maybe it would be a good idea to have a simple tutorial on the other repo, for folks like me who know enough to contribute but not enough to make "complete" submissions? |
You are welcome!
Domain names correspond to the remote servers contacted by the trackers to send collected data. You can find them by analyzing the network traffic of an application which uses a given tracker or by inspecting the binary looking for URLs or domains.
Unfortunately, I am a bit busy. Anyway, once you have listed path names (you probably mean Java packages, you will find mode details here) you have to check what packages correspond to a tracker. Then, you can create a new one in ETIP and provide information you have gathered about the tracker.
It would be nice to have tutorials for ETIP and Exodus-core, unfortunately, I do not have time :-/ But anybody can create a tutorial and we will be happy to put it at the right place ;-) |
|
Thanks @U039b – and I know exactly what you mean by "not enough time", as that's my situation, too … And yes, that's what I meant by "static analysis" – though I use a different tool for it. |
|
Added basic tracker submission template and first example to the Yale repo with the intention of enabling a quick and easy way to get proper new tracker info into Exodus - https://github.com/YalePrivacyLab/tracker-profiles |
|
That Taplytics profile looks great, thanks. Will make sure these go upstream, so if it's lower barrier-to-entry to submit to the YPL repo that's fine (then there's less reason to bother EP and @U039b for Etip accounts etc. as well). |
|
Added 8 more taken from here, will keep using the commit summary with "basic tracker" to help ID when they go up in this format. fyi i check the yale tracker list, https://reports.exodus-privacy.eu.org/trackers/ and https://etip.exodus-privacy.eu.org/ before adding them, so hopefully that covers everything. |
|
@U039b In the past, sometimes when I entered in network signatures and gradle string, I didn't really know the regex pattern to use. Some had version numbers in the gradle files I could find so I didn't know how you put those in to scan on your end. Are you all doing any fixes to our submissions when you see them like this? example deltaDNA: And from their network traffic they follow this pattern: but I just entered @IzzySoft as for the extra fields such as This is where I just start googling for their developer documentation. That's where it takes a little research. Say, for deltaDNA I would search Google for blahblah sdk. deltaDNA sdk And then I click around until I find the android documentation and search for words like has a little thingy at the bottom that says maven and shows and a second tab called gradle that shows It was just all guesses to see if I could find what the etip site was looking for and all of these 3rd party companies seem to have these sorts of steps in their documentation to integrate ads & analytics |
Right, that should. We'll be adding quite a few more to the YPL repo as part of the crowdsourcing I'm doing via Mozilla Open Leaders project. |
|
Over the past couple of weeks I've done quite a bit of work on the etip site. Filled in a lot of blanks on existing signatures and added maybe 20-40... I can't tell. Anyways, has anyone from the project taken a look at them? I'd like to fix my mistakes if I did anything incorrect. My main concerns are the format of the regex on the network signatures as well as what we do if the build.gradle entries could have versions. like if com.example.sdk.1.2.3 is what we find, that would assume there are other versions, so would we not use a regular expression to look for the consistent information? Additionally, do the scans just try to look for at least one of these identifiying bits or if all of the characteristics are there (code signature, network signature, maven & gradle info must all be found or at least one of them must be found?) The reason I ask is because I went into existing entries and added maven repository information if I could find it but it looks like some tracker sdk's give instructions to proguard their code so I wonder if it may mean Exodus may never see that information on which to detect them, thus adding that information may break the detection rule if Exodus identifies a tracker only if all identifying bits are present. Also, is there a preference to which repository to which volunteers should contribute: etip vs. YPL? It seems like doing things twice. |
|
fwiw, I've kept adding more and occasionally looked at existing entries. For example, Unity Ads is likely underreporting. After reviewing their developer documentation, a code signature of com.unity3d.ads would just pick up their legacy sdk version. Their newer sdk would be com.unity3d.services. Now I'm taking a look at some of the apps exodus lists as having no trackers and finding ones missed :P I don't know how to represent some of the situations as there are a lot of companies in the business of sdk's to manage an app's other trackers. As an added bonus, doing so also results in more international companies being found. Actually, this might be a good practice moving fwd as this filter of No Tracker apps should almost be a representation of "clean" apps, which actually makes it a pretty compelling set of apps to review. |
|
Hi @jawz101 !
Thanks a lot for your work (and sorry for the late reply), it is greatly appreciated by the Exodus Privacy team :). Unfortunately we have not got the time recently to look through the new entries in ETIP and import the data from ETIP to exodus. We plan to work on this and will try to find some time in the coming weeks but this is quite a tedious task.
As it is explained on this page, what we look for is the signature of the tracker. So AFAIK the maven & gradle information will not affect the tracker identification. Cheers ! |
We are now closing this issue. If you want an ETIP account, feel free to send an email to [email protected] specifying your desired username + email address and we will send you a temporary password. Once registered, you will be able to freely contribute to the tracker identification process. Thanks again to everyone contributing to the tracker identification :). |
In https://reports.exodus-privacy.eu.org/reports/37/:
com/applovin/adview/AppLovinInterstitialAdDialogcom/avocarrot/sdk/nativeassets/model/NativeAdDatacom/appnext/ads/com/inlocomedia/android/ads/AdTypecom/moat/analytics/mobile/aol/NativeVideoTrackercom/mopub/common/GpsHelpercom/nativex/monetization/mraid/objects/CurrentPositioncom/unity3d/ads/android/UnityAdscom/vungle/publisher/AdConfigcom/youappi/ai/sdk/YouAPPiWhy the fuck this application requires
org/apache/commons/math3/optimization?The text was updated successfully, but these errors were encountered: