devicemap-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reza Naghibi <re...@apache.org>
Subject Re: DM effectiveness
Date Sat, 28 Mar 2015 20:06:04 GMT
Attached are the unknown (and generic) devices with counts. So as
Konstantin noted, these comprise a little less than 30% of the devices. The
first 40 entries in the attachment comprise around 13% of this latter set,
so by adding these 40 devices, we can increase the accuracy of the original
list to a tad under 85%. Adding the top 100 devices would bring that almost
to 90% accuracy.

Also, over 90% of these unknown devices are Android. Konstantin, is there
any value in that Android identification for you?

What device attributes are important here? Its worth noting that in 2.0, we
can still parse out the device name for unknown well formed android user
agents. But obviously we know nothing else about said devices.

Its also worth noting these are mostly Russian devices :)


On Sat, Mar 28, 2015 at 2:21 PM, Konstantin Papkovskiy <
konstantin@papkovskiy.com> wrote:

> I made the sample UA list from logs of our backend servers for mobile apps.
> Here is the list:
> https://www.dropbox.com/s/ne35o5etd7oj40f/ua_mobile_sample.csv.zip?dl=0
>
> You are welcome.
>
> On Sat, Mar 28, 2015 at 8:12 PM, Reza Naghibi <rezan@apache.org> wrote:
>
> > If you can make JIRA tickets for the missing devices, that would be
> great.
> > We still have one more 1.0.x release scheduled for the spring/summer.
> >
> > Also, yes, if you can upload the user-agent list you used somewhere, that
> > would be great. Where did you get these user agents from?
> >
> > thanks!
> >
> > On Sat, Mar 28, 2015 at 11:10 AM, Konstantin Papkovskiy <
> > konstantin@papkovskiy.com> wrote:
> >
> > > Hello all,
> > >
> > > I tested how effective DeviceMap in user device detection. I ran tests
> > on a
> > > dataset of 1M user agents (mostly mobile). Here are my results.
> > >
> > > Top 10 devices
> > > Device ID# of detections% of
> allgenericAndroid27784227.78%iPad648816.49%
> > > iPhone468034.68%NokiaN8-00345173.45%GT-I9300242152.42%unknown180231.80%
> > >
> >
> GT-I9100159501.60%GT-P3100147581.48%Nokia5800d140041.40%GT-P5100123321.23%
> > >
> > >
> > > DeviceMap data version: 1.0.2
> > > DeviceMap gem version: 0.1.1
> > > Number of user agents: 1 000 000
> > > Number of unique user agents: 18 743
> > > Number of successful detections: 981 977
> > > Detection rate: 98.20 %
> > > Number of unique successful detections: 1 027
> > > Detection rate (unique UA): 5.48 %
> > > *Detection rate (without genericAndroid): 70.42 %*
> > >
> > > As you can see, if I don't count "genericAndroid" and "unknown" then
> > > detection rate is about 70 %, which is rather low. If you add some of
> the
> > > most popular android UA to DM, its effectiveness will improve
> > drastically.
> > > But it will get progressively harder to increase this metric.
> > >
> > > I can provide the most popular UA from our logs if it will be useful.
> > >
> > >
> > > Regards
> > >
> > > Konstantin
> > >
> >
>

Mime
View raw message