Statistics on plugin users

jiminy-billy-bob

Hey guys,

I've been collecting some data from Layers Panel users. I thought some of you might find this interesting:
http://thomashauchecorne.com/layers/analytics.php

I started collecting since Aug 25, 2013. So you can only go up to that day.
It's kinda interesting to see the whole range. Especially seeing IE11 going up.
(Well, IE8 on the chart, as there was a bug in my PHP script. IE11 was saved as IE8 or IE7 before Jan 24)

Let me know what you think, and if you want me to collect other data.

jiminy-billy-bob

The chart showing the number of unique users per day is quite funny. You can see the weeks pattern, where the user count drops to almost a half during the weekends.

dacastror

very interesting!, could teach how to do this with php?
and show some code?

pgarmyn

@dacastror said:

very interesting!, could teach how to do this with php?
and show some code?

Yes, very usfulll.
It would even bee better to combine it with the Sketchucation pluginstore, to have statistics for all plugins !

cotty

@jiminy-billy-bob said:

The chart showing the number of unique users per day is quite funny. You can see the weeks pattern, where the user count drops to almost a half during the weekends.

... and the xmas gap

kaas

@jiminy-billy-bob said:

I started collecting since Aug 25, 2013. So you can only go up to that day.Let me know what you think, and if you want me to collect other data.

I can understand why this might be very useful for your coding but in my opinion you should have mentioned in your very first post of your plugin you were going to do this.

In my opinion: ask before you start collecting.

Greetings

jiminy-billy-bob

@dacastror said:

very interesting!, could teach how to do this with php?
and show some code?

It's quite simple. When the webdialog is ready, ruby tells it to open an iframe pointing to my server, with a url containing GET data (SU version, etc). A PHP script saves the GET data and some more it gathers from the browser itself. I'll show some code when I have time.

@pgarmyn said:

It would even bee better to combine it with the Sketchucation pluginstore, to have statistics for all plugins !

That would be great indeed.

@cotty said:

... and the xmas gap

Yeah ^^

@kaas said:

I can understand why this might be very useful for your coding but in my opinion you should have mentioned in your very first post of your plugin you were going to do this.

In my opinion: ask before you start collecting.

Greetings

I always did, since the beginning.
Warning here : http://sketchucation.com/forums/viewtopic.php?f=323%26amp;t=53906
Warning here : http://sketchucation.com/plugin/751-layers_panel
Warning here : http://extensions.sketchup.com/en/content/layers-panel-beta
And warning in the main .rb file.

kaas

@jiminy-billy-bob said:

I always did, since the beginning.
Warning here : http://sketchucation.com/forums/viewtopic.php?f=323%26amp;t=53906
Warning here : http://sketchucation.com/plugin/751-layers_panel
Warning here : http://extensions.sketchup.com/en/content/layers-panel-beta
And warning in the main .rb file.

My apologies. I missed those, my fault. I didn't expect this with plugins on Sketchucation. Lesson for me.

greetings

TIG

@kaas

This the only plugin I know of that collects info back to a server every time you use it.
But it does warn you in its notes and the data is anonymous.

The SketchUcation PluginStore usage also collects data about you - it just has to if you think about it...
It needs to know you are a member etc [using an encrypted cookie made when you login].
It also remembers all of the plugins you have downloaded - this is to give you structured data feedback in the PluginStore dialog.
When you use the PluginStore dialog it also sends data to the server about what plugins/extensions you currently have installed/enabled/disabled/uninstalled, and also what computer/sketchup-version you are using at that time.
This data is used to compile the individually tailored 'My Installs' and 'My Updates' lists, for you to use when managing your plugins installations... In the future this might be further enhanced - effectively becoming 'My Syncs' - where you could then keep the plugins/extensions on all of your computers/sketchup-versions 'in step'...

Many Plugins do not use webdialogs at all, so there is no easy chance for them to send data across the Internet.
For them to do this they'd need to open an 'off-screen dialog' to send the data - this is 'sneaky' ?

kaas

@TIG

My problem with handing over additional info in general is that privacy apparently is of no concern anymore to anyone. Firms, governments and some programmers apparently have no moral problem with collecting any data they can at any time. I really don't like this kind of thinking. If you need to know something for bug fixing or you are curious in general, just ask or start a poll but don't just take it and bury it in (lots of) disclaimers nobody reads anyway.

I realize the PluginStore has to send some data about the plugins I use to its server and I don't have a problem with that because, I thought, it's only info about the plugin, SU version and I can choose when to start it.

greetings

kaas

Talking about gathering people's data on a whole different scale... last week three firms with in total 160 computer shops all over the Netherlands were exposed tracking smartphone mac addresses, wifi, Bluetooth and infrared data. They collected data of people entering their shops and of every passer by.

Their response: we need this info for optimizing planning the amount of assistants in shops.

I don't get why at no point no-one didn't think this was sneaky and not the way to go...

Rich O Brien

We know when you download something.

We know what version of SketchUp you use.

We know your computer ID for displaying per device installs.

We know what you rated.

We know when you login.

We don't know when you turn on the tool. Apache server does keeps logs (up to 4 weeks) of activity but we don't use this for data collection. If we do need to use the Apache logs it is only for debugging an error. Which is rarely done.

So in terms of data collection the My profile tab is displaying the majority of the info collected.

If we introduced something that tracks usage it would be done in a fashion that Plugin Authors would need to inject code into their plugins to make use of it. I see no benefit in us knowing when you use TIG's Mirror tool when you are on holidays in Barbados.

I do see a benefit in providing Plugin Authors info on usage especially when that is used to further the development of a tool as in Jiminy's example earlier. But ideally that side of things should be developed within the spirit Plugin ecosystem that evolved here.

We could develop our API to allow cool things to happen based on the info about each plugin uploaded to the store. Try this for instance...

apicall=String.new("http://plugin.sketchucation.com/api.php?")
apicall.concat("APIVERSION=1")
apicall.concat("&APIKEY=test")
apicall.concat("&CALLINGPLUGIN=testWidget")
apicall.concat("&COMMAND=get_download_count")
apicall.concat("&PLUGIN_ID=Layers_Panel")
dlg = UI;;WebDialog.new("SketchUcation API", true,
   "SketchUcationAPI", 100, 100, 150, 150, true);
 dlg.set_url apicall
 dlg.show
 dlg.add_action_callback("api_reply") {|dialog, params|
   UI.messagebox  params.to_s
 }

...it returns the download count of Jiminy's Layers Panel tool. Pretty useless huh?

But the possibilities are pretty immense if we could get Authors to adopt to a set convention.

jiminy-billy-bob

@kaas said:

My problem with handing over additional info in general is that privacy apparently is of no concern anymore to anyone. Firms, governments and some programmers apparently have no moral problem with collecting any data they can at any time. I really don't like this kind of thinking. If you need to know something for bug fixing or you are curious in general, just ask or start a poll but don't just take it and bury it in (lots of) disclaimers nobody reads anyway.

In Layers Panel, it is completely anonymous. I can't track a user in any way. I can't tell if the user has check in yesterday, or the day before, when he started using the plugin, who he is, how many devices he uses, etc.
It's simply anonymous. How is this violating privacy?
I am concerned about privacy, that's why I don't want a more accurate tool to track users.

I'm sorry if you don't read disclaimers. But you should take care of warnings when using any tool, especially one you don't know.

kaas

@jiminy-billy-bob said:

In Layers Panel, it is completely anonymous. I can't track a user in any way. I can't tell if the user has check in yesterday, or the day before, when he started using the plugin, who he is, how many devices he uses, etc.
It's simply anonymous. How is this violating privacy?
I am concerned about privacy, that's why I don't want a more accurate tool to track users.

I'm sorry if you don't read disclaimers. But you should take care of warnings when using any tool, especially one you don't know.

Hello Jiminy-billy-bob,

I'm just fed up in general with third parties collecting bits and pieces of information of everyone's private life. Even if it just ends up in one big anonymous bin of data - it's still private data that is collected and if I could have it my way I would like to keep it private and use the plugin but be able to opt out.

Greetings

Aerilius

I think we also have to distinguish two points:
• an application/mobile app/plugin sends data to its own server with the user's consent
• an application/mobile app/plugin does this in a way that only its own server receives the data (ie. using a secure connection/encryption)

I'd happily allow the first point if I know and trust the plugin author, however the second point suggests that it's better to send data sparingly, only if there is a benefit.

I had the idea "what if my plugin launcher [LaunchUp] also searches online sources or submits calculations to wolfram alpha etc.". That would be cool, but it would broadcast what people do in their SketchUp, ie. Select, PushPull, CurviLoft, Select, Delete…

Garry K

I have just released a plugin for circular stair cases. The coding and design of the plugin was and is a huge undertaking and will consume a lot of time.

I have been working on the Web portion of my plugin for the last month. Most of my time has been spent to ensure a high level of security.

This plugin checks my web site on first time use in Sketchup session to see if the user is in a trial, is licensed, not registered etc.

I am totally up front with this. Currently you can only obtain my StairMaker plugin by going to my website and downloading it.

Hopefully in the future I will be able to evolve the process and to be able to use the plugin store. Currently I do not know what the process is. All I know is it should simplify updates.

I would love to learn, follow and to push a set of plugin standards. The only part I currently know is what I've learned from reading everything on Dan Rathburn's suggested reading list when building plugins.

Where can I go to find a set of plugin standards?

Now for the second part on sites secretly collected data. At a previous job we had 100's of reports and really did not know which ones were used, which ones could be consolidated and which ones could be dropped. So we stared to log report usage. We were able to analyse the information which allowed us to drop 25% of the existing (almost legacy) reports which were still being maintained. With 60% other reports we were able to consolidate.

jiminy-billy-bob

@kaas said:

I'm just fed up in general with third parties collecting bits and pieces of information of everyone's private life. Even if it just ends up in one big anonymous bin of data - it's still private data that is collected and if I could have it my way I would like to keep it private and use the plugin but be able to opt out.

I'm concerned by how privacy is violated sometimes, without people knowing. But let's not be too paranoid about this.

What I do is like you walk down a street, and there are people surveying what you're wearing and on what side you're walking, but not your identity or anything that could identify you. The next day it's different people surveying so they can't recognize you. On top of that, there's a sign at each end of the street warning you what's going on.
These people compile this data and say, "ok we have 5% red shirts and 67% pants in this street".
Would you feel they violate your privacy?
I know I wouldn't, especially if that could improve my comfort in this street.

It's give and take. My plugin is free, you can use it as much as you want, but you're willing to give me that little bit of anonymous data, so I can improve it.
I don't ask for anything besides that, and in the end it benefits the users...
You're warned about this, you can always choose not to use the plugin.

tt_su

@garry k said:

Where can I go to find a set of plugin standards?

The Extension Warehouse has a set of technical requirements which is based on patterns to avoid plugins clashing. These requirements are parts of what we consider good practice:
http://extensions.sketchup.com/developer

A couple of years ago I wrote a short article with some key things to remember:
http://www.thomthom.net/thoughts/2012/01/golden-rules-of-sketchup-plugin-development/

Followed up by this check-list:
http://www.thomthom.net/thoughts/2013/02/sketchup-plugin-checklist/

(Edit: did you mean standards in terms of security when transferring data over the internet?)

Garry K

Thomas,

I did mean standards. To protect data and web site.
BTW - I have now read all the articles.
I believe that my plugins are mostly in compliance but I did come up with 2 possibly 3 issues.

My current documentation for DoorMaker and StairMaker is in pdf and is part of the install. I'm seeing that you want an external URL for documentation. Is there a spot in the Sketchup Extensions to provide the URL directly to help Page? Or is it sufficient to provide URL of the Home page?
I see what you are talking about as far as not using Sketchup.find_support_file, currently I am. shouldn't be too tough to change.
I don't believe that I am using typename - but I'll check this as well.

Garry K

This probably should be moved to another thread.

I think my new plugin is ok for testing. The DoorMaker plugin has been around for a while but there was 1 global that I had to change. Currently there have been 327 downloads of the DoorMaker - and 1 complaint.

No longer using Sketchup.find_support_file.
Users just create a loader.rb file with a require_all and put into real plugins folder
I only put a single rb file into plugins folder. This file is prefaced with GKWare_
I create a GKWare folder for all of my plugins.
I create a main module GKWare and for each plugin create a module under that
module StairMaker, module DoorMaker
My plugins are not dependent on each other nor on anyone elses
No globals
No modifying or extending base classes
I do not use typename
Units work properly - based on model
I work with the Sketchup.Extensions - one entry per plugin
Performance is fast.
I use as few string manipulations as possible
package is rbz

It isn't perfect - and there are situations where bugs creep in - mostly from unrealistic edge cases. I do trap for a few obvious ones.