[Talk] Plugins Quarantine

[chrisglasier]

@chrisglasier said:

... one more on this point

@tig said:

So if some code contains the phrase Sketchup::Group you'd ban it - NO, because that occurs is many ...is_a?(...) test !

Thinks - if all were rejected it would force developers to use Modules. Module names can be duplicated if accommodated on the lines noted in the last post.

Or on second thoughts just check all is in Module; if not wrap it in one with plug-in name. Can it be that simple?

[Dan Rathbun]

Chris, I am sorry. None of your ideas are feasible.

Ruby is much different than JS.
Ruby is a dynamic extensible language, whose modules and classes are MEANT to be modified.
In fact, some of the Extended Ruby libraries modify or extend the base classes, to good effect.
The problem we have is people who do not understand Ruby very well (or do not understand the SketchUp's Ruby environment is shared,) are modifying or extending these classes to the detriment of everyone.
So we cannot freeze base classes permanently, otherwise plugins using some of the extended Ruby libraries will not work.

Also.. Ruby ITSELF has a set of modules, that contain sub-modules and classes that are considered part of the Ruby Core. Just because a class is wrapped within a module, does not make it OK.

TIG is correct, there are many ways in Ruby to change things, without using a class definition block.
ie:

Sketchup;;Group.class_eval {
  def my_funky_method()
    puts("Funky Man!")
  end
}

I CAN think of ways to test some plugins as they are loaded, but that would need to override the global require() and load() methods, which is exactly the kind of thing we are trying to prevent.

Agreed that an author can simply scramble his code or write a compiled C extension, to circumvent any "Ruby Police" utility we may write.

Sorry...

[Dan Rathbun]

It should also be noted that the new Trimble API Terms of Service prohibit what some of these bad scripts do:
http://www.sketchup.com/resources/api-terms-of-service.pdf

See Section 4 Prohibitions:

@unknownuser said:

4. You will not interfere with or disrupt the APIs or the servers or networks providing the APIs.

[chrisglasier]

@dan rathbun said:

Chris, I am sorry. None of your ideas are feasible.

Sorry...

No sorry as my Chinese friends say.

I do not have anything like the needed depth of knowledge of Ruby to react further but maybe an ignoramus's proposal may excite more people to question and explore new approaches.

Really my primary interest is exploring reloading imported files to accommodate duplication of names in environments other than html/javascript.

On your last post: As an American you probably understand Prohibition better than me!

[chrisglasier]

Now that you are awake I want to say that I find what appear to me to be contradictions in what Dan said but I don't have the wherewithal to take them up in a proactive way. For example he wrote:

@unknownuser said:

Ruby is a dynamic extensible language, whose modules and classes are MEANT to be modified.

But then warns of Trimble's prohibition

@unknownuser said:

4. You will not interfere with or disrupt the APIs or the servers or networks providing the APIs.

and

@unknownuser said:

So we cannot freeze base classes permanently, otherwise plugins using some of the extended Ruby libraries will not work.

If these are not contradictions what I am misunderstanding?

I would really like to see the discussion come to a conclusion/recommendation that can be understood by all ... or at least move in that direction.

[TIG]

No, Dan is right.
Ruby is a language designed to allow modifications and additions to all classes. and the overwriting of methods etc.
It's just that these changes are often going to break others' tools that anticipate that the API based classes/methods will work as they should.

If selfish/lazy authors publish such code it can be 'blacklisted' when it is discovered, but automatically detecting it as it loads is a whole other ball-game...

We have to trust 3rd-party scripts will do no harm [especially .rbs ones where we can't see the inner workings], but the new RBZ installer does warn users that what they are installing could cause problems...

Unlike Java/JS where actions can happen 'remotely' across the Internet etc and where therefore these languages are 'limited by design' in what they can do to your 'system'/'files', it is relatively simple to write a short Ruby that'd cause complete and utter havoc: for example it could auto-load then auto-run and then find all files in the Plugins and Tools folders etc and then erase them [including itself, to remove the evidence!] - but please don't try this at home !
Actually a few of my scripts already do this kind of thing already discreetly [!]. Say that some main-code files were updated and moved into a subfolder, having the older ones load from Plugins would cause problems - you can tell users to remove them on the reinstall, but some won't d things properly - so my tool auto-loads and auto-runs and if the now-unwanted-files are found in Plugins it deletes them, a dialog tells the user what's happened, and that they need to restart; it's a one-off operation because thereafter the files are not there to be found/erased! But... you see how unscrupulous authors could mess with your Sketchup installation to their own ends - especially with .rbs files !!

[thomthom]

It's not that JavaScript and Ruby is different in what it allow the developer to do. It's that Websites doesn't share their environment - each window or tab get a separate environment. Each WebDialog is the same.

If JS was picked in SketchUp as the API language you'd have the same problem of namespace issues. A function not wrapped in a namespace would conflict with another.

One could ask why each plugin doesn't get their won environment - but then consider how much overhead that would require to run a separate instance of Ruby for each and everyone of hundreds of plugins. Also - each plugin is accessing the same model entities - how could you place each plugin in a separate environment and still give access to the same model data? You'd get much of the same issues you get with multi-threading.

[chrisglasier]

How many Ruby plug-in users know about these unbelievable things you have noted especially:

@tig said:

We have to trust 3rd-party scripts will do no harm

it is relatively simple to write a short Ruby that'd cause complete and utter havoc

for example it could auto-load then auto-run and then find all files in the Plugins and Tools folders etc[!*] and then erase them [including itself, to remove the evidence!] - but please don't try this at home !

But... you see how unscrupulous authors could mess with your Sketchup installation to their own ends - especially with .rbs files !!

The only protection is Prohibition no 4?

*emphasised by me

[chrisglasier]

@thomthom said:

It's not that JavaScript and Ruby is different in what it allow the developer to do. It's that Websites doesn't share their environment - each window or tab get a separate environment. Each WebDialog is the same.

If JS was picked in SketchUp as the API language you'd have the same problem of namespace issues. A function not wrapped in a namespace would conflict with another.

One could ask why each plugin doesn't get their won environment - but then consider how much overhead that would require to run a separate instance of Ruby for each and everyone of hundreds of plugins. Also - each plugin is accessing the same model entities - how could you place each plugin in a separate environment and still give access to the same model data? You'd get much of the same issues you get with multi-threading.

I think there are things here that could be usefully discussed but for the moment I've been totally floored by TIG's exposure.

[Dan Rathbun]

@chrisglasier said:

Now that you are awake I want to say that I find what appear to me to be contradictions in what Dan said but I don't have the wherewithal to take them up in a proactive way. For example he wrote:

@unknownuser said:

Ruby is a dynamic extensible language, whose modules and classes are MEANT to be modified.

But then warns of Trimble's prohibition

@unknownuser said:

4. You will not interfere with or disrupt the APIs or the servers or networks providing the APIs.

and

@unknownuser said:

So we cannot freeze base classes permanently, otherwise plugins using some of the extended Ruby libraries will not work.

If these are not contradictions what I am misunderstanding?

In fact, they ARE! Much like most things in life.

A balance is required.

Keep in mind 99% of the SketchUp API is in OEM (Trimble) modules. Trimble could, at any time, elect to freeze all their modules and classes. (I'd hate personally to see it happen, as I like breadboarding API changes so I can file workable API feature requests.) However, even the freezing of module and class objects can be easily circumvented.

The SketchUp API leverages the extensibility of Ruby to extend a few of the base classes ( Numeric and Array, etc.,) but they KNEW what they were doing, and did it in a way that benefits ALL.

There is a big difference when a newbie hacks a base class to only benefit HIS plugin. We KNOW he is a newbie, otherwise he would have created his own subclass of Array, and modified that custom subclass for his own use, and would not have affected any one else's plugins.

So as ThomThom says, ... it's a matter of education first. (And yes there are education issues, now, that we have been wanting to solve on that front. But this is a separate although related issue.)
We only have a problem with a handfull of coders.

[Dan Rathbun]

What could be done NOW ??

• Start a Quarantine List - DONE (good idea Jim!)

• Create an "API Health Check Utility" - Some work done by TIG.

• Create a code validator / best practice / pitfall finder / absolute no-no checker that coders can run their code through.

• Create a run-time environment sentinel tester to see what scrambled or compiled C extensions do. (This cannot be run all the time because it itself must make changes to the Ruby environment. So it's only run during a test session, to test a plugin that's suspected of doing naughty things.) - I think ThomThom is playing with this now?

[Dan Rathbun]

Also I bring up an old suggestion, by one of our Pythonist members (Jesse James)

He suggested that unwrapped "macro" scripts, be autowrapped on loading within a outer module (I myself played with using the module names Macro or User::Macro,) and an inner module using the basename of the file, as it is done in Python.

So for instance.. a menu item loads a script named "cylinder.rb", but the code in it is unwrapped.
A module would be created for it as: User::Macro::Cylinder, and the code in the file would be evaluated within that module. (I also entertained the idea that a timer would be started, and the module unloaded if the user did not use it again before the timer ran out.)

However this only works for archaic macro-like linear scripts, not modern event driven plugins using observers that need to stay in memory.

[chrisglasier]

There is too much here that I feel uncomfortable with so I won't intrude any more .. but will still watch with interest.

[Trogluddite]

Hi all,
Firstly the disclaimer! - object oriented programming is for me just, err, dis-orientating! I don't understand it, and won't pretend to have any insight into the technical issues.
Second - thanks, Jim. For sure, there may be arguments about the implementation of "quarantining", but the idea of a central reference for downloads that might cause problems is a good one.
Here's my two'pennorth worth as a user of SU plugins, and as a creator of plugins of a very different kind (VST music/audio DSP).

• I agree with Chris that problem plugins should not be removed as a matter of course. It might be detrimental to future developments - i.e. where a plugin is a good idea or does clever geometric stuff, but has been made by a novice who could benefit from some help, or is willing to let another developer collaborate on their project. Forum users (or anyone for that matter) should be assumed to be intelligent enough to assess the risks and make their own informed choices (even if we know that some are not!). However, I would make an exception for posts where there is a consensus that the intention is purely malicious - e.g. the above "delete all my files" example.
• Moving such plugins to "quarantine" and linking from the original post is IMHO a good solution - unlike a simple note/icon etc., having to go to another location for the download means that the warning cannot easily be missed. As a novice developer, I would not be offended by this - I would have no desire to impact on the usefulness of SU for other people, and highlighting the problems in this way makes them more visible to other developers interested in helping to realise the OP's ideas. Any developer who gets stroppy about having such serious bugs pointed out to them obviously has no care for their user-base, and I would regard an angry response as little more than flaming!
• SU is useful in its "vanilla" installation, but it really is the scope of customisation with (mostly free!) plugins that makes it so powerful. Maybe this should be emphasised more when folks initially join the forum - i.e. the initial registration e-mail could contain links to, e.g. Dave's plugin installation tutorial, this quarantine list etc. and an explicit warning that plugins may interfere with each other. OK, it won't help with the idiots who don't RTFM, but even as as someone with previous plugin experience in various bits of software, it took me a while to suss out that "infectious" plugins might be the issue when I had the inevitable SketchyPhysics problems - it was just not something that I had ever had come across with e.g. VST, photoshop etc. plugins and scripts; usually just deleting the "broken" thing always worked, rather than looking for something that IS working but has ill-effects on something benign.
• It's been pointed out already that folks download SU plugins from other sources, thus missing any warnings. How about a download somewhere here (maybe top of "quarantine" thread), some simple software that would scan the plugins folder and report back any .rb files associated with known dodgy plugins. No file deletion or anything nasty like that, just a crude "virus scanner" for the plugin folder that users could run (hopefully before posting thier bug report!)

And to all the Ruby guru's who display such patience with the "how on earth do they manage to dress themselves" brigade - respect, and much sympathy!.
I dread making my VST plugins public - they can be used (theoretically!) in any of several dozen different host programs (more than any amateur dev' could ever possibly afford to own), each of which will look in an utterly different location for the 'plugin' folder, be compatible with a different update of the VST specification, have a different memory for legacy standards, which appeal to teenage egotists with pretensions of being the "next big thing" in the music industry (i.e. nothing is EVER their fault)...... Fun,fun,fun!

[thomthom]

Some plugin in my system adds a Geodesic under the SketchUp module. Not sure which one it is though.

[driven]

is it in

> rouge=%Q(Geodesic)
@rouge=rouge
path=%Q(/Library/Application\ Support/Google\ SketchUp\ 8/SketchUp)
@path=path
suspect=(%x(find "#{@path}" \\; | grep "#{@rouge}"))

=> /Library/Application Support/Google SketchUp 8/SketchUp/plugins!/Geodesic_SketchUp.rb
john

[TIG]

It's probably called something like 'Geodesic_SketchUp.rb' or similar by Gavin Kistner 2004.
It also messes with Group etc unnecessarily...
I have my own 'edited' version that avoids all of this...
I'll PM it to you...

[thomthom]

@tig said:

It's probably called something like 'Geodesic_SketchUp.rb' or similar by Gavin Kistner 2004.
It also messes with Group etc unnecessarily...
I have my own 'edited' version that avoids all of this...
I'll PM it to you...

Just peaked at the original file. Noticed the modifications to Group and Vector.

[Trogluddite]

@chrisglasier said:

One could ask why each plugin doesn't get their won environment - but then consider how much overhead that would require to run a separate instance of Ruby for each and everyone of hundreds of plugins. Also - each plugin is accessing the same model entities - how could you place each plugin in a separate environment and still give access to the same model data? You'd get much of the same issues you get with multi-threading

Re-reading this thread after encountering similar issues with another Ruby API entirely (FlowStone DSP programming software), which brought up this very problem of "sandboxing".

Simply, it is a non-starter in Ruby - even if it were desirable (careful what you wish for!), it couldn't be done. Ruby employs a 'Global Interpreter Lock' (GIL). In short, it is only possible to have one Ruby interpreter per process (the SU application being the 'process' in our case).
This is a decision made, and stuck with, by Matz who first created the language. The intention is that this ensures thread safety - and since all plugins must access the same model data, this is pretty important.
It's also why the API cannot access multiple processor cores - the interpreter runs in a single thread. Thus there is, on purpose, very little truly concurrent processing in Ruby - if there were, method calls could be processed "out of order", with the potential to seriously mess up any shared data.

From what I have seen here, it seems that the Ruby 'gurus' have found, and are actively promoting, the most effective and least intrusive solution to the problem. Over at the site for that other software, we are trying to build a similar framework of best practices - and the research we've done indicates that the techniques used here to manage the namespace are pretty much as good as it gets - given the limitations that using 'C' Ruby inherently imposes.

When writing a plugin, then as a developer, I would expect that my geometric calculations etc. were spot on so that the thing actually worked for its intended purpose - and I don't see how expecting the "housekeeping" parts of the plugin to be equally robust is an unreasonable responsibility. Not treading on the toes of the API or other plugins is part of the 'specification' for a good plugin, as important as getting lines and faces in the right place.

Maybe Ruby just makes it too easy? - folks using the more typical C SDKs of other applications have to work a lot harder to make robust add-ons, and things like thread safety etc. have to be considered and accommodated within their designs. Comparatively, wrapping everything in a module, and not messing with the kernel, is really not much to expect - anyone who can understand the API doc's should have no trouble understanding those things too.
The Ruby 'gurus' and mod's do a great job of advising us noob developers, but cannot force amateur dev's to heed their warnings. The techniques involved are not terribly complex (even I understand them!), and I have no problem with developers who consistently refuse to take good advice being censured for the ill effects that their plugins can cause.

[thomthom]

@trogluddite said:

Maybe Ruby just makes it too easy? - folks using the more typical C SDKs of other applications have to work a lot harder to make robust add-ons, and things like thread safety etc. have to be considered and accommodated within their designs. Comparatively, wrapping everything in a module, and not messing with the kernel, is really not much to expect - anyone who can understand the API doc's should have no trouble understanding those things too.

My impression is that many people, already used to Ruby from web development is used to have full control over their environment. (Even though they run into problems with namespaces if they use multiple frameworks.)

Yesterday I began to write down a checklist of things I've found important to keep in mind when developing SketchUp plugins: http://www.thomthom.net/thoughts/2013/02/sketchup-plugin-checklist/

What I missed when I began developing SketchUp plugins was that there was no firm guidelines. And the documentation even contains some very bad examples. I do hope this will change and I do think it will now with Trimble SketchUp.

It's something that is important when an API is developed - to provide good documentation with guidelines of good practices. Makes the developers using the API happy.