Unzipping archive from Ruby (Mac and Windows)

eneroth3 · 4 Dec 2014, 16:32

TIG: I experimented a little with Zlib before but I couldn't find a way to save multiple files in the same archive which I need for my plugin, it doesn't seem to be supported.

TIG · 4 Dec 2014, 16:36

Mine only loads zlib.so and requires the ruby zip.rb stuff if it's not already been installed.
So if another plugin already installed it it uses that.
You can of course so it entirely in vbs/mac-shell for all versions....
These are all inside begin/rescue/ensure sections...
` require('zlib') unless defined?(Zlib)

in case it is already loaded, and in case it fails it's in a 'rescue'`

To allow my require('zip') to work I briefly add the path to the plugins own subfolder into $: thus:
` ...
rubyzip=File.join(DATA, 'Rubyzip')

where DATA is a constant referring to that plugin's subfolder named 'Rubyzip'

$: << rubyzip
...
require('zip') unless defined?(Zip)

in case not already installed by another plugin...

...
$:.delete(rubyzip)

to tidy up, ad it's no longer needed...

...`

The Rubyzip methods allow you to add multiple files into a ZIP file - in fact you can do most ZIP/unZIP things with it in conjunction with zlib...

TIG · 4 Dec 2014, 16:49

Using Rubyzip...

zipped = Zip;;File.open(zip, Zip;;File;;CREATE) do |zipfile|
  # - The name of the file as it will appear in the archive
  # - The original file, including the path to find it
  zipfile.add(skp, tempskp)
  zipfile.add(txt, temptxt)
end

This opens a ZIP file, whose full-path ref is 'zip' [in this case a new one].
It then adds a file to that ZIP, whose full-path ref is in 'tempskp', and which will appear in the ZIP as the reference 'skp' - in this case it's the SKP's name...
It then adds another file to the ZIP, whose full-path ref is in 'temptxt', and which will appear in the ZIP as the reference 'txt' - in this case it's the TXT file's name...
You can also add subfolders etc as desired - read the Rubyzip usage notes...

You don't need to install the 'gem' for the users - just look how I am pre-including them in ../PluginsZipUp/Data/Rubyzip ... with the RBZ installer...

driven · 4 Dec 2014, 17:03

@tig
how do you 'test' a zipped folder with Rubyzip?

for example with unzip you have the -t flag...

      def is_texture(file)
        %x( unzip -t "#{file}"  ).include? "ref\/.*g" # file = path to skm and ref/ means it has a texture folder
      end

TIG · 4 Dec 2014, 17:32

Yours is the simple MAC way !
You can find/read the contents of an existing ZIP for a match in this kind of example...

Zip;;File.open(path_to_some_foo_zip') do |zip_file|
  # Handle entries one by one
  zip_file.each do |entry|
    # Extract to file/directory/symlink
    puts "Extracting #{entry.name}"
    entry.extract(dest_file)
    # Read into memory
    content = entry.get_input_stream.read
  end
  # Find specific entry
  entry = zip_file.glob('*.csv').first
  puts entry.get_input_stream.read
end

You need to set up your reference variables...

eneroth3 · 4 Dec 2014, 19:43

These examples use rubyzip but from a subdirectory rather than a Gem. Is it possible to only use the Zlib without rubyzip to put multiple files in the same zip archive?

I use these archives to store information for what can be drawn with my plugin. In my previous plugin I stored railroad tracks in a similar way but in their own folders instead of archives. Each folder contained a few .skp models for things to draw when drawing the track, e.g profiles, a preview image for web dialogs and a text file containing some information dynamically displayed in the web dialogs without loading the models and littering the definition list.

In this project I want to store all that data in one single file to make it easier for users to exchange them. I've already managed to do that but I don't know what's considered best practice, installing a Gem or copy the code into my own plugin's folder.

driven · 4 Dec 2014, 19:59

have you tried this...

system( 'ROBOCOPY', files_directory, new_zip_folder_path )

if it works [I don't have a PC to test], The mac version is almost the same...

system('zip', '-r', new_zip_folder_path,  files_directory)

john

TIG · 4 Dec 2014, 22:18

Your MAC ' zip' example is much like I use on older SketchUp version MACs with no Ruby zip built-in [with -q ?].

Your ' ROBOCOPY' examples just makes a copy of the files in the source-folder in a new folder named ' xxxx.zip' - not zipped ! and it flashes up a black cmd window - just like in any WIN system call.

On PCs you can make a longer set of strings that are VB commands. and put them into a ' xxxx.vbs' file which then runs silently using UI.openURL("file:///#{path_to_vbs_file}"), and it will make a proper ZIP file... I use that in older SketchUp version PCs with no Ruby zip built-in.

txt="Const FOF_CREATEPROGRESSDLG = &H0&
Const MyZip = \"#{zip.tr("/","\\")}\"
Const File1 = \"#{tskp1.tr("/","\\")}\"
Const File2 = \"#{tskp2.tr("/","\\")}\"
'Create the basis of new zip file
CreateObject(\"Scripting.FileSystemObject\").CreateTextFile(MyZip, True).Write \"PK\" & Chr(5) & Chr(6) & String(18, vbNullChar)
'get ready to add files to zip
With CreateObject(\"Shell.Application\")
'add files
.NameSpace(MyZip).CopyHere File1, FOF_CREATEPROGRESSDLG
wScript.Sleep 200
.NameSpace(MyZip).CopyHere File2
End With
' wait 3 secs, to let it finish...
wScript.Sleep 3000
'''

Where ' zip' is the path to the new ZIP file, and ' tskp1' etc paths to SKP files - the tr makes / into \ for VBS use
You can get it to write a temp txt file at the end so you know it's done...

fredo6 · 4 Dec 2014, 23:03

Yes, I noticed there is a Zip module that seems to ship in Ruby 2.0 in the standard Sketchup installation.

I did not find the documentation however, but this could be the answer, at least for SU2014 and SU2015.

Fredo

driven · 4 Dec 2014, 23:11

http://ruby-doc.org/stdlib-2.0/libdoc/zlib/rdoc/Zlib.html

tt_su · 5 Dec 2014, 13:03

@eneroth3 said:

Thanks! The gem rubyzip gem does exactly what I wanted .

However I don't know what's the best practice to use a gem. Should I ask the user to install it or should I check if it exists when the plugin loads and otherwise install it? Should I in that case use statusbar texts to tell the user the gem is being installed and that it may take some time?

What I've done is wrap the require in a begin/rescue that catch LoadErrors (you don't want to catch other errors) and then use Gem.install to install it if it's missing.

tt_su · 5 Dec 2014, 13:04

@fredo6 said:

Yes, I noticed there is a Zip module that seems to ship in Ruby 2.0 in the standard Sketchup installation.

I did not find the documentation however, but this could be the answer, at least for SU2014 and SU2015.

Fredo

It doesn't support ZIP files, it for general compression and GZ files.

driven · 5 Dec 2014, 13:12

cheers for the explanation...

as I follow up to your last reply, this works on mac...

      def is_texture(file)
          result = false
          Zip;;InputStream;;open(file) {|io|
            while (entry = io.get_next_entry)
              result = true if entry.to_s.include? "ref\/"
            end
          result
          }
      end

could you test on Win with a skm path as file...
john
EDIT: added result = false although it is working...

fredo6 · 5 Dec 2014, 15:00

@driven said:

http://ruby-doc.org/stdlib-2.0/libdoc/zlib/rdoc/Zlib.html

This is ZLib. Is the Zip module just a wrapper of it?

Fredo

Dan Rathbun · 5 Dec 2014, 16:25

@driven said:

could you test on Win with a skm path as file...
john

John the following is what I will have used in my SKM Import Library:
(Yes it works on Windows and should also on Mac.)

<span class="syntaxdefault">  def skm_has_texture</span><span class="syntaxkeyword">?(</span><span class="syntaxdefault">file_path</span><span class="syntaxkeyword">)<br /></span><span class="syntaxdefault">    </span><span class="syntaxcomment"># validate file_path here<br /></span><span class="syntaxdefault">    bool </span><span class="syntaxkeyword">=</span><span class="syntaxdefault"> nil<br />    Zip</span><span class="syntaxkeyword">;;</span><span class="syntaxdefault">File</span><span class="syntaxkeyword">.</span><span class="syntaxdefault">open</span><span class="syntaxkeyword">(</span><span class="syntaxdefault">file_path</span><span class="syntaxkeyword">)</span><span class="syntaxdefault"> do </span><span class="syntaxkeyword">|</span><span class="syntaxdefault">skm</span><span class="syntaxkeyword">|<br /></span><span class="syntaxdefault">      tag </span><span class="syntaxkeyword">=</span><span class="syntaxdefault"> skm</span><span class="syntaxkeyword">.</span><span class="syntaxdefault">read</span><span class="syntaxkeyword">(</span><span class="syntaxstring">'document.xml'</span><span class="syntaxkeyword">).</span><span class="syntaxdefault">slice</span><span class="syntaxkeyword">(/<</span><span class="syntaxdefault">mat</span><span class="syntaxkeyword">;</span><span class="syntaxdefault">material</span><span class="syntaxkeyword">\</span><span class="syntaxdefault">s</span><span class="syntaxkeyword">(.*)(\</span><span class="syntaxdefault">s</span><span class="syntaxkeyword">*>|\</span><span class="syntaxdefault">s</span><span class="syntaxkeyword">*\/>)/)<br /></span><span class="syntaxdefault">      unless tag</span><span class="syntaxkeyword">.</span><span class="syntaxdefault">nil</span><span class="syntaxkeyword">?<br /></span><span class="syntaxdefault">        bool </span><span class="syntaxkeyword">=(</span><span class="syntaxdefault"> tag </span><span class="syntaxkeyword">!~</span><span class="syntaxdefault"> </span><span class="syntaxkeyword">/(</span><span class="syntaxdefault">hasTexture</span><span class="syntaxkeyword">=\</span><span class="syntaxstring">"0\")/ rescue false )<br />      else<br />        puts("</span><span class="syntaxdefault">XML Read Error</span><span class="syntaxkeyword">;</span><span class="syntaxdefault"> Could not extract </span><span class="syntaxkeyword">\</span><span class="syntaxstring">"mat;material\" element.>"</span><span class="syntaxkeyword">)<br /></span><span class="syntaxdefault">      end<br />    end </span><span class="syntaxcomment"># open skmfile<br /></span><span class="syntaxdefault">    </span><span class="syntaxcomment">#<br /></span><span class="syntaxdefault">  rescue </span><span class="syntaxkeyword">=></span><span class="syntaxdefault"> e<br />    puts</span><span class="syntaxkeyword">(</span><span class="syntaxstring">"Error; #<#{e.class.name}; #{e.message}.>"</span><span class="syntaxkeyword">)<br /></span><span class="syntaxdefault">    return nil<br />  else<br />    return bool<br />  end </span><span class="syntaxcomment"># skm_has_texture?()<br />&nbsp;</span><span class="syntaxdefault"></span>

driven · 5 Dec 2014, 17:35

@dan
yes, that works on mac,
I'm want to test if reading the stream works as it is meant to be the closest to unzip -t...
avoiding reading the file is said to be more efficient...

does my example even run on your PC?
john

Dan Rathbun · 5 Dec 2014, 18:11

I had not tried it yet.

IMHO, your current code iterates all of the entries, and I think the test weak on the existence of the "ref" sub-directory. I decided myself not to rely upon that.

Reading a small text (xml) file should be relatively fast. It is not like this will be done thousands of times in a modeling operation. The only delays I notice are the output of console debug information.

driven · 5 Dec 2014, 18:30

@dan rathbun said:

It is not like this will be done thousands of times in a modeling operation.

I'm using it to add icons to batches of skm's, possibly hundreds at a time...
I want a quick check before unzipping, as I need to use different file for color v material...
john

Dan Rathbun · 5 Dec 2014, 18:40

This works on Windows:

<span class="syntaxdefault">def is_texture</span><span class="syntaxkeyword">(</span><span class="syntaxdefault"> file</span><span class="syntaxkeyword">=</span><span class="syntaxdefault">UI</span><span class="syntaxkeyword">.</span><span class="syntaxdefault">openpanel</span><span class="syntaxkeyword">()</span><span class="syntaxdefault"> </span><span class="syntaxkeyword">)<br /></span><span class="syntaxdefault">  result </span><span class="syntaxkeyword">=</span><span class="syntaxdefault"> false<br />  Zip</span><span class="syntaxkeyword">;;</span><span class="syntaxdefault">InputStream</span><span class="syntaxkeyword">;;</span><span class="syntaxdefault">open</span><span class="syntaxkeyword">(</span><span class="syntaxdefault">file</span><span class="syntaxkeyword">)</span><span class="syntaxdefault"> </span><span class="syntaxkeyword">{|</span><span class="syntaxdefault">io</span><span class="syntaxkeyword">|<br /></span><span class="syntaxdefault">    while </span><span class="syntaxkeyword">(</span><span class="syntaxdefault">entry </span><span class="syntaxkeyword">=</span><span class="syntaxdefault"> io</span><span class="syntaxkeyword">.</span><span class="syntaxdefault">get_next_entry</span><span class="syntaxkeyword">)<br /></span><span class="syntaxdefault">      if entry</span><span class="syntaxkeyword">.</span><span class="syntaxdefault">to_s</span><span class="syntaxkeyword">.include?</span><span class="syntaxdefault"> </span><span class="syntaxstring">"ref\/"<br /></span><span class="syntaxdefault">        result </span><span class="syntaxkeyword">=</span><span class="syntaxdefault"> true<br />        break<br />      end<br />    end<br />  </span><span class="syntaxkeyword">}<br /></span><span class="syntaxdefault">  puts </span><span class="syntaxstring">"\"#{file}\"\n  is_texture; #{result.inspect}"</span><span class="syntaxdefault"> </span><span class="syntaxcomment"># debug<br /></span><span class="syntaxdefault">  return result<br />end<br /></span>

Just added a break inside the while loop, and moved the return result outside the block (because the block always returns the io object accd'g to the docs.)

Set the default arg to UI.openpanel() just so I could pick SKMs for test.

eneroth3 · 5 Dec 2014, 21:03

@tt_su said:

@eneroth3 said:

Thanks! The gem rubyzip gem does exactly what I wanted .

However I don't know what's the best practice to use a gem. Should I ask the user to install it or should I check if it exists when the plugin loads and otherwise install it? Should I in that case use statusbar texts to tell the user the gem is being installed and that it may take some time?

What I've done is wrap the require in a begin/rescue that catch LoadErrors (you don't want to catch other errors) and then use Gem.install to install it if it's missing.

What do you think is the best implementation? What about showing a web dialog or UI.messagebox when the plugin loads and the gem isn't installed telling the user that the gem is required? The user can choose to do it later (next time plugin is loaded) and the extension wont load or choose to download it now and there's a message saying it may take some time. The user would also be informed that this only has to be done once (per user and computer). Of course the extension warehouse info page would say that the extension will ask the user to install the gem and then automatically do it.